← Back to Paper Radar

Survey on Efficient Large Language Models: Principles, Algorithms, Applications, and Open Issues.

Jian Cheng, Haidong Kang, Yuxin Shao, Nan Li, Pengjun Chen, Rui Wang, Saiqin Long, Xiaochun Yang, Lianbo Ma
IEEE Transactions on Neural Networks and Learning Systems | 2025
This survey organizes LLM inference acceleration techniques (quantization, pruning, distillation, efficient architectures, compilation, and hardware-aware methods) into a unified taxonomy. It analyzes them across the LLM lifecycle of training, fine-tuning, and serving to give practitioners actionable guidance for deploying scalable, efficient LLMs.

Problem Statement

As LLMs grow in parameter count and complexity, inference cost, latency, and memory footprint have become major barriers to real-world deployment, especially on constrained hardware or at high serving throughput. The literature on efficiency is fragmented across compression, architecture design, compilers, and hardware co-design, so it is hard to compare methods or understand how they interact. This survey tries to consolidate that landscape and clarify which techniques fit which stage of development and deployment.

Key Novelty

  • A new taxonomy that places model compression (quantization, pruning, distillation) alongside system-level methods (efficient architectures, compilation, hardware-aware optimization) in one framework
  • A lifecycle-oriented analysis of how efficiency techniques interact with pre-training, fine-tuning, and serving, rather than treating each method in isolation
  • A synthesis of application domains for efficient LLMs and a structured discussion of open problems and emerging trends

Evaluation Highlights

  • Qualitative comparative analysis of inference optimization families across principles, algorithms, and deployment contexts (no new experimental benchmarks reported in the abstract)
  • Coverage of real-world applications and practical deployment considerations, used to derive guidance for researchers and practitioners

Signal Assessment

3/10 As a survey, it adds no new algorithms or empirical results; its value lies in organization and synthesis. The lifecycle framing and combined compression-plus-systems taxonomy are useful, but the area is already covered by several surveys of LLM efficiency and compression.

Methodology

  1. Introduce foundational concepts of LLM inference cost: compute, memory bandwidth, latency, and throughput
  2. Propose a taxonomy grouping methods into quantization, pruning, knowledge distillation, efficient architectures, compilation, and hardware-aware techniques
  3. Map each technique onto the LLM lifecycle (training, fine-tuning, serving) and analyze how the techniques interact
  4. Review application domains where efficient LLMs are deployed
  5. Identify open issues and future research directions in efficient LLM systems

System Components

Quantization

Reduces the numerical precision of weights, activations, and KV cache (e.g., INT8/INT4, post-training vs. quantization-aware) to cut memory and speed up inference

Pruning

Removes redundant weights, heads, or layers (unstructured, semi-structured, or structured sparsity) to shrink compute and model size

Knowledge Distillation

Transfers capabilities from a large teacher LLM to a smaller student model to preserve performance at lower cost

Efficient Architectures

Architectural changes such as efficient attention variants, mixture-of-experts, and linear or state-space alternatives that lower inference complexity

Compilation

Graph-level and kernel-level optimizations (operator fusion, scheduling, code generation) that map models efficiently onto hardware

Hardware-Aware Methods

Co-design of algorithms with accelerator characteristics (memory hierarchy, specialized units) to maximize real-world throughput and latency gains

Lifecycle Integration

Analysis of where each technique applies across training, fine-tuning, and serving, and how techniques compose

Results

Metric/Benchmark Baseline This Paper Delta
Taxonomy scope Prior surveys often focus mainly on model compression Unified taxonomy spanning compression, architecture, compilation, and hardware Broader, system-level coverage
Analysis perspective Technique-by-technique catalogs Lifecycle-oriented view (training, fine-tuning, serving) Clarifies how techniques interact
Practical guidance Limited deployment-oriented recommendations Applications and open issues with actionable insights Qualitative improvement in practitioner utility
Empirical results N/A No new experiments reported in abstract N/A

Key Takeaways

  • Treat efficiency as a full-stack problem: algorithmic compression gains (e.g., low-bit quantization, sparsity) translate into real speedups only with compiler and hardware support
  • Choose techniques by lifecycle stage. Post-training quantization and pruning suit serving-time optimization without retraining, while distillation and quantization-aware or architecture changes require investment during training or fine-tuning
  • Use the taxonomy as a checklist when designing deployment pipelines for LLM agents or multimodal systems, and consider how methods stack (e.g., quantization plus efficient attention plus optimized kernels) instead of applying them in isolation

Abstract

With the rapid advancement of large language models (LLMs) in both academia and industry, their growing size and complexity have introduced significant challenges in terms of computational cost and deployment efficiency. To address these issues, a wide range of inference optimization techniques-including but not limited to model compression-have been proposed to accelerate LLM inference while preserving model performance. This survey provides a comprehensive overview of LLM inference acceleration strategies, analyzing them from multiple perspectives, including foundational principles, algorithmic techniques, real-world applications, and open research challenges. We begin by introducing core concepts underlying inference optimization and propose a new taxonomy that categorizes existing approaches, including quantization, pruning, distillation, efficient architectures, compilation, and hardware-aware methods. Following the lifecycle of LLM development and deployment, we examine how these techniques interact with model training, fine-tuning, and serving. Furthermore, we highlight key applications of efficient LLMs and discuss emerging trends and unresolved issues in the field. By synthesizing recent advances, this survey aims to provide actionable insights and practical guidance for researchers and practitioners working with scalable and efficient LLM systems.

Generated automatically from available metadata and abstract on 2026-11-09.