Survey on Efficient Large Language Models: Principles, Algorithms, Applications, and Open Issues.
Problem Statement
As LLMs grow in parameter count and complexity, inference cost, latency, and memory footprint have become major barriers to real-world deployment, especially on constrained hardware or at high serving throughput. The literature on efficiency is fragmented across compression, architecture design, compilers, and hardware co-design, so it is hard to compare methods or understand how they interact. This survey tries to consolidate that landscape and clarify which techniques fit which stage of development and deployment.
Key Novelty
- A new taxonomy that places model compression (quantization, pruning, distillation) alongside system-level methods (efficient architectures, compilation, hardware-aware optimization) in one framework
- A lifecycle-oriented analysis of how efficiency techniques interact with pre-training, fine-tuning, and serving, rather than treating each method in isolation
- A synthesis of application domains for efficient LLMs and a structured discussion of open problems and emerging trends
Evaluation Highlights
- Qualitative comparative analysis of inference optimization families across principles, algorithms, and deployment contexts (no new experimental benchmarks reported in the abstract)
- Coverage of real-world applications and practical deployment considerations, used to derive guidance for researchers and practitioners
Signal Assessment
Methodology
- Introduce foundational concepts of LLM inference cost: compute, memory bandwidth, latency, and throughput
- Propose a taxonomy grouping methods into quantization, pruning, knowledge distillation, efficient architectures, compilation, and hardware-aware techniques
- Map each technique onto the LLM lifecycle (training, fine-tuning, serving) and analyze how the techniques interact
- Review application domains where efficient LLMs are deployed
- Identify open issues and future research directions in efficient LLM systems
System Components
Reduces the numerical precision of weights, activations, and KV cache (e.g., INT8/INT4, post-training vs. quantization-aware) to cut memory and speed up inference
Removes redundant weights, heads, or layers (unstructured, semi-structured, or structured sparsity) to shrink compute and model size
Transfers capabilities from a large teacher LLM to a smaller student model to preserve performance at lower cost
Architectural changes such as efficient attention variants, mixture-of-experts, and linear or state-space alternatives that lower inference complexity
Graph-level and kernel-level optimizations (operator fusion, scheduling, code generation) that map models efficiently onto hardware
Co-design of algorithms with accelerator characteristics (memory hierarchy, specialized units) to maximize real-world throughput and latency gains
Analysis of where each technique applies across training, fine-tuning, and serving, and how techniques compose
Results
| Metric/Benchmark | Baseline | This Paper | Delta |
|---|---|---|---|
| Taxonomy scope | Prior surveys often focus mainly on model compression | Unified taxonomy spanning compression, architecture, compilation, and hardware | Broader, system-level coverage |
| Analysis perspective | Technique-by-technique catalogs | Lifecycle-oriented view (training, fine-tuning, serving) | Clarifies how techniques interact |
| Practical guidance | Limited deployment-oriented recommendations | Applications and open issues with actionable insights | Qualitative improvement in practitioner utility |
| Empirical results | N/A | No new experiments reported in abstract | N/A |
Key Takeaways
- Treat efficiency as a full-stack problem: algorithmic compression gains (e.g., low-bit quantization, sparsity) translate into real speedups only with compiler and hardware support
- Choose techniques by lifecycle stage. Post-training quantization and pruning suit serving-time optimization without retraining, while distillation and quantization-aware or architecture changes require investment during training or fine-tuning
- Use the taxonomy as a checklist when designing deployment pipelines for LLM agents or multimodal systems, and consider how methods stack (e.g., quantization plus efficient attention plus optimized kernels) instead of applying them in isolation
Abstract
With the rapid advancement of large language models (LLMs) in both academia and industry, their growing size and complexity have introduced significant challenges in terms of computational cost and deployment efficiency. To address these issues, a wide range of inference optimization techniques-including but not limited to model compression-have been proposed to accelerate LLM inference while preserving model performance. This survey provides a comprehensive overview of LLM inference acceleration strategies, analyzing them from multiple perspectives, including foundational principles, algorithmic techniques, real-world applications, and open research challenges. We begin by introducing core concepts underlying inference optimization and propose a new taxonomy that categorizes existing approaches, including quantization, pruning, distillation, efficient architectures, compilation, and hardware-aware methods. Following the lifecycle of LLM development and deployment, we examine how these techniques interact with model training, fine-tuning, and serving. Furthermore, we highlight key applications of efficient LLMs and discuss emerging trends and unresolved issues in the field. By synthesizing recent advances, this survey aims to provide actionable insights and practical guidance for researchers and practitioners working with scalable and efficient LLM systems.