Dynamic Token Pruning

Last Updated 11 August, 2026

by David Spuler, Ph.D.

Dynamic Token Pruning: Book Excerpts and Blog Articles

Free online book excerpts with full text chapters online and free PDF downloads, and the Aussie AI blog, including related articles:

Token compression is an LLM inference optimization that reduces an input text sequence to a shorter sequence of text or tokens. There are various methods that can be used before sending the tokens to the LLM, or during the LLM inference processes. Efficiency of inference is improved when there are fewer tokens for the LLM to process... more about Token Compression »
Token folding is an LLM inference optimization that "folds" or merges two or more tokens together. This is a type of "token reduction" that results in fewer overall tokens for the LLM to process. Folding of tokens can occur in prompt preprocessing or dynamically during inference computations ... more about Token folding »
Token dropping is an LLM optimization that reduces token processing by "dropping" some of them. It is similar to "token pruning", but often refers to dropping tokens during the training phase, whereas token pruning is mostly an inference optimization ... more about Token dropping »
Token fusion is an LLM inference optimization that fuses two or more input tokens together. This results in fewer tokens for the LLM to process. Static token fusion occurs prior to LLM processing (during prompt preprocessing), whereas dynamic token fusion occurs during inference (when important tokens can be identified) ... more about Token fusion »
Tokenization: The tokenizer does not receive as much attention in the research literature as other parts of large language models. This is probably because the tokenization phase itself is not a bottleneck in either inference or training, when compared to the many layers of multiplication operations on weights. However, the choice of the tokenizer algorithm, and the resulting size of the vocabulary, has a direct impact on the speed (latency) of model inference ... more about Tokenization »
Token merging is an LLM inference optimization that merges two or more adjacent tokens into a single token. This optimization reduces the number of tokens for a GPU to process, thereby directly improving speed, and reducing compute. Token merging is a type of token reduction, and is closely related to token pruning, which is simply removing tokens from processing. Merging of multiple tokens together can be performed before sending the prompt to the LLM, or dynamically during inference based on analysis of which tokens are important ... more about Token merging »
Token pruning is a type of model "length pruning" that aims to address the cost of processing input sequences and the related embeddings. It is closely related to "embeddings pruning", but is orthogonal to depth pruning (e.g. layer pruning) or width pruning (e.g. attention head pruning) ... more about Token pruning »
Token reduction is an LLM inference optimization method that aims to speed up LLMs by reducing the number of tokens processed. This is relevant to any inference process, but is particularly applicable to optimizing multi-step reasoning algorithms, such as Chain-of-Thought, because their interim reasoning steps generate long sequences of tokens. ... more about Token reduction »
Token selection is the final phase of LLM processing, also called the "decoding algorithm", whereby the output token is chosen. There are numerous variants including greedy decoding, top-k decoding, top-p decoding, and many more. Most models work by outputting one token at a time, but newer models use "multi-token prediction" (MTP) or "multi-token decoding" to output two or more tokens in parallel... more about Token selection »
Token skipping is an LLM inference optimization methods that "skips" the processing of some input tokens. Fewer total tokens to process means less GPU compute cost and faster inference. Variants of this technique include token pruning, token dropping, and token merging ... more about Token skipping »
Vision attention optimization is a speed optimization for the inference in Vision Transformers (ViTs). This uses techniques similar to image attention optimization, such as pruning or merging patches or sections of the images ... more about Vision attention optimization »
Vision token merging is the combining of two or more areas of an image for faster AI processing. Vision models tokenize the images into segments, which must each be processed by the LLM. Merging them together allows the model to process only a single area, with reduced accuracy ... more about Vision token merging »
Vision token pruning is the removal and avoidance of processing portions of an image or video stream. The idea generalizes token pruning in text models, which remove words, to avoid processing some segments of visual images. This is an effective speedup for image processing or machine vision because many of the areas of a large image are not important, and their ... more about Vision token pruning »
Vision token reduction is an LLM inference optimization technique for vision models that involves pruning or merging tokens. Tokenization of vision data is based on image tokenization, and can be optimized in various ways. Vision token pruning is avoiding the processing of redundant or unimportant segments of images, such as recurring blank background regions. ... more about Vision token reduction »
Salient tokens are an LLM optimization strategy that focuses inference cost on important or "salient" tokens. Unimportant or non-salient tokens can be pruned to reduce overall computations to focus on the salient ones. The strategy of pruning non-salient tokens can be applied to the regular inference tokens, or the same strategy can be applied in the KV cache, or both.... more about Salient tokens »
Prompt compression is the shortening of LLM prompts automatically so that they contain fewer tokens and can be processed more efficiently. This isn't used on short user queries, but there are many other parts of a prompt that are longer, such as the entire conversational history, retrieved documents, or other types of "context" for the prompt. For this reason, this technique is often called "context compression" or "token reduction".... more about Prompt compression »
Length pruning is weight pruning on one of the three axes of pruning. The other two axes are width pruning (e.g. attention head pruning) and depth pruning (e.g. layer pruning and early exit). All three types of pruning are mostly orthogonal to each other and can be combined into triple pruning. The main types of length pruning along the "lengthwise" dimension of the inputs are... more about Length pruning »
Context compression is an LLM inference optimization that reduces processing of tokens in the context of a query. It is a type of "prompt compression" that involves aspects of techniques such as token pruning or token merging ... more about Context compression »
Input compression is an LLM inference optimization that reduces an input text sequence to a shorter sequence of text or tokens for the LLM to process faster. LLM inference efficiency is improved if there are fewer tokens for the LLM to process. ... more about Input compression »
David Spuler, Ph.D., Feb 6th, 2026 (updated), 500+ LLM Inference Optimization Techniques, Aussie AI Blog, https://www.aussieai.com/blog/llm-inference-optimization
David Spuler, March 2024, Chapter 46. Structured Pruning, in book "Generative AI in C++", https://www.aussieai.com/book/ch46-structured-pruning
David Spuler, March 2024, Chapter 49. Length Pruning, in book "Generative AI in C++", https://www.aussieai.com/book/ch49-length-pruning
David Spuler, March 2024, Generative AI in C++: Coding Transformers and LLMs, https://www.aussieai.com/book/toc PDF: https://www.aussieai.com/pdf/BOOK-Generative-AI-CPP-Spuler-2024.pdf

Research on Dynamic Token Pruning

Research papers include:

Jiaao Li, Kaiyuan Li, Chen Gao, Yong Li, Xinlei Chen, 21 Jul 2025, EgoPrune: Efficient Token Pruning for Egomotion Video Reasoning in Embodied Agent, https://arxiv.org/abs/2507.15428
Junwei Luo, Yingying Zhang, Xue Yang, Kang Wu, Qi Zhu, Lei Liang, Jingdong Chen, Yansheng Li, 24 Jul 2025, When Large Vision-Language Model Meets Large Remote Sensing Imagery: Coarse-to-Fine Text-Guided Token Pruning, https://arxiv.org/abs/2503.07588
Jiajun Cao, Qizhe Zhang, Peidong Jia, Xuhui Zhao, Bo Lan, Xiaoan Zhang, Xiaobao Wei, Sixiang Chen, Zhuo Li, Yang Wang, Liyun Li, Xianming Liu, Ming Lu, Shanghang Zhang, 31 Jul 2025, FastDriveVLA: Efficient End-to-End Driving via Plug-and-Play Reconstruction-based Token Pruning, https://arxiv.org/abs/2507.23318
Taehan Lee, Hyukjun Lee, 3 Aug 2025, Token Pruning in Audio Transformers: Optimizing Performance and Decoding Patch Importance, https://arxiv.org/abs/2504.01690
Wenhui Zhu, Xiwen Chen, Zhipeng Wang, Shao Tang, Sayan Ghosh, Xuanzhao Dong, Rajat Koner, Yalin Wang, 16 Aug 2025, EVTP-IVS: Effective Visual Token Pruning For Unifying Instruction Visual Segmentation In Multi-Modal Large Language Models, https://arxiv.org/abs/2508.11886
Kaiyuan Li and Xiaoyue Chen and Chen Gao and Yong Li and Xinlei Chen, 23 Oct 2025, Balanced Token Pruning: Accelerating Vision Language Models Beyond Local Optimization, https://arxiv.org/abs/2505.22038
Pu Zhang, Yuwei Li, Xingyuan Xian, Guoming Tang, 20 Oct 2025, ZSPAPrune: Zero-Shot Prompt-Aware Token Pruning for Vision-Language Models, https://arxiv.org/abs/2510.17197
Ye Li, Yuan Meng, Zewen Sun, Kangye Ji, Chen Tang, Jiajun Fan, Xinzhu Ma, Shutao Xia, Zhi Wang, Wenwu Zhu, 3 Oct 2025, SP-VLA: A Joint Model Scheduling and Token Pruning Approach for VLA Model Acceleration, https://arxiv.org/abs/2506.12723
Atul Shree, Harshith Jupuru, 10 Oct 2025, FLToP CTC: Frame-Level Token Pruning via Relative Threshold for Efficient and Memory-Saving Decoding on Diverse Platforms, https://arxiv.org/abs/2510.09085
Jaemin Son, Sujin Choi, Inyong Yun, 8 Sep 2025, Index-Preserving Lightweight Token Pruning for Efficient Document Understanding in Vision-Language Models, https://arxiv.org/abs/2509.06415
Eugene Kwek and Wenpeng Yin, 8 Sep 2025, COMPACT: Common-token Optimized Model Pruning Across Channels and Tokens, https://arxiv.org/abs/2509.06836
Wenda Qin, Andrea Burns, Bryan A. Plummer, Margrit Betke, 18 Sep 2025, Walk and Read Less: Improving the Efficiency of Vision-and-Language Navigation via Tuning-Free Multimodal Token Pruning, https://arxiv.org/abs/2509.15250
Yicheng Ji, Jun Zhang, Heming Xia, Jinpeng Chen, Lidan Shou, Gang Chen, Huan Li, 22 Aug 2025, SpecVLM: Enhancing Speculative Decoding of Video LLMs via Verifier-Guided Token Pruning, https://arxiv.org/abs/2508.16201
Zicong Tang, Ziyang Ma, Suqing Wang, Zuchao Li, Lefei Zhang, Hai Zhao, Yun Li, Qianren Wang, 24 Aug 2025, CoViPAL: Layer-wise Contextualized Visual Token Pruning for Large Vision-Language Models, https://arxiv.org/abs/2508.17243
Yiwu Zhong, Zhuoming Liu, Yin Li, Liwei Wang, 29 Jul 2025, AIM: Adaptive Inference of Multi-Modal LLMs via Token Merging and Pruning, https://arxiv.org/abs/2412.03248
Ao Li, Yuxiang Duan, Jinghui Zhang, Congbo Ma, Yutong Xie, Gustavo Carneiro, Mohammad Yaqub, Hu Wang, 28 Jul 2025, TransPrune: Token Transition Pruning for Efficient Large Vision-Language Model, https://arxiv.org/abs/2507.20630
Wenhao Zeng, Yaoning Wang, Chao Hu, Yuling Shi, Chengcheng Wan, Hongyu Zhang, Xiaodong Gu, 8 Aug 2025, Pruning the Unsurprising: Efficient Code Reasoning via First-Token Surprisal, https://arxiv.org/abs/2508.05988
Benjamin Minixhofer, Edoardo Maria Ponti, Ivan Vulić, 13 May 2024, Zero-Shot Tokenizer Transfer, https://arxiv.org/abs/2405.07883 (Overcoming the limitation that the tokenizer is fixed for the model, by training the tokenizer to embeddings mapping so as to use different tokenizers, including effective input token pruning reducing tokens in the input with a larger vocabulary.)
Yubin Qin; Yang Wang; Dazheng Deng; Xiaolong Yang, Ayaka: A Versatile Transformer Accelerator With Low-Rank Estimation and Heterogeneous Dataflow, https://ieeexplore.ieee.org/abstract/document/10530252 (Cross layer random prediction to allow sparsification of attention and linear layers.)
Yao Fu, 14 May 2024, Challenges in Deploying Long-Context Transformers: A Theoretical Peak Performance Analysis, https://arxiv.org/abs/2405.08944
Zhihang Lin, Mingbao Lin, Luxi Lin, Rongrong Ji, 9 May 2024, Boosting Multimodal Large Language Models with Visual Tokens Withdrawal for Rapid Inference, https://arxiv.org/abs/2405.05803 Code: https://github.com/lzhxmu/VTW (Removing all visual tokens in later layers of a multimodal model, which is effectively early exiting or token pruning, but affecting only the vision part of the multimodal Transformer.)
Shujian Zhang, Korawat Tanwisuth, Chengyue Gong, Pengcheng He, Mingyuan Zhou, 7 May 2024, Switchable Decision: Dynamic Neural Generation Networks, https://arxiv.org/abs/2405.04513 (Switching and skipping sub-layer components such as attention heads, FFNs, or input token skipping, using decisions made based on allocating computation resources.)
Zixuan Zhou, Xuefei Ning, Ke Hong, Tianyu Fu, Jiaming Xu, Shiyao Li, Yuming Lou, Luning Wang, Zhihang Yuan, Xiuhong Li, Shengen Yan, Guohao Dai, Xiao-Ping Zhang, Yuhan Dong, Yu Wang, 22 Apr 2024, A Survey on Efficient Inference for Large Language Models, https://arxiv.org/abs/2404.14294
Yashas Samaga B L, Varun Yerram, Chong You, Srinadh Bhojanapalli, Sanjiv Kumar, Prateek Jain, Praneeth Netrapalli, 14 Feb 2024, HiRE: High Recall Approximate Top-k Estimation for Efficient LLM Inference, https://arxiv.org/abs/2402.09360 (Attempts to estimate the output of top-k decoding, so as to prune computations on two dimensions earlier in the inference computations.)
Ignacio de Gregorio, April 2024, Mixture-of-Depths, a Dazzling New AI Breakthrough: Conditional Computing is Finally Here, Medium, https://medium.com/@ignacio.de.gregorio.noblejas/mixture-of-depths-a-dazzling-new-ai-breakthrough-be958fc629b2 (Mixture of depths is a layer-wise per-token limit to attention head computations, which is like width pruning with dynamic depth.)
David Raposo, Sam Ritter, Blake Richards, Timothy Lillicrap, Peter Conway Humphreys, Adam Santoro, 2 Apr 2024, Mixture-of-Depths: Dynamically allocating compute in transformer-based language models, https://arxiv.org/abs/2404.02258 (Per-layer pruning of which tokens can be in the attention computations to give a type of mixed lengthwise pruning combined with a dynamic width pruning or slimmable network approach.)
Hou-I Liu, Marco Galindo, Hongxia Xie, Lai-Kuan Wong, Hong-Han Shuai, Yung-Yui Li, Wen-Huang Cheng, 8 Apr 2024, Lightweight Deep Learning for Resource-Constrained Environments: A Survey, https://arxiv.org/abs/2404.07236 (A survey of various optimizations, with a lot of focus on image and vision models, including CNNs, RNNs, and Transformers.)
M Sponner, B Waschneck, A Kumar , 2024, Adapting Neural Networks at Runtime: Current Trends in At-Runtime Optimizations for Deep Learning, ACM Computing Surveys,, PDF: https://dl.acm.org/doi/pdf/10.1145/3657283 (Survey of various adaptive inference optimization techniques with much focus on image and video processing optimization for LLMs.)
Wangbo Zhao, Jiasheng Tang, Yizeng Han, Yibing Song, Kai Wang, Gao Huang, Fan Wang, Yang You, 18 Mar 2024, Dynamic Tuning Towards Parameter and Inference Efficiency for ViT Adaptation, https://arxiv.org/abs/2403.11808 (PEFT and adaptive inference and token pruning in Vision Transformers.)
Yansong Xu, Dongxu Lyu, Zhenyu Li, Zilong Wang, Yuzhou Chen, Gang Wang, Zhican Wang, Haomin Li, Guanghui He, 16 Mar 2024, DEFA: Efficient Deformable Attention Acceleration via Pruning-Assisted Grid-Sampling and Multi-Scale Parallel Processing, https://arxiv.org/abs/2403.10913 (Attention optimizations in Vision Transformer with pruning of feature maps, and extensive parallelization with consideration of the hardware layer.)
Yuzhang Shang, Mu Cai, Bingxin Xu, Yong Jae Lee, Yan Yan, 25 Mar 2024, LLaVA-PruMerge: Adaptive Token Reduction for Efficient Large Multimodal Models, https://arxiv.org/abs/2403.15388 Code: https://llava-prumerge.github.io/ (Compresses input images based on redundant sections.)
Tanvir Mahmud, Burhaneddin Yaman, Chun-Hao Liu, Diana Marculescu, 24 Mar 2024, PaPr: Training-Free One-Step Patch Pruning with Lightweight ConvNets for Faster Inference, https://arxiv.org/abs/2403.16020 (Pruning of patches in input images, which is a form of token or channel pruning.)
Mengwei Xu, Wangsong Yin, Dongqi Cai, Rongjie Yi, Daliang Xu, Qipeng Wang, Bingyang Wu, Yihao Zhao, Chen Yang, Shihe Wang, Qiyang Zhang, Zhenyan Lu, Li Zhang, Shangguang Wang, Yuanchun Li, Yunxin Liu, Xin Jin, Xuanzhe Liu, 16 Jan 2024, A Survey of Resource-efficient LLM and Multimodal Foundation Models, https://arxiv.org/abs/2401.08092 Project: https://github.com/UbiquitousLearning/Efficient_Foundation_Model_Survey (Broad survey with many optimizations including this topic.)
Dewen Zeng, Nan Du, Tao Wang, Yuanzhong Xu, Tao Lei, Zhifeng Chen, Claire Cui, 26 Nov 2023, Learning to Skip for Language Modeling, https://arxiv.org/abs/2311.15436 (Generalizes token-based early exiting to skip entire layers.)
Xupeng Miao, Gabriele Oliaro, Zhihao Zhang, Xinhao Cheng, Hongyi Jin, Tianqi Chen, Zhihao Jia, 23 Dec 2023, Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systems, https://arxiv.org/abs/2312.15234
Guangji Bai, Zheng Chai, Chen Ling, Shiyu Wang, Jiaying Lu, Nan Zhang, Tingwei Shi, Ziyang Yu, Mengdan Zhu, Yifei Zhang, Carl Yang, Yue Cheng, Liang Zhao, 4 Jan 2024, Beyond Efficiency: A Systematic Survey of Resource-Efficient Large Language Models https://arxiv.org/abs/2401.00625 (A general survey paper with coverage of many techniques including this one.)
J Ainslie, T Lei, M de Jong, S Ontañón, 2023, Colt5: Faster long-range transformers with conditional computation, https://arxiv.org/abs/2303.09752
L. Denoyer and P. Gallinari. Deep sequential neural network. arXiv preprint arXiv:1410.0510, 2014. https://arxiv.org/abs/1410.0510
Krishna Teja Chitty-Venkata, Sparsh Mittal, Murali Emani, Venkatram Vishwanath, Arun K. Somani, 2023, A Survey of Techniques for Optimizing Transformer Inference, https://arxiv.org/abs/2307.07982
Sotiris Anagnostidis, Dario Pavllo, Luca Biggio, Lorenzo Noci, Aurelien Lucchi, Thomas Hofmann, Dynamic Context Pruning for Efficient and Interpretable Autoregressive Transformers, arXiv preprint, 2023, https://arxiv.org/abs/2305.15805
Canwen Xu, Julian McAuley, Nov 2022, A Survey on Model Compression and Acceleration for Pretrained Language Models, https://arxiv.org/abs/2202.07105
Siyuan Wei, Tianzhu Ye, Shen Zhang, Yao Tang, Jiajun Liang, Joint Token Pruning and Squeezing Towards More Aggressive Compression of Vision Transformers, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023, pp. 2092-2101, http://openaccess.thecvf.com/content/CVPR2023/html/Wei_Joint_Token_Pruning_and_Squeezing_Towards_More_Aggressive_Compression_of_CVPR_2023_paper.html https://arxiv.org/abs/2304.10716 Code: https://github.com/megvii-research/TPS-CVPR2023
Zihang Dai, Zhilin Yang, Yiming Yang, Jaime Carbonell, Quoc V Le, and Ruslan Salakhutdinov. Transformer-xl: Attentive language models beyond a fixed-length context. arXiv preprint arXiv:1901.02860, 2019. https://arxiv.org/abs/1901.02860
Zihang Dai, Guokun Lai, Yiming Yang, and Quoc V Le. Funnel-transformer: Filtering out sequential redundancy for efficient language processing. arXiv preprint arXiv:2006.03236, 2020. https://arxiv.org/abs/2006.03236 Code: https://github.com/laiguokun/Funnel-Transformer
Xin Huang, Ashish Khetan, Rene Bidart, and Zohar Karnin. Pyramid-BERT: Reducing complexity via successive core-set based token selection. arXiv preprint arXiv:2203.14380, 2022. https://arxiv.org/abs/2203.14380
Ji Xin, Raphael Tang, Zhiying Jiang, Yaoliang Yu, Jimmy Lin, July 2022, Building an Efficiency Pipeline: Commutativity and Cumulativeness of Efficiency Operators for Transformers, arXiv preprint arXiv:2208.00483, https://arxiv.org/abs/2208.00483
Jean Senellart, Dakun Zhang, Bo Wang, Guillaume Klein, Jean-Pierre Ramatchandirin, Josep Crego, and Alexander Rush. 2018, OpenNMT system description for WNMT 2018: 800 words/sec on a single-core CPU. In Proc. of WNG, 2018. https://www.aclweb.org/anthology/W18-2715
Minchul Kim, Shangqian Gao, Yen-Chang Hsu, Yilin Shen, Hongxia Jin, 2 Dec 2023, Token Fusion: Bridging the Gap between Token Pruning and Token Merging, https://arxiv.org/abs/2312.01026
BAP Matrix, 2024, Zero-TPrune: Zero-Shot Token Pruning through Leveraging of the Attention Graph in Pre-Trained Transformers, https://openaccess.thecvf.com/content/CVPR2024/supplemental/Wang_Zero-TPrune_Zero-Shot_Token_CVPR_2024_supplemental.pdf
Wenxuan Huang, Yunhang Shen, Jiao Xie, Baochang Zhang, Gaoqi He, Ke Li, Xing Sun, Shaohui Lin, 31 Mar 2024, A General and Efficient Training for Transformer via Token Expansion, https://arxiv.org/abs/2404.00672 Code: https://github.com/Osilly/TokenExpansion (Token merging to accelerate training.)
Moein Heidari, Reza Azad, Sina Ghorbani Kolahi, René Arimond, Leon Niggemeier, Alaa Sulaiman, Afshin Bozorgpour, Ehsan Khodapanah Aghdam, Amirhossein Kazerouni, Ilker Hacihaliloglu, Dorit Merhof, 28 Mar 2024, Enhancing Efficiency in Vision Transformer Networks: Design Techniques and Insights, https://arxiv.org/abs/2403.19882 Project: https://github.com/mindflow-institue/Awesome-Attention-Mechanism-in-Medical-Imaging (Survey of optimization techniques for Vision Transformers, with particular focus on attention optimizations.)
Alessandro Baiocchi, Indro Spinelli, Alessandro Nicolosi, Simone Scardapane, 26 Jan 2024, Adaptive Point Transformer, https://arxiv.org/abs/2401.14845
Jianjian Cao, Peng Ye, Shengze Li, Chong Yu, Yansong Tang, Jiwen Lu, Tao Chen, 5 Mar 2024, MADTP: Multimodal Alignment-Guided Dynamic Token Pruning for Accelerating Vision-Language Transformer, https://arxiv.org/abs/2403.02991
Simone Scardapane, Alessandro Baiocchi, Alessio Devoto, Valerio Marsocci, Pasquale Minervini, Jary Pomponi, 12 Mar 2024, Conditional computation in neural networks: principles and research trends, https://arxiv.org/abs/2403.07965 (Investigated three types of dynamic inference: MoE, early exit, and token selection.)
Muhammad Adnan, Akhil Arunkumar, Gaurav Jain, Prashant J. Nair, Ilya Soloveychik, Purushotham Kamath, 14 Mar 2024, Keyformer: KV Cache Reduction through Key Tokens Selection for Efficient Generative Inference, https://arxiv.org/abs/2403.09054 (Reducing KV cache in-memory size and related computations by focusing on a subset of tokens.)
Jurek Leonhardt, Henrik Müller, Koustav Rudra, Megha Khosla, Abhijit Anand, Avishek Anand, Nov 2023, Efficient Neural Ranking using Forward Indexes and Lightweight Encoders, https://arxiv.org/abs/2311.01263
Maxim Bonnaerens, Nov 2023, Resource-Efficient Deep Learning for Computer Vision, Ph.D. thesis, Ghent University, https://biblio.ugent.be/publication/01HEMGWENRT8C255N2RD9KAEJC/file/01HEMGZ9JYP8NXPSQJZM14ACT9 (Examines various vision Transformer optimizations including a NAS approached based on building blocks and also combined token pruning/merging for input compression.)
Nikolay Bogoychev, Pinzhen Chen, Barry Haddow, Alexandra Birch, Nov 2023, Large Language Model Inference with Lexical Shortlisting, https://arxiv.org/abs/2311.09709 (Shortlisting the vocabulary to common words for reduced tokens and embedding matrix size.)
Jungmin Yun, Mihyeon Kim, Youngbin Kim, 2023, Focus on the Core: Efficient Attention via Pruned Token Compression for Document Classification https://aclanthology.org/2023.findings-emnlp.909.pdf
Chang Liu, Chongyang Tao, Jianxin Liang, Jiazhan Feng, Tao Shen, 2023, Quzhe Huang, Dongyan Zhao,Length-Adaptive Distillation: Customizing Small Language Model for Dynamic Token Pruning, Findings of the Association for Computational Linguistics: EMNLP 2023, pages 4452–4463, December 6-10, 2023, https://aclanthology.org/2023.findings-emnlp.294.pdf (Explores combining static model compression via knowledge distillation with dynamic adaptive inference via token pruning. This creates a modified distillation algorithm that prepares the model for token pruning during training.)
Yifei Liu, Mathias Gehrig, Nico Messikommer, Marco Cannici, Davide Scaramuzza, 2024, Revisiting Token Pruning for Object Detection and Instance Segmentation, IEEE Winter Conference on Applications of Computer Vision (WACV), Waikoloa, 2024, https://rpg.ifi.uzh.ch/docs/WACV24_Liu.pdf Code: https://github.com/uzh-rpg/svit/ (Examination of token pruning in image processing including re-activating pruned tokens and advanced techniques such as a dynamic pruning rate.)
Jinyu Chen, Wenchao Xu, Zicong Hong, Song Guo, Haozhao Wang, Jie Zhang, Deze Zeng, 10 Jan 2024, OTAS: An Elastic Transformer Serving System via Token Adaptation, https://arxiv.org/abs/2401.05031
Lu Yin, You Wu, Zhenyu Zhang, Cheng-Yu Hsieh, Yaqing Wang, Yiling Jia, Mykola Pechenizkiy, Yi Liang, Zhangyang Wang, Shiwei Liu, Oct 2023, Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity, https://arxiv.org/abs/2310.05175
Ali Modarressi, Hosein Mohebbi, and Mohammad Taher Pilehvar. 2022. AdapLeR: Speeding up inference by adaptive length reduction. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1–15, Dublin, Ireland. Association for Computational Linguistics, https://doi.org/10.18653/v1/2022.acl-long.1
B Sun, X Ye, Z Wang, H Li, Z Wang 2023 Exploring Coarse-to-Fine Action Token Localization and Interaction for Fine-grained Video Action Recognition, https://dl.acm.org/doi/abs/10.1145/3581783.3612206
F Shi, L Wang, Oct 2023 Bridging The Gaps Between Token Pruning and Full Pre-training via Masked Fine-tuning, arXiv preprint arXiv:2310.17177, https://arxiv.org/pdf/2310.17177.pdf
M Berchansky, P Izsak, A Caciularu, I Dagan, Oct 2023, Optimizing Retrieval-augmented Reader Models via Token Elimination, https://arxiv.org/pdf/2310.13682.pdf Code: https://github.com/mosheber/token_elimination
Zhengyang Zhuge, Peisong Wang, Xingting Yao1, Jian Cheng, 2024, Towards Efficient Spiking Transformer: a Token Sparsification Framework for Training and Inference Acceleration, https://openreview.net/pdf?id=yL6hljtjW4
Tao Yang, Dongyue Li, Zhuoran Song, Yilong Zhao, Fangxin Liu, Zongwu Wang, Zhezhi He, and Li Jiang, 2022, DTQAtten: Leveraging Dynamic Token-based Quantization for Efficient Attention Architecture, https://mxhx7199.github.io/files/%5BDATE-22%5DDTQAtten_preprint.pdf
Zheng Qu, Liu Liu, Fengbin Tu, Zhaodong Chen, Yufei Ding, Yuan Xie, 2022, DOTA: Detect and Omit Weak Attentions for Scalable Transformer Acceleration, ASPLOS ’22, February 28 ś March 4, 2022, Lausanne, Switzerland, PDF: https://dl.acm.org/doi/pdf/10.1145/3503222.3507738
David Spuler, March 2024, Chapter 49. Length Pruning, Generative AI in C++: Coding Transformers and LLMs, https://www.amazon.com/dp/B0CXJKCWX9
Y Wang, K Chen, H Tan, K Guo, 2023, Tabi: An Efficient Multi-Level Inference System for Large Language Models, EuroSys '23: Proceedings of the Eighteenth European Conference on Computer Systems, Rome, Italy May 2023, Pages 233–248, https://doi.org/10.1145/3552326.3587438 https://dl.acm.org/doi/10.1145/3552326.3587438 PDF: https://cse.hkust.edu.hk/~kaichen/papers/tabi-eurosys23.pdf (Dynamic routing to small or large LLMs based on the query.)
Sehoon Kim, 2023, Learned Token Pruning for Efficient Transformer Inference, Thesis, https://www2.eecs.berkeley.edu/Pubs/TechRpts/2023/EECS-2023-119.pdf
P Dong, M Sun, A Lu, Y Xie, K Liu, 2023, Heatvit: Hardware-efficient adaptive token pruning for vision transformers, https://ieeexplore.ieee.org/abstract/document/10071047/ https://arxiv.org/pdf/2211.08110
T. Ge, J. Hu, X. Wang, S.-Q. Chen, and F. Wei, “In-context autoencoder for context compression in a large language model,” arXiv preprint arXiv:2307.06945, 2023. https://arxiv.org/abs/2307.06945
Quandong Wang, Yuxuan Yuan, Xiaoyu Yang, Ruike Zhang, Kang Zhao, Wei Liu, Jian Luan, Daniel Povey, Bin Wang, 17 Jun 2024 (v2), SUBLLM: A Novel Efficient Architecture with Token Sequence Subsampling for LLM, https://arxiv.org/abs/2406.06571
Zhiyu Guo, Hidetaka Kamigaito, Taro Watanabe, 18 Jun 2024, Attention Score is not All You Need for Token Importance Indicator in KV Cache Reduction: Value Also Matters, https://arxiv.org/abs/2406.12335 (Extensions of KV cache token pruning methods tjat ise attention scores to find pivotal tokens, generalized to also consider L1 vector norms of value vectors.)
Nikolay Bogoychev, Pinzhen Chen, Barry Haddow, Alexandra Birch, June 20, 2024, The Ups and Downs of Large Language Model Inference, with Vocabulary Trimming by Language Heuristics, School of Informatics, University of Edinburgh, Proceedings of the Fifth Workshop on Insights from Negative Results in NLP, pages 148–153 https://aclanthology.org/2024.insights-1.17.pdf
Wonbeom Lee, Jungi Lee, Junghwan Seo, Jaewoong Sim, 28 Jun 2024, InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Management, https://arxiv.org/abs/2406.19707 (KV caching optimization using salient token pruning for the attention layer.)
Yun-Chia Yu; Mao-Chi Weng; Ming-Guang Lin; An-Yeu Andy Wu, May 2024, Retraining-free Constraint-aware Token Pruning for Vision Transformer on Edge Devices, 2024 IEEE International Symposium on Circuits and Systems (ISCAS), 19-22 May 2024, https://ieeexplore.ieee.org/abstract/document/10558603
Yuqing Yang, Yuedong Xu, Lei Jiao, 7 Jul 2024, A Queueing Theoretic Perspective on Low-Latency LLM Inference with Variable Token Length, https://arxiv.org/abs/2407.05347
Mootez Saad, José Antonio Hernández López, Boqi Chen, Dániel Varró, Tushar Sharma, 4 Jul 2024, ALPINE: An adaptive language-agnostic pruning method for language models for code, https://arxiv.org/abs/2407.04147
Siddharth Jha, Lutfi Eren Erdogan, Sehoon Kim, Kurt Keutzer, Amir Gholami, 11 Jul 2024, Characterizing Prompt Compression Methods for Long Context Inference, https://arxiv.org/abs/2407.08892
Haonan Wang, Jie Liu, Jie Tang, Gangshan Wu, Bo Xu, Yanbing Chou, Yong Wang, 16 Jul 2024 (v2), GTPT: Group-based Token Pruning Transformer for Efficient Human Pose Estimation, https://arxiv.org/abs/2407.10756
Qichen Fu, Minsik Cho, Thomas Merth, Sachin Mehta, Mohammad Rastegari, Mahyar Najibi, 19 Jul 2024, LazyLLM: Dynamic Token Pruning for Efficient Long Context LLM Inference, https://arxiv.org/abs/2407.14057
Junyoung Park, Myeonggu Kang, Yunki Han, Yanggon Kim, Jaekang Shin, Lee-Sup Kim, 21 Jul 2024, Token-Picker: Accelerating Attention in Text Generation with Minimized Memory Transfer via Probability Estimation, https://arxiv.org/abs/2407.15131
David Gu, July 18, 2024, Text Compression for Efficient Language Generation, Master’s Thesis, Distributed Computing Group, Computer Engineering and Networks Laboratory, ETH Zürich, https://pub.tik.ee.ethz.ch/students/2023-HS/MA-2023-19.pdf (Training and inference at the sentence level, including caching of embeddings per sentence, which also has the side-effect of compressing the input prompts and reducing computation analogously to token pruning.)
Oshin Dutta, Ritvik Gupta, Sumeet Agarwal, 2024, Efficient LLM Pruning with Global Token-Dependency Awareness and Hardware-Adapted Inference, https://openreview.net/pdf?id=cqhAzteLzc
Haode Qi, Cheng Qian, Jian Ni, Pratyush Singh, Reza Fazeli, Gengyu Wang, Zhongzheng Shu, Eric Wayne, Juergen Bross, 21 Aug 2024, Practical token pruning for foundation models in few-shot conversational virtual assistant systems, https://arxiv.org/abs/2408.11799 (Using token pruning for faster intent classification.)
Kazi Hasan Ibn Arif, JinYi Yoon, Dimitrios S. Nikolopoulos, Hans Vandierendonck, Deepu John, Bo Ji, 20 Aug 2024, HiRED: Attention-Guided Token Dropping for Efficient Inference of High-Resolution Vision-Language Models in Resource-Constrained Environments, https://arxiv.org/abs/2408.10945
Yufei Huang, Xu Han, Maosong Sun, 12 Aug 2024, FastFiD: Improve Inference Efficiency of Open Domain Question Answering via Sentence Selection, https://arxiv.org/abs/2408.06333 https://github.com/thunlp/FastFiD (Sentence-level pruning after encoding on the input text dimension.)
Bartosz Wójcik, Alessio Devoto, Karol Pustelnik, Pasquale Minervini, Simone Scardapane, 15 Dec 2023, Adaptive Computation Modules: Granular Conditional Computation For Efficient Inference, https://arxiv.org/abs/2312.10193
Li, Yanyu, Aug 2024, Accelerating large scale generative AI : a comprehensive study, Ph.D. Thesis, Northeastern University, Boston, Massachusetts, USA, https://hdl.handle.net/2047/D20669654 https://repository.library.northeastern.edu/files/neu:ms35wj107 https://repository.library.northeastern.edu/files/neu:ms35wj107/fulltext.pdf
CE Song, A Moradifirouzabadi, T Rosing, M Kang, 2024, Efficient Transformer Acceleration via Reconfiguration for Encoder and Decoder Models and Sparsity-Aware Algorithm Mapping, PDF: https://dl.acm.org/doi/pdf/10.1145/3665314.3670798
Purvish Jajal, Nick John Eliopoulos, Benjamin Shiue-Hal Chou, George K. Thiravathukal, James C. Davis, Yung-Hsiang Lu, 11 Sep 2024, Token Turing Machines are Efficient Vision Models, https://arxiv.org/abs/2409.07613
Weihao Ye, Qiong Wu, Wenhao Lin, Yiyi Zhou, 16 Sep 2024, Fit and Prune: Fast and Training-free Visual Token Pruning for Multi-modal Large Language Models, https://arxiv.org/abs/2409.10197 https://github.com/ywh187/FitPrune
Wang, X, Sep 2024, KELTP: Keyword-Enhanced Learned Token Pruning for Knowledge-Grounded Dialogue. In: Wand, M., Malinovská, K., Schmidhuber, J., Tetko, I.V. (eds) Artificial Neural Networks and Machine Learning – ICANN 2024. ICANN 2024. Lecture Notes in Computer Science, vol 15022. Springer, Cham. https://doi.org/10.1007/978-3-031-72350-6_16 https://link.springer.com/chapter/10.1007/978-3-031-72350-6_16 (Adaptive removal of low-attention tokens during inference.)
Zhenmei Shi, Yifei Ming, Xuan-Phi Nguyen, Yingyu Liang, Shafiq Joty, 25 Sep 2024, Discovering the Gems in Early Layers: Accelerating Long-Context LLMs with 1000x Input Token Reduction, https://arxiv.org/abs/2409.17422 https://github.com/SalesforceAIResearch/GemFilter (Use the early layers of a model to choose the most relevant tokens, similar to early exiting, and then compress the input token sequences based on the importance of these tokens. Notably, this reduces latency and also increases accuracy on long contexts.)
Jinming Lou, Wenyang Luo, Yufan Liu, Bing Li, Xinmiao Ding, Weiming Hu, Jiajiong Cao, Yuming Li, Chenguang Ma, 27 Sep 2024, Token Caching for Diffusion Transformer Acceleration, https://arxiv.org/abs/2409.18523
Yaniv Leviathan, Matan Kalman, Yossi Matias, 3 Oct 2024, Selective Attention Improves Transformer, https://arxiv.org/abs/2410.02703 (Allowing adjacent tokens to predict whether required for attention.)
Yuxin Li, Yiheng Li, Xulei Yang, Mengying Yu, Zihang Huang, Xiaojun Wu, Chai Kiat Yeo, 9 Oct 2024, Learning Content-Aware Multi-Modal Joint Input Pruning via Bird's-Eye-View Representation, https://arxiv.org/abs/2410.07268
K. Shi et al., "Fitop-Trans: Maximizing Transformer Pipeline Efficiency through Fixed-Length Token Pruning on FPGA," 2024 34th International Conference on Field-Programmable Logic and Applications (FPL), Torino, Italy, 2024, pp. 243-249, doi: 10.1109/FPL64840.2024.00041. https://ieeexplore.ieee.org/abstract/document/10705515/
Yefei He, Feng Chen, Jing Liu, Wenqi Shao, Hong Zhou, Kaipeng Zhang, Bohan Zhua, 11 Oct 2024, ZipVL: Efficient Large Vision-Language Models with Dynamic Token Sparsification and KV Cache Compression, https://arxiv.org/abs/2410.08584
Long Xing, Qidong Huang, Xiaoyi Dong, Jiajie Lu, Pan Zhang, Yuhang Zang, Yuhang Cao, Conghui He, Jiaqi Wang, Feng Wu, Dahua Lin, 22 Oct 2024, PyramidDrop: Accelerating Your Large Vision-Language Models via Pyramid Visual Redundancy Reduction, https://arxiv.org/abs/2410.17247
Jort Vincenti, Karim Abdel Sadek, Joan Velja, Matteo Nulli, Metod Jazbec, 24 Oct 2024, Dynamic Vocabulary Pruning in Early-Exit LLMs, https://arxiv.org/abs/2410.18952
X. Shen et al., "HotaQ: Hardware Oriented Token Adaptive Quantization for Large Language Models," in IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, doi: 10.1109/TCAD.2024.3487781. https://ieeexplore.ieee.org/abstract/document/10737419/
Xiangyu Zhang, Yu Zhou, Guang Yang, Harald C. Gall, Taolue Chen, 11 Nov 2024, Anchor Attention, Small Cache: Code Generation with Large Language Models, https://arxiv.org/abs/2411.06680
M Xu, D Cai, W Yin, S Wang, X Jin, X Liu - ACM Computing Surveys, 2024, Resource-efficient Algorithms and Systems of Foundation Models: A Survey, https://dl.acm.org/doi/pdf/10.1145/3706418
Christopher Keith, Michael Robinson, Francis Duncan , Allan Worthington, Joseph Wilson, Soa Harris, October 22nd, 2024, Optimizing large language models: A novel approach through dynamic token pruning, https://doi.org/10.21203/rs.3.rs-5293588/v1 https://assets-eu.researchsquare.com/files/rs-5293588/v1_covered_52b30393-790e-4ab2-8bee-e6e7d4b16895.pdf?c=1729565713 (Reduces the cost of the unembedding step in the early exit tests by dynamically pruning vocabularies, thereby dynamically reducing the size of the unembedding matrix used in early exit testing.)
OpenAI, Dec 2024, OpenAI o1 and new tools for developers, https://openai.com/index/o1-and-new-tools-for-developers/ ("Lower latency: o1 uses on average 60% fewer reasoning tokens than o1-preview for a given request.")
Guoxuan Chen, Han Shi, Jiawei Li, Yihang Gao, Xiaozhe Ren, Yimeng Chen, Xin Jiang, Zhenguo Li, Weiyang Liu, Chao Huang, 17 Dec 2024 (v2), SepLLM: Accelerate Large Language Models by Compressing One Segment into One Separator, https://arxiv.org/abs/2412.12094 http://sepllm.github.io/
Yu Kang, Xianghui Sun, Liangyu Chen, Wei Zou, 16 Dec 2024, C3oT: Generating Shorter Chain-of-Thought without Compromising Effectiveness, https://arxiv.org/abs/2412.11664 (Token pruning and prompt compression for Chain-of-Thought.)
Yueqian Lin, Yuzhe Fu, Jingyang Zhang, Yudong Liu, Jianyi Zhang, Jingwei Sun, Hai "Helen" Li, Yiran Chen, 16 Dec 2024, SpeechPrune: Context-aware Token Pruning for Speech Information Retrieval, https://arxiv.org/abs/2412.12009 https://speechprune.github.io/
Wenchao Xu, Jinyu Chen, Peirong Zheng, Xiaoquan Yi, Tianyi Tian, Wenhui Zhu, Quan Wan, Haozhao Wang, Yunfeng Fan, Qinliang Su, Xuemin Shen, https://arxiv.org/abs/2412.13437 18 Dec 2024, Deploying Foundation Model Powered Agent Services: A Survey, (A survey of not just deployment, but many inference optimization techniques.)
Xiaohu Huang, Hao Zhou, Kai Han, 20 Dec 2024, PruneVid: Visual Token Pruning for Efficient Video Large Language Models, https://arxiv.org/abs/2412.16117
Xuanli He, Iman Keivanloo, Yi Xu, Xiang He, Belinda Zeng, Santosh Rajagopalan, Trishul Chilimbi, 30 Oct 2021, Magic Pyramid: Accelerating Inference with Early Exiting and Token Pruning, https://arxiv.org/abs/2111.00230
Yue Guan, Zhengyi Li, Jingwen Leng, Zhouhan Lin, Minyi Guo, 15 May 2022, Transkimmer: Transformer Learns to Layer-wise Skim, https://arxiv.org/abs/2205.07324
Fabio Montello, Ronja Güldenring, Simone Scardapane, Lazaros Nalpantidis, 13 Jan 2025, A Survey on Dynamic Neural Networks: from Computer Vision to Multi-modal Sensor Fusion, https://arxiv.org/abs/2501.07451 (Survey of adaptive inference optimizations: early exit, dynamic routing, token skimming.)
Tuowei Wang, Xingyu Chen, Kun Li, Ting Cao, Ju Ren, Yaoxue Zhang, 15 Jan 2025, LeMo: Enabling LEss Token Involvement for MOre Context Fine-tuning, https://arxiv.org/abs/2501.09767
Weizhi Fei, Xueyan Niu, Guoqing Xie, Yingqing Liu, Bo Bai, Wei Han, 22 Jan 2025, Efficient Prompt Compression with Evaluator Heads for Long-Context Transformer Inference, https://arxiv.org/abs/2501.12959 (Input token scanning efficiencly using early exit during prefill to prune tokens for the decoding phase.)
Xiaoyu Liang, Chaofeng Guan, Jiaying Lu, Huiyao Chen, Huan Wang, Haoji Hu, 24 Jan 2025, Dynamic Token Reduction during Generation for Vision Language Models, https://arxiv.org/abs/2501.14204 (Dynamic pruning of visual tokens.)
Heejun Lee, Geon Park, Jaduk Suh, Sung Ju Hwang, 13 Feb 2025, InfiniteHiP: Extending Language Model Context Up to 3 Million Tokens on a Single GPU, https://arxiv.org/abs/2502.08910 (Using dynamic token pruning and KV cache data offloading to CPU memory.)
Zichen Wen, Yifeng Gao, Shaobo Wang, Junyuan Zhang, Qintong Zhang, Weijia Li, Conghui He, Linfeng Zhang, 17 Feb 2025, Stop Looking for Important Tokens in Multimodal Language Models: Duplication Matters More, https://arxiv.org/abs/2502.11494 https://github.com/ZichenWen1/DART
Shang Yang, Junxian Guo, Haotian Tang, Qinghao Hu, Guangxuan Xiao, Jiaming Tang, Yujun Lin, Zhijian Liu, Yao Lu, Song Han, 20 Feb 2025, LServe: Efficient Long-sequence LLM Serving with Unified Sparse Attention, https://arxiv.org/abs/2502.14866
Haoyu Wang, Tong Teng, Tianyu Guo, An Xiao, Duyu Tang, Hanting Chen, Yunhe Wang, 20 Feb 2025, Unshackling Context Length: An Efficient Selective Attention Approach through Query-Key Compression, https://arxiv.org/abs/2502.14477
Shaibal Saha, Lanyu Xu, 26 Feb 2025, Vision Transformers on the Edge: A Comprehensive Survey of Model Compression and Acceleration Strategies, https://arxiv.org/abs/2503.02891
Z. Zou et al., "PF2A-ViT: Parameter-Free and Feature-Aware Dynamic Token Pruning Accelerator With Complementary Quantization-Encoding for Vision Transformer," in IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, doi: 10.1109/TCAD.2025.3565468, https://ieeexplore.ieee.org/abstract/document/10979990
Aoming Liu, Reuben Tan, Boqing Gong, Bryan A. Plummer, 24 Jun 2025, Beyond Token Pruning: Operation Pruning in Vision-Language Models, https://arxiv.org/abs/2507.02909 https://github.com/zxcvfd13502/GSOP (Pruning of entire kernel operations.)
Dhruv Deshmukh, Saurabh Goyal, Nipun Kwatra, Ramachandran Ramjee, 18 Dec 2025, Kascade: A Practical Sparse Attention Method for Long-Context LLM Inference, https://arxiv.org/abs/2512.16391 (Chooses the likely output tokens exactly in early layers and then only computes with those, pruning less likely tokens.)
Rei Taniguchi, Yuyang Dong, Makoto Onizuka, Chuan Xiao, 12 Jan 2026, Adaptive Layer Selection for Layer-Wise Token Pruning in LLM Inference, https://arxiv.org/abs/2601.07667
Yushi Bai, Qian Dong, Ting Jiang, Xin Lv, Zhengxiao Du, Aohan Zeng, Jie Tang, Juanzi Li, 12 Mar 2026, IndexCache: Accelerating Sparse Attention via Cross-Layer Index Reuse, https://arxiv.org/abs/2603.12201
Zekai Li, Jintu Zheng, Ji Liu, Han Liu, Haowei Zhu, Zeping Li, Fuwei Yang, Haiduo Huang, Jinzhang Peng, Dong Li, Lu Tian, Emad Barsoum, 16 Dec 2024, FTP: A Fine-grained Token-wise Pruner for Large Language Models via Token Routing, https://arxiv.org/abs/2412.11494 https://arxiv.org/pdf/2412.11494
Guo-Hao Xu, Jingzhen Ding, Huping Ding, Zhao Xu, Kaifu Zhang, Sep 2024, FTP: Efficient Prefilling for Long-Context LLM Inference via FFN Token Pruning, https://openreview.net/forum?id=fL8Zp8o6RL PDF: https://openreview.net/pdf?id=fL8Zp8o6RL
David Spuler, Ph.D., Feb 6th, 2026 (updated), 500+ LLM Inference Optimization Techniques, Aussie AI Blog, https://www.aussieai.com/blog/llm-inference-optimization
Di Xiu, Hongyin Tang, Bolin Rong, Lizhi Yan, Jingang Wang, Yifan Lu, Xunliang Cai, 3 Dec 2025, A Preliminary Study on the Promises and Challenges of Native Top- Sparse Attention, https://arxiv.org/abs/2512.03494
Xuan Shen, Peiyan Dong, Lei Lu, Zhenglun Kong, Zhengang Li, Ming Lin, Chao Wu, Yanzhi Wang, 21 Apr 2025 (v2), Agile-Quant: Activation-Guided Quantization for Faster Inference of LLMs on the Edge, https://arxiv.org/abs/2312.05693 https://github.com/shawnricecake/agile-quant
David Spuler, March 2024, Chapter 46. Structured Pruning, in book "Generative AI in C++", https://www.aussieai.com/book/ch46-structured-pruning
David Spuler, March 2024, Chapter 49. Length Pruning, in book "Generative AI in C++", https://www.aussieai.com/book/ch49-length-pruning
David Spuler, March 2024, Generative AI in C++: Coding Transformers and LLMs, https://www.aussieai.com/book/toc PDF: https://www.aussieai.com/pdf/BOOK-Generative-AI-CPP-Spuler-2024.pdf
Surendra Pathak, Bo Han, 11 Apr 2026 (v2), Towards Efficient Large Vision-Language Models: A Comprehensive Survey on Inference Strategies, https://arxiv.org/abs/2603.27960
Ziqing Yang, Yiming Cui, and Zhigang Chen. 2022. TextPruner: A Model Pruning Toolkit for Pre-Trained Language Models. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 35–43, Dublin, Ireland. Association for Computational Linguistics. https://aclanthology.org/2022.acl-demo.4/ (Token and vocabulary pruning in text.)
Haozhe Hu, Hao Wu, Anhao Zhao, Longwei Ding, Peiran Yin, Yunpu Ma, Xiaoyu Shen, 8 Jun 2026, Beyond FLOPs: Benchmarking Real Inference Acceleration of LLM Pruning under a GEMM-Centric Taxonomy, https://arxiv.org/abs/2606.09080 https://github.com/EIT-NLP/LLM-Pruning/tree/main/PruningInferSim