Token Skipping

What is Token Skipping?

Token skipping is an LLM inference optimization methods that "skips" the processing of some input tokens. Fewer total tokens to process means less GPU compute cost and faster inference. Variants of this technique include token pruning, token dropping, and token merging.

Releated research areas include:

• Token reduction
• Prompt compression
• Context compression
• Token pruning
• Reasoning token reduction
• Token merging
• Token dropping

Token Skipping: Book Excerpts and Blog Articles

Free online book excerpts with full text chapters online and free PDF downloads, and the Aussie AI blog, including related articles:

• Vision attention optimization is a speed optimization for the inference in Vision Transformers (ViTs). This uses techniques similar to image attention optimization, such as pruning or merging patches or sections of the images ... more about Vision attention optimization »
• Vision token merging is the combining of two or more areas of an image for faster AI processing. Vision models tokenize the images into segments, which must each be processed by the LLM. Merging them together allows the model to process only a single area, with reduced accuracy ... more about Vision token merging »
• Vision token pruning is the removal and avoidance of processing portions of an image or video stream. The idea generalizes token pruning in text models, which remove words, to avoid processing some segments of visual images. This is an effective speedup for image processing or machine vision because many of the areas of a large image are not important, and their ... more about Vision token pruning »
• Vision token reduction is an LLM inference optimization technique for vision models that involves pruning or merging tokens. Tokenization of vision data is based on image tokenization, and can be optimized in various ways. Vision token pruning is avoiding the processing of redundant or unimportant segments of images, such as recurring blank background regions. ... more about Vision token reduction »

Research on Token Skipping

Research papers include:

• Shujian Zhang, Korawat Tanwisuth, Chengyue Gong, Pengcheng He, Mingyuan Zhou, 7 May 2024, Switchable Decision: Dynamic Neural Generation Networks, https://arxiv.org/abs/2405.04513 (Switching and skipping sub-layer components such as attention heads, FFNs, or input token skipping, using decisions made based on allocating computation resources.)
• Wangbo Zhao, Jiasheng Tang, Yizeng Han, Yibing Song, Kai Wang, Gao Huang, Fan Wang, Yang You, 18 Mar 2024, Dynamic Tuning Towards Parameter and Inference Efficiency for ViT Adaptation, https://arxiv.org/abs/2403.11808 (PEFT and adaptive inference and token pruning in Vision Transformers.)
• Foozhan Ataiefard, Walid Ahmed, Habib Hajimolahoseini, Saina Asani, Farnoosh Javadi, Mohammad Hassanpour, Omar Mohamed Awad, Austin Wen, Kangling Liu, Yang Liu, 27 Jan 2024, SkipViT: Speeding Up Vision Transformers with a Token-Level Skip Connection, https://arxiv.org/abs/2401.15293
• Fabio Montello, Ronja Güldenring, Simone Scardapane, Lazaros Nalpantidis, 13 Jan 2025, A Survey on Dynamic Neural Networks: from Computer Vision to Multi-modal Sensor Fusion, https://arxiv.org/abs/2501.07451 (Survey of adaptive inference optimizations: early exit, dynamic routing, token skimming.)
• Heming Xia, Yongqi Li, Chak Tou Leong, Wenjie Wang, Wenjie Li, 17 Feb 2025, TokenSkip: Controllable Chain-of-Thought Compression in LLMs, https://arxiv.org/abs/2502.12067
• Rohan Choudhury, Shanchuan Lin, Jianyi Wang, Hao Chen, Qi Zhao, Feng Cheng, Lu Jiang, Kris Kitani, Laszlo A. Jeni, 9 Oct 2025, SkipSR: Faster Super Resolution with Token Skipping, https://arxiv.org/abs/2510.08799

More AI Research Topics

Read more about:

• 500+ LLM Inference Optimization Techniques
• What's Hot in LLM Inference Optimization in 2025?
• Inference Optimization Research
• « Research Home