Shallow Prefill

Research on Shallow Prefill

Research papers include:

• Jungsuk Oh, Hyeseo Jeon, Hyunjune Ji, Kyongmin Kong, Jay-Yoon Lee, 7 May 2026, Shallow Prefill, Deep Decoding: Efficient Long-Context Inference via Layer-Asymmetric KV Visibility, https://arxiv.org/abs/2605.06105
• Yingjie Xia, Tao Liu, Jinglei Shi, Qingsong Xie, Heng Guo, Jian Yang, Xi Wang, 14 Mar 2026 (v2), ShaRP: SHAllow-LayeR Pruning for Efficient Video Large Language Models, https://arxiv.org/abs/2512.05385
• Junhui He, Zhihui Fu, Jun Wang, Qingan Li, 16 Apr 2026 (v2), POP: Prefill-Only Pruning for Efficient Large Model Inference, https://arxiv.org/abs/2602.03295
• Woojeong Kim, Junxiong Wang, Jing Nathan Yan, Mohamed Abdelfattah, Alexander M. Rush, 11 Aug 2025, OverFill: Two-Stage Models for Efficient Language Model Decoding, https://arxiv.org/abs/2508.08446
• Sangmin Yoo, Srikanth Malla, Chiho Choi, Wei D. Lu, Joon Hee Choi, 7 Jan 2026, ADEPT: Adaptive Dynamic Early-Exit Process for Transformers, https://arxiv.org/abs/2601.03700 (Early exit of layers during prefill with a mechanism to fix the KV cache for missing layers when later needed in the decoding phase.)
• Haiquan Lu, Zigeng Chen, Gongfan Fang, Xinyin Ma, Xinchao Wang, 19 May 2026, Mix-Quant: Quantized Prefilling, Precise Decoding for Agentic LLMs, https://arxiv.org/abs/2605.20315 (A kind of PD-disaggregation from using different levels of quantization of the same model, with prefill more quantized and decoding more dense.)
• Qihang Fan, Huaibo Huang, Zhiying Wu, Juqiu Wang, Bingning Wang, Ran He, 6 Mar 2026, FlashPrefill: Instantaneous Pattern Discovery and Thresholding for Ultra-Fast Long-Context Prefilling, https://arxiv.org/abs/2603.06199
• Huiqiang Jiang, Yucheng Li, Chengruidong Zhang, Qianhui Wu, Xufang Luo, Surin Ahn, Zhenhua Han, Amir H. Abdi, Dongsheng Li, Chin-Yew Lin, Yuqing Yang, Lili Qiu, 30 Oct 2024 (v2), MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attention, Advances in Neural Information Processing Systems, 37:52481–52515, 2024 https://arxiv.org/abs/2407.02490

AI Books from Aussie AI

The Sweetest Lesson: Your Brain Versus AI: new book on AI intelligence theory: Your brain is 50 times bigger than the best AI engines. Truly intelligent AI will require more compute! Another case of the bitter lesson? Maybe it's the opposite of that: the sweetest lesson. Get your copy from Amazon: The Sweetest Lesson

RAG Optimization: Accurate and Efficient LLM Applications: new book on RAG architectures: Smarter RAG Faster RAG Cheaper RAG Agentic RAG RAG reasoning Get your copy from Amazon: RAG Optimization

Generative AI Applications book: Deciding on your AI project Planning for success and safety Designs and LLM architectures Expediting development Implementation and deployment Get your copy from Amazon: Generative AI Applications

Generative AI programming book: Generative AI coding in C++ Transformer engine speedups LLM models Phone and desktop AI Code examples Research citations Get your copy from Amazon: Generative AI in C++

CUDA C++ Optimization book: Faster CUDA C++ kernels Optimization tools & techniques Compute optimization Memory optimization Get your copy from Amazon: CUDA C++ Optimization

CUDA C++ Debugging book: Debugging CUDA C++ kernels Tools & techniques Self-testing & reliability Common GPU kernel bugs Get your copy from Amazon: CUDA C++ Debugging

C++ AVX Optimization: CPU SIMD Vectorization: Introduction to AVX SIMD intrinsics Vectorization and horizontal reductions Low latency tricks and branchless programming Instruction-level parallelism and out-of-order execution Loop unrolling & double loop unrolling Get your copy from Amazon: C++ AVX Optimization: CPU SIMD Vectorization

C++ Ultra-Low Latency: Multithreading and Low-Level Optimizations: Low-level C++ efficiency techniques C++ multithreading optimizations AI LLM inference backend speedups Low latency data structures Multithreading optimizations General C++ optimizations Get your copy from Amazon: C++ Ultra-Low Latency

More AI Research Topics

Read more about:

• 500+ LLM Inference Optimization Techniques
• What's Hot in LLM Inference Optimization in 2025?
• Inference Optimization Research
• « Research Home