Aussie AI

In-Context Learning (ICL)

Last Updated 22 October, 2025

by David Spuler, Ph.D.

What is In-Context Learning (ICL)?

In-Context Learning (ICL) is the general idea of using knowledge from the LLM's input prompt in answering a question. This doesn't sound very revolutionary these days, since we're all familiar with RAG architectures, but there was a time when it was a novel concept. When researchers put all their energy into pre-training the parametric knowledge of a model, it wasn't immediately obvious that it could be "augmented" with extra facts, just by putting them into the input string.

After all, the RAG technique itself was once an unproven research paper. The authors of the first RAG paper have gone on the record saying that, if they'd known how popular it would become, they would have chosen a better name!

Augmentation of knowledge via extra context tokens in the middle of the prompt is no longer new bananas. ICL is the underpinning idea behind various LLM prompt augmentation methods:

Research on ICL

Research papers on ICL include:

João Monteiro, Étienne Marcotte, Pierre-André Noël, Valentina Zantedeschi, David Vázquez, Nicolas Chapados, Christopher Pal, Perouz Taslakian, 23 Apr 2024, XC-Cache: Cross-Attending to Cached Context for Efficient LLM Inference, https://arxiv.org/abs/2404.15420
Andrea Matarazzo, Riccardo Torlone, 3 Jan 2025, A Survey on Large Language Models with some Insights on their Capabilities and Limitations, https://arxiv.org/abs/2501.04040 (Broad survey with many LLM topics covered from history to architectures to optimizations.)
Tong Xiao, Jingbo Zhu, 16 Jan 2025, Foundations of Large Language Models, https://arxiv.org/abs/2501.09223 (Huge 230 page paper on many topics such as training, prompting, alignment, and long context.)
Son, M., Won, Y.-J., & Lee, S. (2025). Optimizing Large Language Models: A Deep Dive into Effective Prompt Engineering Techniques. Applied Sciences, 15(3), 1430. https://doi.org/10.3390/app15031430 https://www.mdpi.com/2076-3417/15/3/1430
Fabio Matricardi, Jan 18, 2025, How a Small Language Model Can Achieve 100% Accuracy: In Context Learning is Underrated — ICL is the secret key to reach performance boosting — teach to an AI how to say “I don’t know” — part 2, https://generativeai.pub/how-a-small-language-model-can-achieve-100-accuracy-323a789ffa83
Xiaoran Liu, Ruixiao Li, Mianqiu Huang, Zhigeng Liu, Yuerong Song, Qipeng Guo, Siyang He, Qiqi Wang, Linlin Li, Qun Liu, Yaqian Zhou, Xuanjing Huang, Xipeng Qiu, 24 Feb 2025, Thus Spake Long-Context Large Language Model, https://arxiv.org/abs/2502.17129 (Impressive survey of many techniques to improve efficiency and accuracy of long context processing in both inference and training, covering text, video and multimodal models.)
Benoit Dherin, Michael Munn, Hanna Mazzawi, Michael Wunder, Javier Gonzalvo, 21 Jul 2025, Learning without training: The implicit dynamics of in-context learning, https://arxiv.org/abs/2507.16003
Jathin Korrapati, Patrick Mendoza, Aditya Tomar, Abein Abraham, 13 Aug 2025, Can Transformers Break Encryption Schemes via In-Context Learning?, https://arxiv.org/abs/2508.10235
Shugang Hao, Hongbo Li and Lingjie Duan, 14 Aug 2025, To Theoretically Understand Transformer-Based In-Context Learning for Optimizing CSMA, https://arxiv.org/abs/2508.09146
Shahriar Golchin, Yanfei Chen, Rujun Han, Manan Gandhi, Tianli Yu, Swaroop Mishra, Mihai Surdeanu, Rishabh Agarwal, Chen-Yu Lee, Tomas Pfister, 22 Jul 2025, Towards Compute-Optimal Many-Shot In-Context Learning, https://arxiv.org/abs/2507.16217
Jihyung Lee, Jin-Seop Lee, Jaehoon Lee, YunSeok Choi, Jee-Hyong Lee, 22 Jul 2025, DCG-SQL: Enhancing In-Context Learning for Text-to-SQL with Deep Contextual Schema Link Graph, https://arxiv.org/abs/2505.19956
Yongyi Yang, Hidenori Tanaka, Wei Hu, 17 Jul 2025, Provable Low-Frequency Bias of In-Context Learning of Representations, https://arxiv.org/abs/2507.13540
Erfan Pirmorad, 20 Jul 2025, Exploring the In-Context Learning Capabilities of LLMs for Money Laundering Detection in Financial Graphs, https://arxiv.org/abs/2507.14785
Xing Shen, Justin Szeto, Mingyang Li, Hengguan Huang, Tal Arbel, 29 Jun 2025, Exposing and Mitigating Calibration Biases and Demographic Unfairness in MLLM Few-Shot In-Context Learning for Medical Image Classification, https://arxiv.org/abs/2506.23298
Shuo Chen, Jianzhe Liu, Zhen Han, Yan Xia, Daniel Cremers, Philip Torr, Volker Tresp, Jindong Gu, 21 Jul 2025, True Multimodal In-Context Learning Needs Attention to the Visual Context, https://arxiv.org/abs/2507.15807
Yijing Lin, Mengqi Huang, Shuhan Zhuang, Zhendong Mao, 20 Jul 2025, RealGeneral: Unifying Visual Generation via Temporal In-Context Learning with Video Models, https://arxiv.org/abs/2503.10406
Hongbo Li, Lingjie Duan and Yingbin Liang, 28 Jul 2025, Provable In-Context Learning of Nonlinear Regression with Transformers, https://arxiv.org/abs/2507.20443
Kacper Kadziolka and Saber Salehkaleybar, 31 Jul 2025, Causal Reasoning in Pieces: Modular In-Context Learning for Causal Discovery, https://arxiv.org/abs/2507.23488
Kwesi Cobbina and Tianyi Zhou, 30 Jul 2025, Where to show Demos in Your Prompt: A Positional Bias of In-Context Learning, https://arxiv.org/abs/2507.22887
Huiyi Chen, Jiawei Peng, Kaihua Tang, Xin Geng, Xu Yang, 30 Jul 2025, Enhancing Multimodal In-Context Learning for Image Classification through Coreset Optimization, https://arxiv.org/abs/2504.14200
Patrik Kenfack, Samira Ebrahimi Kahou, Ulrich A\"ivodji, 1 Aug 2025, Towards Fair In-Context Learning with Tabular Foundation Models, https://arxiv.org/abs/2505.09503
Thomas F Burns, Tomoki Fukai, Christopher J Earls, 4 Aug 2025, Associative memory inspires improvements for in-context learning using a novel attention residual stream architecture, https://arxiv.org/abs/2412.15113
Ruixing Zhang, Bo Wang, Tongyu Zhu, Leilei Sun, Weifeng Lv, 5 Aug 2025, Urban In-Context Learning: Bridging Pretraining and Inference through Masked Diffusion for Urban Profiling, https://arxiv.org/abs/2508.03042
Simon Lepage, Jeremie Mary and David Picard, 5 Aug 2025, Markov Chain Estimation with In-Context Learning, https://arxiv.org/abs/2508.03934
Usman Anwar, Johannes Von Oswald, Louis Kirsch, David Krueger, Spencer Frei, 5 Aug 2025, Understanding In-Context Learning of Linear Models in Transformers Through an Adversarial Lens, https://arxiv.org/abs/2411.05189
Yanshu Li, Yi Cao, Hongyang He, Qisen Cheng, Xiang Fu, Xi Xiao, Tianyang Wang, Ruixiang Tang, 8 Aug 2025, M$^2$IV: Towards Efficient and Fine-grained Multimodal In-Context Learning via Representation Engineering, https://arxiv.org/abs/2504.04633
Hengzhe Zhang, Qi Chen, Bing Xue, Wolfgang Banzhaf, Mengjie Zhang, 8 Aug 2025, LLM-Meta-SR: In-Context Learning for Evolving Selection Operators in Symbolic Regression, https://arxiv.org/abs/2505.18602
Chenrui Liu, Falong Tan, Chuanlong Xie, Yicheng Zeng and Lixing Zhu, 12 Aug 2025, In-Context Learning as Nonparametric Conditional Probability Estimation: Risk Bounds and Optimality, https://arxiv.org/abs/2508.08673
Jaeyeon Kim, Sehyun Kwon, Joo Young Choi, Jongho Park, Jaewoong Cho, Jason D. Lee, Ernest K. Ryu, 12 Aug 2025, Task Diversity Shortens the ICL Plateau, https://arxiv.org/abs/2410.05448
Trevine Oorloff, Vishwanath Sindagi, Wele Gedara Chaminda Bandara, Ali Shafahi, Amin Ghiasi, Charan Prakash, Reza Ardekani, 13 Aug 2025, Stable Diffusion Models are Secretly Good at Visual In-Context Learning, https://arxiv.org/abs/2508.09949
Dake Bu, Wei Huang, Andi Han, Atsushi Nitanda, Taiji Suzuki, Qingfu Zhang, Hau-San Wong, 13 Aug 2025, Provably Transformers Harness Multi-Concept Word Semantics for Efficient In-Context Learning, https://arxiv.org/abs/2411.02199
Chuanliu Fan, Zicheng Ma, Jun Gao, Nan Yu, Jun Zhang, Ziqiang Cao, Yi Qin Gao, Guohong Fu, 17 Aug 2025, ProtTeX-CC: Activating In-Context Learning in Protein LLM via Two-Stage Instruction Compression, https://arxiv.org/abs/2508.12212
Chase Goddard, Lindsay M. Smith, Vudtiwat Ngampruetikorn, David J. Schwab, 18 Aug 2025, When can in-context learning generalize out of task distribution?, https://arxiv.org/abs/2506.05574
Aleksandra Bakalova, Yana Veitsman, Xinting Huang, Michael Hahn, 22 Aug 2025, Contextualize-then-Aggregate: Circuits for In-Context Learning in Gemma-2 2B, https://arxiv.org/abs/2504.00132
Fernando Martinez-Lopez, Tao Li, Yingdong Lu, Juntao Chen, 8 Aug 2025, In-Context Reinforcement Learning via Communicative World Models, https://arxiv.org/abs/2508.06659
Aditya Varre, Gizem Y\"uce, Nicolas Flammarion, 18 Aug 2025, Learning In-context $\pmb{n}$-grams with Transformers: Sub-$\pmb{n}$-grams Are Near-stationary Points, https://arxiv.org/abs/2508.12837
Quan Nguyen and Thanh Nguyen-Tang, 20 Aug 2025, One-Layer Transformers are Provably Optimal for In-context Reasoning and Distributional Association Learning in Next-Token Prediction Tasks, https://arxiv.org/abs/2505.15009
Wentao Wang, Guangyuan Jiang, Tal Linzen, Brenden M. Lake, 4 Sep 2025, Rapid Word Learning Through Meta In-Context Learning, https://arxiv.org/abs/2502.14791
Jacob Russin, Ellie Pavlick, Michael J. Frank, 4 Sep 2025, The dynamic interplay between in-context and in-weight learning in humans and neural networks, https://arxiv.org/abs/2402.08674
Ziniu Zhang, Zhenshuo Zhang, Dongyue Li, Lu Wang, Jennifer Dy, Hongyang R. Zhang, 27 Aug 2025, Linear-Time Demonstration Selection for In-Context Learning via Gradient Estimation, https://arxiv.org/abs/2508.19999
Rushitha Santhoshi Mamidala, Anshuman Chhabra, Ankur Mali, 22 Aug 2025, Rethinking Reasoning in LLMs: Neuro-Symbolic Local RetoMaton Beyond ICL and CoT, https://arxiv.org/abs/2508.19271
Ruobing Wang, Qiaoyu Tan, Yili Wang, Ying Wang, Xin Wang, 27 Aug 2025, CrystalICL: Enabling In-Context Learning for Crystal Generation, https://arxiv.org/abs/2508.20143
Souradeep Nanda, Anay Majee, Rishabh Iyer, 28 Aug 2025, InSQuAD: In-Context Learning for Efficient Retrieval via Submodular Mutual Information to Enforce Quality and Diversity, https://arxiv.org/abs/2508.21003
Gen Li, Yuchen Jiao, Yu Huang, Yuting Wei, Yuxin Chen, 28 Aug 2025, Transformers Meet In-Context Learning: A Universal Approximation Theory, https://arxiv.org/abs/2506.05200
Renat Sergazinov, Shao-An Yin, 30 Aug 2025, Chunked TabPFN: Exact Training-Free In-Context Learning for Long-Context Tabular Data, https://arxiv.org/abs/2509.00326
Stefano Fioravanti, Matteo Zavatteri, Roberto Confalonieri, Kamyar Zeinalipour, Paolo Frazzetto, Alessandro Sperduti, Nicol\`o Navarin, 1 Sep 2025, Iterative In-Context Learning to Enhance LLMs Abstract Reasoning: The Case-Study of Algebraic Tasks, https://arxiv.org/abs/2509.01267
Sachin Goyal, David Lopez-Paz, Kartik Ahuja, 1 Sep 2025, Distilled Pretraining: A modern lens of Data, In-Context Learning and Test-Time Scaling, https://arxiv.org/abs/2509.01649
Weicao Deng, Sangwoo Park, Min Li, and Osvaldo Simeone, 1 Sep 2025, Optimizing In-Context Learning for Efficient Full Conformal Prediction, https://arxiv.org/abs/2509.01840
Hao Yang, Zhiyu Yang, Yunjie Zhang, Shanyi Zhu, Lin Yang, 1 Sep 2025, Rethinking the Chain-of-Thought: The Roles of In-Context Learning and Pre-trained Priors, https://arxiv.org/abs/2509.01236
I. Shavindra Jayasekera, Jacob Si, Wenlong Chen, Filippo Valdettaro, A. Aldo Faisal, Yingzhen Li, 2 Sep 2025, Variational Uncertainty Decomposition for In-Context Learning, https://arxiv.org/abs/2509.02327
Teeradaj Racharak, Chaiyong Ragkhitwetsagul, Chommakorn Sontesadisai, Thanwadee Sunetnanta, 8 Sep 2025, Test It Before You Trust It: Applying Software Testing for Trustworthy In-context Learning, https://arxiv.org/abs/2504.18827
Michele Joshua Maggini, Dhia Merzougui, Rabiraj Bandyopadhyay, Ga\"el Dias, Fabrice Maurel, Pablo Gamallo, 9 Sep 2025, Are LLMs Enough for Hyperpartisan, Fake, Polarized and Harmful Content Detection? Evaluating In-Context Learning vs. Fine-Tuning, https://arxiv.org/abs/2509.07768
Adrian de Wynter, 12 Sep 2025, Is In-Context Learning Learning?, https://arxiv.org/abs/2509.10414
Haoyu Dong, Pengkun Zhang, Mingzhe Lu, Yanzhen Shen, Guolin Ke, 12 Sep 2025, MachineLearningLM: Scaling Many-shot In-context Learning via Continued Pretraining, https://arxiv.org/abs/2509.06806
Daniil Ignatev, Nan Li, Hugh Mee Wong, Anh Dang, Shane Kaszefski Yaschuk, 11 Sep 2025, DeMeVa at LeWiDi-2025: Modeling Perspectives with In-Context Learning and Label Distribution Learning, https://arxiv.org/abs/2509.09524
J\^onata Tyska Carvalho and Stefano Nolfi, 11 Sep 2025, LLMs for sensory-motor control: Combining in-context and iterative learning, https://arxiv.org/abs/2506.04867
Vaibhav Singh, Soumya Suvra Ghosal, Kapu Nirmal Joshua, Soumyabrata Pal, Sayak Ray Chowdhury, 19 Sep 2025, KITE: Kernelized and Information Theoretic Exemplars for In-Context Learning, https://arxiv.org/abs/2509.15676
Josip Juki\'c, Jan \v{S}najder, 18 Sep 2025, Disentangling Latent Shifts of In-Context Learning with Weak Supervision, https://arxiv.org/abs/2410.01508
Seongho Joo, Hyukhun Koh, Kyomin Jung, 13 Sep 2025, Public Data Assisted Differentially Private In-Context Learning, https://arxiv.org/abs/2509.10932
Chi Han, Ziqi Wang, Han Zhao, Heng Ji, 12 Sep 2025, Understanding Emergent In-Context Learning from a Kernel Regression Perspective, https://arxiv.org/abs/2305.12766
Kazumi Kasaura, Naoto Onda, Yuta Oriike, Masaya Taniguchi, Akiyoshi Sannai, Sho Sonoda, 16 Sep 2025, Discovering New Theorems via LLMs with In-Context Proof Learning in Lean, https://arxiv.org/abs/2509.14274
Samet Demir, Zafer Dogan, 18 Sep 2025, Asymptotic Study of In-context Learning with Random Transformers through Equivalent Models, https://arxiv.org/abs/2509.15152
Kishan Padayachy, Ronald Richman, Salvatore Scognamiglio, Mario V. W\"uthrich, 9 Sep 2025, In-Context Learning Enhanced Credibility Transformer, https://arxiv.org/abs/2509.08122
Bishnu Bhusal, Manoj Acharya, Ramneet Kaur, Colin Samplawski, Anirban Roy, Adam D. Cobb, Rohit Chadha, Susmit Jha, 17 Sep 2025, Privacy-Aware In-Context Learning for Large Language Models, https://arxiv.org/abs/2509.13625
Haolong Zheng, Yekaterina Yegorova, Mark Hasegawa-Johnson, 16 Sep 2025, TICL: Text-Embedding KNN For Speech In-Context Learning Unlocks Speech Recognition Abilities of Large Multimodal Models, https://arxiv.org/abs/2509.13395