LLM Memory Architectures

Last Updated 10 May, 2026

by David Spuler, Ph.D.

What are LLM Memory Architectures?

LLM memory is the use of extra information storage so that AI engines can retain information. For example, if you interact repeatedly with a model, you want it to remember your name. By default, LLMs don't have this type of permanent memory and are stateless architectures.

This page is about allowing AIs to have memory for facts. A separate page examines the other type of "memory" (chips) inside the computers that run AI: memory optimizations for LLM backend coding.

Research on LLM Memory Architectures

Research papers include:

Shenggang Li, Jul 30, 2024, Mem0: Is This the Future of AI Memory Management? https://ai.gopubby.com/mem0-is-this-the-future-of-ai-memory-management-1e228dc8220a
Aurimas Griciūnas, Oct 30, 2024, Memory in Agent Systems, https://www.newsletter.swirlai.com/p/memory-in-agent-systems
Zihong He, Weizhe Lin, Hao Zheng, Fan Zhang, Matt Jones, Laurence Aitchison, Xuhai Xu, Miao Liu, Per Ola Kristensson, Junxiao Shen, 1 Nov 2024, Human-inspired Perspectives: A Survey on AI Long-term Memory, https://arxiv.org/abs/2411.00489
Debmalya Biswas, Dec 2024, Long-term Memory for AI Agents: Why Vector Databases are not sufficient for Memory Management of Agentic AI Systems? https://ai.gopubby.com/long-term-memory-for-agentic-ai-systems-4ae9b37c6c0f
Mingda Chen, Yang Li, Karthik Padthe, Rulin Shao, Alicia Sun, Luke Zettlemoyer, Gargi Gosh, Wen-tau Yih, 24 Dec 2024, Improving Factuality with Explicit Working Memory, https://arxiv.org/abs/2412.18069
Ben Dickson, December 13, 2024, New LLM optimization technique slashes memory costs up to 75%, https://venturebeat.com/ai/new-llm-optimization-technique-slashes-memory-costs-up-to-75/
Edoardo Cetin, Qi Sun, Tianyu Zhao, Yujin Tang, 6 Dec 2024 (v3), An Evolved Universal Transformer Memory, https://arxiv.org/abs/2410.13166
Mingyu Jin, Weidi Luo, Sitao Cheng, Xinyi Wang, Wenyue Hua, Ruixiang Tang, William Yang Wang, Yongfeng Zhang, 21 Nov 2024 (v2), Disentangling Memory and Reasoning Ability in Large Language Models, https://arxiv.org/abs/2411.13504 https://github.com/MingyuJ666/Disentangling-Memory-and-Reasoning
Alhassan Mumuni, Fuseini Mumuni, 6 Jan 2025, Large language models for artificial general intelligence (AGI): A survey of foundational principles and approaches, https://arxiv.org/abs/2501.03151
Andrea Matarazzo, Riccardo Torlone, 3 Jan 2025, A Survey on Large Language Models with some Insights on their Capabilities and Limitations, https://arxiv.org/abs/2501.04040 (Broad survey with many LLM topics covered from history to architectures to optimizations.)
Ben Dickson, January 16, 2025, Google’s new neural-net LLM architecture separates memory components to control exploding costs of capacity and compute, https://venturebeat.com/ai/googles-new-neural-net-architecture-separates-memory-components-to-control-exploding-costs/
Mohamed A. Taha, 14 Jan 2025, Logarithmic Memory Networks (LMNs): Efficient Long-Range Sequence Modeling for Resource-Constrained Environments, https://arxiv.org/abs/2501.07905
Ali Behrouz, Peilin Zhong, Vahab Mirrokni, 31 Dec 2024, Titans: Learning to Memorize at Test Time, https://arxiv.org/abs/2501.00663
Tong Xiao, Jingbo Zhu, 16 Jan 2025, Foundations of Large Language Models, https://arxiv.org/abs/2501.09223 (Huge 230 page paper on many topics such as training, prompting, alignment, and long context.)
Sergey Legtchenko, Ioan Stefanovici, Richard Black, Antony Rowstron, Junyi Liu, Paolo Costa, Burcu Canakci, Dushyanth Narayanan, Xingbo Wu, 16 Jan 2025, Managed-Retention Memory: A New Class of Memory for the AI Era, https://arxiv.org/abs/2501.09605
Dr. Ashish Bamania, Jan 2025, Memory Layers Are Supercharging LLMs Like Never Before, https://levelup.gitconnected.com/memory-layers-are-supercharging-llms-like-never-before-056b99ea75cd
Vincent-Pierre Berges, Barlas Oğuz, Daniel Haziza, Wen-tau Yih, Luke Zettlemoyer, Gargi Ghosh, 20 Dec 2024 (v2), Memory Layers at Scale, https://arxiv.org/abs/2412.09764 https://github.com/facebookresearch/memory
Guillaume Lample, Alexandre Sablayrolles, Marc'Aurelio Ranzato, Ludovic Denoyer, Hervé Jégou, 16 Dec 2019 (v2), Large Memory Layers with Product Keys, https://arxiv.org/abs/1907.05242
Sainbayar Sukhbaatar, Arthur Szlam, Jason Weston, Rob Fergus, 24 Nov 2015 (v5), End-To-End Memory Networks, https://arxiv.org/abs/1503.08895 (Early paper as precursor to memory layers.)
Paul Sawers, January 23, 2025, Meta’s Yann LeCun predicts a ‘new AI architectures paradigm’ within 5 years and ‘decade of robotics’, https://techcrunch.com/2025/01/23/metas-yann-lecun-predicts-a-new-ai-architectures-paradigm-within-5-years-and-decade-of-robotics/
Haomiao Xiong, Zongxin Yang, Jiazuo Yu, Yunzhi Zhuge, Lu Zhang, Jiawen Zhu, Huchuan Lu, 23 Jan 2025, Streaming Video Understanding and Multi-round Interaction with Memory-enhanced Knowledge, https://arxiv.org/abs/2501.13468 https://github.com/hmxiong/StreamChat
Libo Wang, 24 Jan 2025, Wormhole Memory: A Rubik's Cube for Cross-Dialogue Retrieval, https://arxiv.org/abs/2501.14846
Avinash Patil, 5 Feb 2025, Advancing Reasoning in Large Language Models: Promising Methods and Approaches, https://arxiv.org/abs/2502.03671
Wujiang Xu, Zujie Liang, Kai Mei, Hang Gao, Juntao Tan, Yongfeng Zhang, 17 Feb 2025, A-MEM: Agentic Memory for LLM Agents, https://arxiv.org/abs/2502.12110 https://github.com/WujiangXu/AgenticMemory
Xiaoran Liu, Ruixiao Li, Mianqiu Huang, Zhigeng Liu, Yuerong Song, Qipeng Guo, Siyang He, Qiqi Wang, Linlin Li, Qun Liu, Yaqian Zhou, Xuanjing Huang, Xipeng Qiu, 24 Feb 2025, Thus Spake Long-Context Large Language Model, https://arxiv.org/abs/2502.17129 (Impressive survey of many techniques to improve efficiency and accuracy of long context processing in both inference and training, covering text, video and multimodal models.)
Emilia David, March 5, 2025, Enhancing AI agents with long-term memory: Insights into LangMem SDK, Memobase and the A-MEM Framework, https://venturebeat.com/ai/enhancing-ai-agents-with-long-term-memory-insights-into-langmem-sdk-memobase-and-the-a-mem-framework/
Asif Razzaq, March 8, 2025, Meet Manus: A New AI Agent from China with Deep Research + Operator + Computer Use + Lovable + Memory, https://www.marktechpost.com/2025/03/08/meet-manus-a-new-ai-agent-from-china-with-deep-research-operator-computer-use-lovable-memory/
Mingyue Cheng, Yucong Luo, Jie Ouyang, Qi Liu, Huijie Liu, Li Li, Shuo Yu, Bohou Zhang, Jiawei Cao, Jie Ma, Daoyu Wang, Enhong Chen, 17 Mar 2025 (v2), A Survey on Knowledge-Oriented Retrieval-Augmented Generation, https://arxiv.org/abs/2503.10677
Character.AI, May 19, 2025, Helping Characters Remember What Matters Most, https://blog.character.ai/helping-characters-remember-what-matters-most/
Nir Diamant, Jun 29, 2025, Memory Optimization Strategies in AI Agents: Building Smarter Agents That Learn and Remember, https://diamantai.substack.com/p/memory-optimization-strategies-in
Zeyu Zhang, Quanyu Dai, Xu Chen, Rui Li, Zhongyang Li, Zhenhua Dong, 4 May 2025, MemEngine: A Unified and Modular Library for Developing Advanced Memory of LLM-based Agents, https://arxiv.org/abs/2505.02099 https://github.com/nuster1128/MemEngine
Yiming Du, Wenyu Huang, Danna Zheng, Zhaowei Wang, Sebastien Montella, Mirella Lapata, Kam-Fai Wong, Jeff Z. Pan, 27 May 2025 (v2), Rethinking Memory in AI: Taxonomy, Operations, Topics, and Future Directions, https://arxiv.org/abs/2505.00675 https://github.com/Elvin-Yiming-Du/Survey_Memory_in_AI
Yaxiong Wu, Sheng Liang, Chen Zhang, Yichao Wang, Yongyue Zhang, Huifeng Guo, Ruiming Tang, Yong Liu, 23 Apr 2025 (v2), From Human Memory to AI Memory: A Survey on Memory Mechanisms in the Era of LLMs, https://arxiv.org/abs/2504.15965
Ningning Wang, Xavier Hu, Pai Liu, He Zhu, Yue Hou, Heyuan Huang, Shengyu Zhang, Jian Yang, Jiaheng Liu, Ge Zhang, Changwang Zhang, Jun Wang, Yuchen Eleanor Jiang, Wangchunshu Zhou, 24 Jul 2025, Efficient Agents: Building Effective Agents While Reducing Cost, https://arxiv.org/pdf/2508.02694 https://github.com/OPPO-PersonalAI/OAgents
Emilia David, August 13, 2025, Google adds limited chat personalization to Gemini, trails Anthropic and OpenAI in memory features, https://venturebeat.com/ai/google-adds-limited-chat-personalization-to-gemini-trails-anthropic-and-openai-in-memory-features/
Nathan Lambert, Aug 15, 2025, Contra Dwarkesh on Continual Learning: Don't try to make your airplane too much like a bird, https://www.interconnects.ai/p/contra-dwarkesh-on-continual-learning
MacKenzie Sigalos, Aug 19 2025 Sam Altman on GPT-6: ‘People want memory’, https://www.cnbc.com/2025/08/19/sam-altman-on-gpt-6-people-want-memory.html
Parsa Omidi, Xingshuai Huang, Axel Laborieux, Bahareh Nikpour, Tianyu Shi, and Armaghan Eshaghi, 14 Aug 2025, Memory-Augmented Transformers: A Systematic Review from Neuroscience Principles to Technical Solutions, https://arxiv.org/abs/2508.10824
Daniel Szelogowski, 29 Jul 2025, Hebbian Memory-Augmented Recurrent Networks: Engram Neurons in Deep Learning, https://arxiv.org/abs/2507.21474
Yi Kong, Dianxi Shi, Guoli Yang, Zhang ke-di, Chenlin Huang, Xiaopeng Li, Songchang Jin, 29 Jul 2025, MapAgent: Trajectory-Constructed Memory-Augmented Planning for Mobile Task Automation, https://arxiv.org/abs/2507.21953
Leyi Ouyang, 2 Aug 2025, Can Memory-Augmented LLM Agents Aid Journalism in Interpreting and Framing News for Diverse Audiences?, https://arxiv.org/abs/2507.21055
Yongyi Wang, Lingfeng Li, Bozhou Chen, Ang Li, Hanyu Liu, Qirui Zheng, Xionghui Yang, Wenxin Li, 6 Aug 2025, Synthetic POMDPs to Challenge Memory-Augmented RL: Memory Demand Structure Modeling, https://arxiv.org/abs/2508.04282
Dongwook Choi, Taeyoon Kwon, Dongil Yang, Hyojun Kim, Jinyoung Yeo, 12 Aug 2025, Designing Memory-Augmented AR Agents for Spatiotemporal Reasoning in Personalized Task Assistance, https://arxiv.org/abs/2508.08774
Gaoke Zhang, Bo Wang, Yunlong Ma, Dongming Zhao, Zifei Yu, 21 Aug 2025, Multiple Memory Systems for Enhancing the Long-term Memory of Agent, https://arxiv.org/abs/2508.15294
Haoran Sun, Shaoning Zeng, 23 Jul 2025, Hierarchical Memory for High-Efficiency Long-Term Reasoning in LLM Agents, https://arxiv.org/abs/2507.22925
Zhen Tan, Jun Yan, I-Hung Hsu, Rujun Han, Zifeng Wang, Long T. Le, Yiwen Song, Yanfei Chen, Hamid Palangi, George Lee, Anand Iyer, Tianlong Chen, Huan Liu, Chen-Yu Lee, Tomas Pfister, 28 Jul 2025, In Prospect and Retrospect: Reflective Memory Management for Long-term Personalized Dialogue Agents, https://arxiv.org/abs/2503.08026
Xiao Wang, Xufeng Lou, Shiao Wang, Ju Huang, Lan Chen, Bo Jiang, 6 Aug 2025, Long-Term Visual Object Tracking with Event Cameras: An Associative Memory Augmented Tracker and A Benchmark Dataset, https://arxiv.org/abs/2403.05839
Qingyue Wang, Yanhe Fu, Yanan Cao, Shuai Wang, Zhiliang Tian, Liang Ding, 25 Aug 2025, Recursively Summarizing Enables Long-Term Dialogue Memory in Large Language Models, https://arxiv.org/abs/2308.15022
Botao Zhu, Jeslyn Wang, Dusit Niyato, Xianbin Wang, 9 Sep 2025, Trust Semantics Distillation for Collaborator Selection via Memory-Augmented Agentic AI, https://arxiv.org/abs/2509.08151
Darshan Deshpande, Varun Gangal, Hersh Mehta, Anand Kannappan, Rebecca Qian, Peng Wang, 1 Oct 2025, MEMTRACK: Evaluating Long-Term Memory and State Tracking in Multi-Platform Dynamic Agent Environments, https://arxiv.org/abs/2510.01353
Junsong Li, Jie Zhou, Bihao Zhan, Yutao Yang, Qianjun Pan, Shilian Chen, Tianyu Huai, Xin Li, Qin Chen, Liang He, 21 Sep 2025, LifeAlign: Lifelong Alignment for Large Language Models with Memory-Augmented Focalized Preference Optimization, https://arxiv.org/abs/2509.17183
Runnan Qi and Yanan Ni and Lumin Jiang and Zongyuan Li and Kuihua Huang and Xian Guo, 21 Oct 2025, Memory-Augmented State Machine Prompting: A Novel LLM Agent Framework for Real-Time Strategy Games, https://arxiv.org/abs/2510.18395
Jizhan Fang, Xinle Deng, Haoming Xu, Ziyan Jiang, Yuqi Tang, Ziwen Xu, Shumin Deng, Yunzhi Yao, Mengru Wang, Shuofei Qiao, Huajun Chen, Ningyu Zhang, 21 Oct 2025, LightMem: Lightweight and Efficient Memory-Augmented Generation, https://arxiv.org/abs/2510.18866
Yifei Dong, Fengyi Wu, Guangyu Chen, Zhi-Qi Cheng, Qiyu Hu, Yuxuan Zhou, Jingdong Sun, Jun-Yan He, Qi Dai, Alexander G Hauptmann, 9 Oct 2025, Unified World Models: Memory-Augmented Planning and Foresight for Visual Navigation, https://arxiv.org/abs/2510.08713
Aneesh Jonelagadda, Christina Hahn, Haoze Zheng, Salvatore Penachio (Kaliber AI), 7 Oct 2025, Mnemosyne: An Unsupervised, Human-Inspired Long-Term Memory Architecture for Edge-Based LLMs, https://arxiv.org/abs/2510.08601
Haoran Sun, Zekun Zhang, Shaoning Zeng, 10 Oct 2025, Preference-Aware Memory Update for Long-Term LLM Agents, https://arxiv.org/abs/2510.09720
Rebecca Westh\"au{\ss}er, Wolfgang Minker, Sebatian Zepf, 9 Oct 2025, Enabling Personalized Long-term Interactions in LLM-based Agents through Persistent Memory and User Profiles, https://arxiv.org/abs/2510.07925
T BAI, J FAN, X WEN, J KANG, H LAN, R ZHAO, P WU, March 2026, Survey on AI Memory: Theories, Taxonomies, Evaluations, and Emerging Trends, https://www.researchgate.net/profile/Jiayang-Fan/publication/403076775_Survey_on_AI_Memory_Theories_Taxonomies_Evaluations_and_Emerging_Trends/links/69c2349ee8c97366295df2f9/Survey-on-AI-Memory-Theories-Taxonomies-Evaluations-and-Emerging-Trends.pdf https://github.com/BAI-LAB/Survey-on-AI-Memory
Juwei Yue, Chuanrui Hu, Jiawei Sheng, Zuyi Zhou, Wenyuan Zhang, Tingwen Liu, Li Guo, Yafeng Deng, 10 Apr 2026 (v2), HyperMem: Hypergraph Memory for Long-Term Conversations https://arxiv.org/abs/2604.08256
Mandar Karhade, MD. PhD., Apr 8, 2026, Andrej Karpathy Killed RAG. Or Did He? The LLM Wiki Pattern: 5,000 stars in 48 hours. A GitHub Gist. No code. Just an idea file. And the entire AI community lost its collective mind. https://pub.towardsai.net/andrej-karpathy-killed-rag-or-did-he-the-llm-wiki-pattern-7824d876e790
Carl Franzen, April 3, 2026, Karpathy shares 'LLM Knowledge Base' architecture that bypasses RAG with an evolving markdown library maintained by AI, https://venturebeat.com/data/karpathy-shares-llm-knowledge-base-architecture-that-bypasses-rag-with-an
Mandar Karhade, MD. PhD., Apr 8, 2026, MemPalace By Mila Jovovich: 96.6% Recall With Zero API Calls (Too Good To Be True?) How a spatial memory system built on raw verbatim text is quietly outperforming every AI memory product on the market. For free, https://ai.gopubby.com/mempalace-by-mila-jovovich-96-6-recall-with-zero-api-calls-too-good-to-be-true-bebcf26271d0
Vincent-Pierre Berges, Barlas Oguz, December 12, 2024, Memory Layers at Scale, Meta, https://ai.meta.com/research/publications/memory-layers-at-scale/ https://github.com/facebookresearch/memory (Augmention of an LLM with an additional key-value associative memory, by replacing some FFNs with a "memory layer".)
Yimin Tang, Zhenghong Yu, Yi Zheng, T. K. Satish Kumar, Jiaoyang Li, Sven Koenig, 6 Jan 2025, Enhancing Lifelong Multi-Agent Path Finding with Cache Mechanism, https://arxiv.org/abs/2501.02803
Yimin Tang, Zhenghong Yu, Yi Zheng, T. K. Satish Kumar, Jiaoyang Li, Sven Koenig, 5 Apr 2024 (v3), Caching-Augmented Lifelong Multi-Agent Path Finding, https://arxiv.org/abs/2403.13421
Shuhang Lin, Zhencan Peng, Lingyao Li, Xiao Lin, Xi Zhu, Yongfeng Zhang, 4 Nov 2025, Cache Mechanism for Agent RAG Systems, https://arxiv.org/abs/2511.02919
HT To, S Nguyen, NH Pham, 7 Jan 2026 (v2), CAHC:A General Conflict-Aware Heuristic Caching Framework for Multi-Agent Path Finding, https://arxiv.org/abs/2512.12243
Ali Behrouz, Meisam Razaviyayn, and Vahab Mirrokni, December 4, 2025, Titans + MIRAS: Helping AI have long-term memory, https://research.google/blog/titans-miras-helping-ai-have-long-term-memory/
Ali Behrouz, Meisam Razaviyayn, Peilin Zhong, Vahab Mirrokni, 17 Apr 2025, It's All Connected: A Journey Through Test-Time Memorization, Attentional Bias, Retention, and Online Optimization, https://arxiv.org/abs/2504.13173
Chaojun Xiao, Pengle Zhang, Xu Han, Guangxuan Xiao, Yankai Lin, Zhengyan Zhang, Zhiyuan Liu, Maosong Sun, 28 May 2024 (v2), InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memory, https://arxiv.org/abs/2402.04617 https://github.com/thunlp/InfLLM