Vector Database Optimizations

What are Vector Databases?

Vector databases are fast methods to look up vector data. Although traditionally used for GIS data (e.g. maps), they are mainly used in AI for looking up embedding vectors for use in vector databases. See also: vector databases, vector hashing, semantic search, RAG systems.

Survey papers on Vector Databases

Review papers on vector databases:

• James Jie Pan, Jianguo Wang, Guoliang Li, 21 Oct 2023, Survey of Vector Database Management Systems, https://arxiv.org/abs/2310.14021 https://link.springer.com/article/10.1007/s00778-024-00864-x
• Zhi Jing, Yongye Su, Yikun Han, Bo Yuan, Haiyun Xu, Chunjiang Liu, Kehai Chen, Min Zhang, 6 Feb 2024 (v2), When Large Language Models Meet Vector Databases: A Survey, https://arxiv.org/abs/2402.01763
• Yikun Han, Chunjiang Liu, Pengfei Wang, 18 Oct 2023, A Comprehensive Survey on Vector Database: Storage and Retrieval Technique, Challenge, https://arxiv.org/abs/2310.11703
• Toni Taipalus, 2024, Vector database management systems: Fundamental concepts, use-cases, and current challenges, Cognitive Systems Research, Volume 85, 101216, ISSN 1389-0417, https://doi.org/10.1016/j.cogsys.2024.101216 https://www.sciencedirect.com/science/article/pii/S1389041724000093

Vector Databases: Book Excerpts and Blog Articles

Free online book excerpts with full text chapters online and free PDF downloads, and the Aussie AI blog, including related articles:

• David Spuler, March 2024, Vector Databases, in Generative AI in C++, https://www.aussieai.com/book/ch6-vector-databases
• David Spuler, March 2024, Semantic Caching and Vector Databases, in Generative AI in C++, https://www.aussieai.com/book/ch29-semantic-caching-vector-databases
• David Spuler, Michael Sharpe, June 2025, Vector Databases, Chapter 8, "RAG Optimization: Accurate and Efficient LLM Applications", https://www.aussieai.com/book/rag-book-8-vector-databases
• David Spuler, Michael Sharpe, 2025, RAG Architectures, Chapter 15, in "Generative AI Applications", https://www.aussieai.com/book/ai-apps-book-15-rag-architectures
• David Spuler, March 2024, Chapter 6. Training, Fine-Tuning & RAG, in book "Generative AI in C++", https://www.aussieai.com/book/ch6-training-ft-rag
• David Spuler, March 2024, Chapter 29. Caching Optimizations, in book "Generative AI in C++", https://www.aussieai.com/book/ch29-caching
• David Spuler, March 2024, Generative AI in C++: Coding Transformers and LLMs, https://www.aussieai.com/book/toc PDF: https://www.aussieai.com/pdf/BOOK-Generative-AI-CPP-Spuler-2024.pdf

Research on Vector Databases

Research papers on vector databases:

• Sebastian Bruch, Jan 2024, Foundations of Vector Retrieval, https://arxiv.org/abs/2401.09350 (Extensive 200+ pages review of vector lookup data structures such as LSH and clustering.)
• Xiaohua Wang, Zhenghua Wang, Xuan Gao, Feiran Zhang, Yixin Wu, Zhibo Xu, Tianyuan Shi, Zhengyuan Wang, Shizheng Li, Qi Qian, Ruicheng Yin, Changze Lv, Xiaoqing Zheng, Xuanjing Huang, 1 Jul 2024, Searching for Best Practices in Retrieval-Augmented Generation, https://arxiv.org/abs/2407.01219 Project: https://github.com/FudanDNN-NLP/RAG (Attempts to optimize the entire RAG system, including the various options for different RAG modules in the RAG pipeline, such as optimal methods for chunking, retrieval, embedding models, vector databases, prompt compression, reranking, repacking, summarizers, and other components.)
• Dr. Ashish Bamania, Jun 18, 2024, Google’s New Algorithms Just Made Searching Vector Databases Faster Than Ever: A Deep Dive into how Google’s ScaNN and SOAR Search algorithms supercharge the performance of Vector Databases, https://levelup.gitconnected.com/googles-new-algorithms-just-made-searching-vector-databases-faster-than-ever-36073618d078
• Chips Ahoy Capital, Jul 02, 2024, Evolution of Databases in the World of AI Apps, https://chipsahoycapital.substack.com/p/evolution-of-databases-in-the-world
• Donald Farmer, 08 Aug 2024, 10 top vector database options for similarity searches, https://www.techtarget.com/searchdatamanagement/tip/Top-vector-database-options-for-similarity-searches
• Pere Martra, Aug 2024 (accessed), Implementing semantic cache to improve a RAG system with FAISS, https://huggingface.co/learn/cookbook/semantic_cache_chroma_vector_database
• Richmond Alake, Apoorva Joshi, Aug 14, 2024, Adding Semantic Caching and Memory to Your RAG Application Using MongoDB and LangChain, MongoDB, https://www.mongodb.com/developer/products/atlas/advanced-rag-langchain-mongodb/
• James Jie Pan, Jianguo Wang, Guoliang Li, 21 Oct 2023, Survey of Vector Database Management Systems, https://arxiv.org/abs/2310.14021 https://link.springer.com/article/10.1007/s00778-024-00864-x
• Zhi Jing, Yongye Su, Yikun Han, Bo Yuan, Haiyun Xu, Chunjiang Liu, Kehai Chen, Min Zhang, 6 Feb 2024 (v2), When Large Language Models Meet Vector Databases: A Survey, https://arxiv.org/abs/2402.01763
• Yikun Han, Chunjiang Liu, Pengfei Wang, 18 Oct 2023, A Comprehensive Survey on Vector Database: Storage and Retrieval Technique, Challenge, https://arxiv.org/abs/2310.11703
• Toni Taipalus, 2024, Vector database management systems: Fundamental concepts, use-cases, and current challenges, Cognitive Systems Research, Volume 85, 101216, ISSN 1389-0417, https://doi.org/10.1016/j.cogsys.2024.101216 https://www.sciencedirect.com/science/article/pii/S1389041724000093
• Zhi Yao, Zhiqing Tang, Jiong Lou, Ping Shen, Weijia Jia, 19 Jun 2024, VELO: A Vector Database-Assisted Cloud-Edge Collaborative LLM QoS Optimization Framework, https://arxiv.org/abs/2406.13399
• David Spuler, March 2024, Vector Databases, in Generative AI in C++, https://www.aussieai.com/book/ch6-vector-databases
• David Spuler, March 2024, Semantic Caching and Vector Databases, in Generative AI in C++, https://www.aussieai.com/book/ch29-semantic-caching-vector-databases
• Chirag Agrawal, Sep 20, 2024, Unlocking the Power of Efficient Vector Search in RAG Applications, https://pub.towardsai.net/unlocking-the-power-of-efficient-vector-search-in-rag-applications-c2e3a0c551d5
• F Sundh, Oct 2024, Evaluating the efficacy of modality conversion in vector databases, Bachelor's Thesis, Computer Science and Engineering, Luleå University of Technology, Department of Computer Science, Electrical and Space Engineering, https://www.diva-portal.org/smash/get/diva2:1905628/FULLTEXT01.pdf
• Tolga Şakar and Hakan Emekci, 30 October 2024, Maximizing RAG efficiency: A comparative analysis of RAG methods, Natural Language Processing. doi:10.1017/nlp.2024.53, https://www.cambridge.org/core/journals/natural-language-processing/article/maximizing-rag-efficiency-a-comparative-analysis-of-rag-methods/D7B259BCD35586E04358DF06006E0A85 https://www.cambridge.org/core/services/aop-cambridge-core/content/view/D7B259BCD35586E04358DF06006E0A85/S2977042424000530a.pdf/div-class-title-maximizing-rag-efficiency-a-comparative-analysis-of-rag-methods-div.pdf
• Sonal Prabhune, Donald J. Berndt, 7 Nov 2024, Deploying Large Language Models With Retrieval Augmented Generation, https://arxiv.org/abs/2411.11895
• Matvey Arye, Avthar Sewrathan, 29 Oct 2024, Vector Databases Are the Wrong Abstraction, https://www.timescale.com/blog/vector-databases-are-the-wrong-abstraction/
• Michael Shen, Muhammad Umar, Kiwan Maeng, G. Edward Suh, Udit Gupta, 16 Dec 2024, Towards Understanding Systems Trade-offs in Retrieval-Augmented Generation Model Inference, https://arxiv.org/abs/2412.11854
• Derrick Quinn, Mohammad Nouri, Neel Patel, John Salihu, Alireza Salemi, Sukhan Lee, Hamed Zamani, Mohammad Alian, 14 Dec 2024, Accelerating Retrieval-Augmented Generation, https://arxiv.org/abs/2412.15246 (Speeding up vector databases using either approximate or exact nearest neighbor search.)
• Harvey Bower, 2024, Debugging RAG Pipelines: Best Practices for High-Performance LLMs, https://www.amazon.com/dp/B0DNWN5RB1
• Shige Liu, Zhifang Zeng, Li Chen, Adil Ainihaer, Arun Ramasami, Songting Chen, Yu Xu, Mingxi Wu, Jianguo Wang, 20 Jan 2025, TigerVector: Supporting Vector Search in Graph Databases for Advanced RAGs, https://arxiv.org/abs/2501.11216
• Bharani Subramaniam, 13 February 2025, Emerging Patterns in Building GenAI Products, https://martinfowler.com/articles/gen-ai-patterns/
• Vasilis Mageirakos, Bowen Wu, Gustavo Alonso, 3 Mar 2025, Cracking Vector Search Indexes, https://arxiv.org/abs/2503.01823
• Nitish Upreti, Krishnan Sundaram, Hari Sudan Sundar, Samer Boshra, Balachandar Perumalswamy, Shivam Atri, Martin Chisholm, Revti Raman Singh, Greg Yang, Subramanyam Pattipaka, Tamara Hass, Nitesh Dudhey, James Codella, Mark Hildebrand, Magdalen Manohar, Jack Moffitt, Haiyang Xu, Naren Datha, Suryansh Gupta, Ravishankar Krishnaswamy, Prashant Gupta, Abhishek Sahu, Ritika Mor, Santosh Kulkarni, Hemeswari Varada, Sudhanshu Barthwal, Amar Sagare, Dinesh Billa, Zishan Fu, Neil Deshpande, Shaun Cooper, Kevin Pilch, Simon Moreno, Aayush Kataria, Vipul Vishal, Harsha Vardhan Simhadri, 9 May 2025, Cost-Effective, Low Latency Vector Search with Azure Cosmos DB, https://arxiv.org/abs/2505.05885
• Adel Ammar, Anis Koubaa, Omer Nacar, Wadii Boulila, 13 May 2025, Optimizing Retrieval-Augmented Generation: Analysis of Hyperparameter Impact on Performance and Efficiency, https://arxiv.org/abs/2505.08445
• Leonardo Kuffo, Peter Boncz, 12 May 2025, Bang for the Buck: Vector Search on Cloud CPUs, https://arxiv.org/abs/2505.07621
• Yaoqi Chen, Jinkai Zhang, Baotong Lu, Qianxi Zhang, Chengruidong Zhang, Jingjia Luo, Di Liu, Huiqiang Jiang, Qi Chen, Jing Liu, Bailu Ding, Xiao Yan, Jiawei Jiang, Chen Chen, Mingxing Zhang, Yuqing Yang, Fan Yang, Mao Yang, 5 May 2025, RetroInfer: A Vector-Storage Approach for Scalable Long-Context LLM Inference, https://arxiv.org/abs/2505.02922
• Michael Ryaboy,· May 16, 2025, The Most Common Vector Search Mistake Is Costing Enterprises Hundreds of Thousands, https://medium.com/kx-systems/the-most-common-vector-search-mistake-is-costing-enterprises-hundreds-of-thousands-dd1ffd0b976d
• Jiongli Zhu, Yue Wang, Bailu Ding, Philip A. Bernstein, Vivek Narasayya, Surajit Chaudhuri, 28 Apr 2025, MINT: Multi-Vector Search Index Tuning, https://arxiv.org/abs/2504.20018
• Latent Space, Aug 20, 2025, "RAG is Dead, Context Engineering is King" — with Jeff Huber of Chroma: What actually matters in vector databases in 2025, why “modern search for AI” is different, and how to ship systems that don’t rot as context grows, https://www.latent.space/p/chroma
• Wei Huang, Anda Cheng, Zhao Zhang, Yinggui Wang, 1 Sep 2025, DPF-CM: A Data Processing Framework with Privacy-Preserving Vector Databases for Chinese Medical LLMs Training and Deployment, https://arxiv.org/abs/2509.01354
• Wenqi Jiang, 23 Oct 2025, RAG-Stack: Co-Optimizing RAG Quality and Performance From the Vector Database Perspective, https://arxiv.org/abs/2510.20296
• Harshil Vejendla, 27 Sep 2025, Drift-Adapter: A Practical Approach to Near Zero-Downtime Embedding Model Upgrades in Vector Databases, https://arxiv.org/abs/2509.23471
• Jiawei Xu, Chia Xin Liang, Ziqian Bi, Xiaoming Li, Danyang Zhang, Zhenyu Yu, Dec 2025, A Comprehensive Survey on Large Language Models: From Pre-training to Autonomous Agents, https://www.researchgate.net/profile/Ziqian_Bi/publication/399059225_A_Comprehensive_Survey_on_Large_Language_Models_From_Pre-training_to_Autonomous_Agents/links/694c94a07e61d05b5312836f/A-Comprehensive-Survey-on-Large-Language-Models-From-Pre-training-to-Autonomous-Agents.pdf
• David Spuler, Michael Sharpe, June 2025, Vector Databases, Chapter 8, "RAG Optimization: Accurate and Efficient LLM Applications", https://www.aussieai.com/book/rag-book-8-vector-databases
• David Spuler, Michael Sharpe, 2025, RAG Architectures, Chapter 15, in "Generative AI Applications", https://www.aussieai.com/book/ai-apps-book-15-rag-architectures
• David Spuler, March 2024, Chapter 6. Training, Fine-Tuning & RAG, in book "Generative AI in C++", https://www.aussieai.com/book/ch6-training-ft-rag
• David Spuler, March 2024, Chapter 29. Caching Optimizations, in book "Generative AI in C++", https://www.aussieai.com/book/ch29-caching
• David Spuler, March 2024, Generative AI in C++: Coding Transformers and LLMs, https://www.aussieai.com/book/toc PDF: https://www.aussieai.com/pdf/BOOK-Generative-AI-CPP-Spuler-2024.pdf
• Michal Sutter, May 10, 2026, Best Vector Databases in 2026: Pricing, Scale Limits, and Architecture Tradeoffs Across Nine Leading Systems, https://www.marktechpost.com/2026/05/10/best-vector-databases-in-2026-pricing-scale-limits-and-architecture-tradeoffs-across-nine-leading-systems/

Vector Database Optimizations

Research papers on vector databases:

• Dr. Ashish Bamania, Jun 18, 2024, Google’s New Algorithms Just Made Searching Vector Databases Faster Than Ever: A Deep Dive into how Google’s ScaNN and SOAR Search algorithms supercharge the performance of Vector Databases, https://levelup.gitconnected.com/googles-new-algorithms-just-made-searching-vector-databases-faster-than-ever-36073618d078
• James Jie Pan, Jianguo Wang, Guoliang Li, 21 Oct 2023, Survey of Vector Database Management Systems, https://arxiv.org/abs/2310.14021 https://link.springer.com/article/10.1007/s00778-024-00864-x
• Michael Shen, Muhammad Umar, Kiwan Maeng, G. Edward Suh, Udit Gupta, 16 Dec 2024, Towards Understanding Systems Trade-offs in Retrieval-Augmented Generation Model Inference, https://arxiv.org/abs/2412.11854
• Derrick Quinn, Mohammad Nouri, Neel Patel, John Salihu, Alireza Salemi, Sukhan Lee, Hamed Zamani, Mohammad Alian, 14 Dec 2024, Accelerating Retrieval-Augmented Generation, https://arxiv.org/abs/2412.15246 (Speeding up vector databases using either approximate or exact nearest neighbor search.)
• Harvey Bower, 2024, Debugging RAG Pipelines: Best Practices for High-Performance LLMs, https://www.amazon.com/dp/B0DNWN5RB1
• Shige Liu, Zhifang Zeng, Li Chen, Adil Ainihaer, Arun Ramasami, Songting Chen, Yu Xu, Mingxi Wu, Jianguo Wang, 20 Jan 2025, TigerVector: Supporting Vector Search in Graph Databases for Advanced RAGs, https://arxiv.org/abs/2501.11216
• Vasilis Mageirakos, Bowen Wu, Gustavo Alonso, 3 Mar 2025, Cracking Vector Search Indexes, https://arxiv.org/abs/2503.01823
• Nitish Upreti, Krishnan Sundaram, Hari Sudan Sundar, Samer Boshra, Balachandar Perumalswamy, Shivam Atri, Martin Chisholm, Revti Raman Singh, Greg Yang, Subramanyam Pattipaka, Tamara Hass, Nitesh Dudhey, James Codella, Mark Hildebrand, Magdalen Manohar, Jack Moffitt, Haiyang Xu, Naren Datha, Suryansh Gupta, Ravishankar Krishnaswamy, Prashant Gupta, Abhishek Sahu, Ritika Mor, Santosh Kulkarni, Hemeswari Varada, Sudhanshu Barthwal, Amar Sagare, Dinesh Billa, Zishan Fu, Neil Deshpande, Shaun Cooper, Kevin Pilch, Simon Moreno, Aayush Kataria, Vipul Vishal, Harsha Vardhan Simhadri, 9 May 2025, Cost-Effective, Low Latency Vector Search with Azure Cosmos DB, https://arxiv.org/abs/2505.05885
• Adel Ammar, Anis Koubaa, Omer Nacar, Wadii Boulila, 13 May 2025, Optimizing Retrieval-Augmented Generation: Analysis of Hyperparameter Impact on Performance and Efficiency, https://arxiv.org/abs/2505.08445
• Leonardo Kuffo, Peter Boncz, 12 May 2025, Bang for the Buck: Vector Search on Cloud CPUs, https://arxiv.org/abs/2505.07621
• Yaoqi Chen, Jinkai Zhang, Baotong Lu, Qianxi Zhang, Chengruidong Zhang, Jingjia Luo, Di Liu, Huiqiang Jiang, Qi Chen, Jing Liu, Bailu Ding, Xiao Yan, Jiawei Jiang, Chen Chen, Mingxing Zhang, Yuqing Yang, Fan Yang, Mao Yang, 5 May 2025, RetroInfer: A Vector-Storage Approach for Scalable Long-Context LLM Inference, https://arxiv.org/abs/2505.02922
• Michael Ryaboy,· May 16, 2025, The Most Common Vector Search Mistake Is Costing Enterprises Hundreds of Thousands, https://medium.com/kx-systems/the-most-common-vector-search-mistake-is-costing-enterprises-hundreds-of-thousands-dd1ffd0b976d
• Jiongli Zhu, Yue Wang, Bailu Ding, Philip A. Bernstein, Vivek Narasayya, Surajit Chaudhuri, 28 Apr 2025, MINT: Multi-Vector Search Index Tuning, https://arxiv.org/abs/2504.20018
• David Spuler, Michael Sharpe, June 2025, Vector Databases, Chapter 8, "RAG Optimization: Accurate and Efficient LLM Applications", https://www.aussieai.com/book/rag-book-8-vector-databases

Vector Database Caching

Research papers on the use of caching to optimize vector databases:

• Yikun Han, Chunjiang Liu, Pengfei Wang, 18 Oct 2023, A Comprehensive Survey on Vector Database: Storage and Retrieval Technique, Challenge, https://arxiv.org/abs/2310.11703
• Pere Martra, Aug 2024 (accessed), Implementing semantic cache to improve a RAG system with FAISS, https://huggingface.co/learn/cookbook/semantic_cache_chroma_vector_database
• Richmond Alake, Apoorva Joshi, Aug 14, 2024, Adding Semantic Caching and Memory to Your RAG Application Using MongoDB and LangChain, MongoDB, https://www.mongodb.com/developer/products/atlas/advanced-rag-langchain-mongodb/

Vector Search Optimizations

• Chips Ahoy Capital, Jul 02, 2024, Evolution of Databases in the World of AI Apps, https://chipsahoycapital.substack.com/p/evolution-of-databases-in-the-world
• Chirag Agrawal, Sep 20, 2024, Unlocking the Power of Efficient Vector Search in RAG Applications, https://pub.towardsai.net/unlocking-the-power-of-efficient-vector-search-in-rag-applications-c2e3a0c551d5
• Pierre-Emmanuel Mazaré, Gergely Szilvasy, Maria Lomeli, Francisco Massa, Naila Murray, Hervé Jégou, Matthijs Douze, 12 Feb 2025, Inference-time sparse attention with asymmetric indexing, https://arxiv.org/abs/2502.08246
• Nitish Upreti, Krishnan Sundaram, Hari Sudan Sundar, Samer Boshra, Balachandar Perumalswamy, Shivam Atri, Martin Chisholm, Revti Raman Singh, Greg Yang, Subramanyam Pattipaka, Tamara Hass, Nitesh Dudhey, James Codella, Mark Hildebrand, Magdalen Manohar, Jack Moffitt, Haiyang Xu, Naren Datha, Suryansh Gupta, Ravishankar Krishnaswamy, Prashant Gupta, Abhishek Sahu, Ritika Mor, Santosh Kulkarni, Hemeswari Varada, Sudhanshu Barthwal, Amar Sagare, Dinesh Billa, Zishan Fu, Neil Deshpande, Shaun Cooper, Kevin Pilch, Simon Moreno, Aayush Kataria, Vipul Vishal, Harsha Vardhan Simhadri, 9 May 2025, Cost-Effective, Low Latency Vector Search with Azure Cosmos DB, https://arxiv.org/abs/2505.05885
• Leonardo Kuffo, Peter Boncz, 12 May 2025, Bang for the Buck: Vector Search on Cloud CPUs, https://arxiv.org/abs/2505.07621
• Laxman Dhulipala, Majid Hadian, Rajesh Jayaram, Jason Lee, Vahab Mirrokni, 29 May 2024, MUVERA: Multi-Vector Retrieval via Fixed Dimensional Encodings, https://arxiv.org/abs/2405.19504 (Multi-vector search optimization.)
• Jiongli Zhu, Yue Wang, Bailu Ding, Philip A. Bernstein, Vivek Narasayya, Surajit Chaudhuri, 28 Apr 2025, MINT: Multi-Vector Search Index Tuning, https://arxiv.org/abs/2504.20018
• David Spuler, Michael Sharpe, June 2025, Vector Databases, Chapter 8, "RAG Optimization: Accurate and Efficient LLM Applications", https://www.aussieai.com/book/rag-book-8-vector-databases
• Nabaraj Subedi, Ahmed Abdelaty, Shivanand Venkanna Sheshappanavar, 9 Jun 2026, When More Documents Hurt RAG: Mitigating Vector Search Dilution with Domain-Scoped, Model-Agnostic Retrieval, https://arxiv.org/abs/2606.11350

AI Books from Aussie AI

The Sweetest Lesson: Your Brain Versus AI: new book on AI intelligence theory: Your brain is 50 times bigger than the best AI engines. Truly intelligent AI will require more compute! Another case of the bitter lesson? Maybe it's the opposite of that: the sweetest lesson. Get your copy from Amazon: The Sweetest Lesson

RAG Optimization: Accurate and Efficient LLM Applications: new book on RAG architectures: Smarter RAG Faster RAG Cheaper RAG Agentic RAG RAG reasoning Get your copy from Amazon: RAG Optimization

Generative AI Applications book: Deciding on your AI project Planning for success and safety Designs and LLM architectures Expediting development Implementation and deployment Get your copy from Amazon: Generative AI Applications

Generative AI programming book: Generative AI coding in C++ Transformer engine speedups LLM models Phone and desktop AI Code examples Research citations Get your copy from Amazon: Generative AI in C++

CUDA C++ Optimization book: Faster CUDA C++ kernels Optimization tools & techniques Compute optimization Memory optimization Get your copy from Amazon: CUDA C++ Optimization

CUDA C++ Debugging book: Debugging CUDA C++ kernels Tools & techniques Self-testing & reliability Common GPU kernel bugs Get your copy from Amazon: CUDA C++ Debugging

More AI Research

Read more about:

• Inference Optimizations
• Loop Optimizations
• Code Optimizations
• « Research Home