Aussie AI Blog
-
by David Spuler, Ph.D.
Latest Blog Articles
- New OpenAI Inference Optimization is Precalculation
- LLM Inference Optimization: State-of-the-Art Research, new book by David Spuler, table of contents, buy on Amazon.
- Early Exit as a Training Optimization
- Structured Decoding
- Training versus Inference Optimization
- Why is Training Loss Calculated Per Batch?
- LLM Training Optimization
- What is Prefill?
- Top-K Vector Theory
- Layerwise Pipelined Overlapping of Prefill and Decode
- FFN Fusion with Tiled Pipelined RELU
- FFN Fusion Optimizations with Piecewise Linear Approximations
- LLM Attention and FFN Optimization are Opposites
- Scaling your API Wrapper Application
- Vector dot product add-as-integer optimization
- AGI might require higher precision FP32 or FP64
- Free PDF download versions of several AI and C++ books now released
- Promising LLM Inference Optimization Research
- CUDA C++ Job Interview Questions
- The Sweetest Lesson: Your Brain Versus AI — AI intelligence, AGI, and why human brains are 50 times bigger than frontier LLMs.
- RAG Optimization — LLM inference optimization for RAG architectures.
- Vector Dot Product Optimization in C++ with Instruction-Level Parallelism
- C++ Low Latency Book
List of Lists
- 600+ Low-Latency and Efficiency C++ Techniques
- 500+ Techniques for LLM Inference Optimization
- LLM Training Optimization
- 200+ CUDA C++ Optimization Techniques
- 100 C++ Memory Safety Techniques
- 100+ AI Smartness Techniques
Most Popular
- Free PDF download versions of several AI and C++ books now released
- PDF: Generative AI in C++ Book
- 500+ Techniques for LLM Inference Optimization
Low Latency C++ Blog Articles
- C++ Low Latency Book
- False Sharing and Cache Line Sizes in Multithreading
- Overview of C++ Multithreading Optimizations
- Low Latency Programming
CUDA C++ Efficiency Articles
- CUDA C++ Optimization
- CUDA Memory Coalescing Optimizations
- CUDA GPU Thread Divergence
- Fused and Shared Epilogues
- Rubin and Feynman Optimizations
- Branchless Coding Tricks
- Hopper and Blackwell Optimizations
- Grace CPU Optimizations
- CUDA C++ BF16x9 Emulation in Blackwell
CUDA C++ Safety Articles
C++ Safety and Debugging Articles
- DIY Preventive C++ Memory Safety
- Canary Values & Redzones for Memory-Safe C++
- User-After-Free Memory Errors in C++
- Array Bounds Violations and Memory Safe C++
- Poisoning Memory Blocks for Safer C++
- Uninitialized Memory Safety in C++
- DIY Memory Safety in C++
- Memory Safe C++ Library Functions
- Smart Stack Buffers for Memory Safe C++
- Safe C++ Text Buffers with snprintf
- Safe C++ Standard and Memory Safety Book
Aussie AI Book Releases
- C++ Low Latency Book
- Generative AI Applications: Planning, Design, and Implementation
- CUDA C++ Optimization
- Debugging CUDA C++ Kernels
- Safe C++ Standard and Memory Safety Book
- Generative AI in C++: Coding Transformers and LLMs
March 2025 Blog Articles
- C++ Low Latency Book
- False Sharing and Cache Line Sizes in Multithreading
- Overview of C++ Multithreading Optimizations
- What's Hot in LLM Inference Optimization in 2025?
- AI Research by Country
- What's New in Speculative Decoding?
February 2025 Blog Articles
- DeepSeek is Good for NVIDIA and the AI Industry
- DeepSeek Upends Progress in Reasoning and AGI
- Low Latency Programming
January 2025 Blog Articles
- Debugging OpenAI Node.js API Wrappers
- Chain-of-Thought Efficiency Optimization
- Reasoning Decoding Algorithms
- Reasoning Inference Optimization
December 2024 Blog Articles
- AI Hitting the Wall?
- Reasoning is the New AI Middleware
- Humans are the Top Layer of the AI Stack
- The AI Application Layer
- Consumer vs Enterprise AI
November 2024 Blog Articles
- DIY Preventive C++ Memory Safety
- Canary Values & Redzones for Memory-Safe C++
- User-After-Free Memory Errors in C++
- Array Bounds Violations and Memory Safe C++
- Poisoning Memory Blocks for Safer C++
- Uninitialized Memory Safety in C++
- DIY Memory Safety in C++
October 2024 Blog Articles
- CUDA C++ Floating Point Exceptions
- Memory Safe C++ Library Functions
- Smart Stack Buffers for Memory Safe C++
- Safe C++ Text Buffers with snprintf
- Weight Clustering Needs a Refresh
- Generalizing Prefix KV Caching to RAG Chunks
- RAG Optimization via Caching
- CUDA Memory Coalescing Optimizations
- CUDA GPU Thread Divergence
September 2024 Blog Articles
- Deciding on Your AI Business Project
- Planning Your AI Business Project
- CUDA Basic C++ Programming Mistakes
- 500+ Techniques for LLM Inference Optimization
August 2024 Blog Articles
- State-of-the-Art LLM Backends
- Hot Inference Optimization Techniques
- Inference Optimization Research Ideas
- Sequential Speculative Decoding
- Generative AI Textbook Free Online
Free AI and C++ Books
Generative AI programming books:
- The Sweetest Lesson: Your Brain Versus AI, November 2025: full text online, free PDF available
- RAG Optimization: Accurate and Efficient LLM Applications, June 2025: full text online, free PDF available
- Generative AI Applications: Planning, Design and Implementation, November 2024: full text online, free PDF available
- Generative AI in C++ (Spuler, March 2024): full text online, free PDF available, table of contents, bonus materials, reference lists, source code
CUDA C++ GPU Programming Books:
- CUDA C++ Optimization: Coding Faster GPU Kernels, July 2024: full text online, bonus materials, free PDF available
- CUDA C++ Debugging: Safer GPU Kernel Programming, July 2024: full text online, free PDF available
Modern C++ Programming Books
- C++ AVX Optimization: CPU SIMD Vectorization, 2025: full text online, free PDF available
- C++ Ultra-Low Latency: Multithreading and Low-Level Optimizations, 2025: full text online, free PDF available
- Advanced C++ Memory Techniques: Efficiency and Safety, 2025: full text online, free PDF available
- Efficient C++ Multithreading: Modern Concurrency Optimization, 2025: free PDF available
- Efficient Modern C++ Data Structures: Container and Algorithm Optimizations, 2025: free PDF available
- C++ Low Latency: Multithreading and Hotpath Optimizations, 2025: free PDF available
- Safe C++: Fixing Memory Safety Issues, Oct 2024: full text online, free PDF available
More AI Research Topics
Read more about: