Star Attention

Star attention is an LLM attention optimization that reduces the cost of attention matrix computations on long token sequences. It is a type of "linear attention" that uses an approximation with "block sparsity" to avoid the quadratic complexity of LLM full attention algorithms. This helps the LLM known which blocks of tokens to pay attention to without needing to compute the QKV tensors for every single token.

Related LLM research areas for long context optimization of the attention methods include:

• Attention optimization
• Local attention
• Linear attention
• Sparse attention
• Multi-Head Attention (MHA)
• Muti-Query Attention (MQA)
• Group-Query Attention (GQA)
• Flash attention
• Paged attention

Other topics in attention research:

• Low-rank matrix attention
• Medusa attention
• Block attention
• Cross attention
• Fused head attention
• Hybrid local-global attention
• FFT attention
• QKV computation optimizations
• Additive attention
• Multiplicative attention
• Graph attention
• Chunked attention
• Attention sink
• Attention steering
• Bilinear attention
• Attention-free methods
• Mixture-of-Heads (MOH) Attention (MoE+MHA)

Research on Star Attention

• Shantanu Acharya, Fei Jia, Boris Ginsburg, 26 Nov 2024, Star Attention: Efficient LLM Inference over Long Sequences, https://arxiv.org/abs/2411.17116
• Aswin Ak, November 28, 2024, NVIDIA AI Research Unveils ‘Star Attention’: A Novel AI Algorithm for Efficient LLM Long-Context Inference, https://www.marktechpost.com/2024/11/28/nvidia-ai-research-unveils-star-attention-a-novel-ai-algorithm-for-efficient-llm-long-context-inference/
• Wenyuan Yang, Zhongxu Li, Qihan He, 2025, Star-PMFI: Star-attention and pyramid multi-scale feature integration network for small object detection in drone imagery, Journal of Visual Communication and Image Representation, 104479, ISSN 1047-3203, https://doi.org/10.1016/j.jvcir.2025.104479 https://www.sciencedirect.com/science/article/abs/pii/S1047320325000938

AI Books from Aussie AI

The Sweetest Lesson: Your Brain Versus AI: new book on AI intelligence theory: Your brain is 50 times bigger than the best AI engines. Truly intelligent AI will require more compute! Another case of the bitter lesson? Maybe it's the opposite of that: the sweetest lesson. Get your copy from Amazon: The Sweetest Lesson

RAG Optimization: Accurate and Efficient LLM Applications: new book on RAG architectures: Smarter RAG Faster RAG Cheaper RAG Agentic RAG RAG reasoning Get your copy from Amazon: RAG Optimization

Generative AI Applications book: Deciding on your AI project Planning for success and safety Designs and LLM architectures Expediting development Implementation and deployment Get your copy from Amazon: Generative AI Applications

Generative AI programming book: Generative AI coding in C++ Transformer engine speedups LLM models Phone and desktop AI Code examples Research citations Get your copy from Amazon: Generative AI in C++

CUDA C++ Optimization book: Faster CUDA C++ kernels Optimization tools & techniques Compute optimization Memory optimization Get your copy from Amazon: CUDA C++ Optimization

CUDA C++ Debugging book: Debugging CUDA C++ kernels Tools & techniques Self-testing & reliability Common GPU kernel bugs Get your copy from Amazon: CUDA C++ Debugging

More AI Research

Read more about:

• Inference Optimizations
• Loop Optimizations
• Code Optimizations
• « Research Home