MoE Shared Experts

Research on MoE Shared Experts

Research papers include:

• Minghao Yang, Ren Togo, Guang Li, Takahiro Ogawa, Miki Haseyama, 1 Oct 2025, Adaptive Shared Experts with LoRA-Based Mixture of Experts for Multi-Task Learning, https://arxiv.org/abs/2510.00570
• Yuanhang Yang, Chaozheng Wang, Jing Li, 23 Oct 2025, UMoE: Unifying Attention and FFN with Shared Experts, https://arxiv.org/abs/2505.07260
• Cheng Li and Jiexiong Liu and Yixuan Chen and Jie ji, 5 Sep 2025, Dynamic Adaptive Shared Experts with Grouped Multi-Head Attention Mixture of Experts, https://arxiv.org/abs/2509.10530
• Sebastian Raschka, PhD, Dec 18, 2025 (updated), The Big LLM Architecture Comparison: From DeepSeek V3 to Mistral 3 Large: A Look At Modern LLM Architecture Design, https://magazine.sebastianraschka.com/p/the-big-llm-architecture-comparison
• Devansh, Apr 2026, Google’s Gemma 4 is Weirder than you Realize: The architecture matters more than the numbers. Here’s what Google actually built, https://machine-learning-made-simple.medium.com/googles-gemma-4-is-weirder-than-you-realize-17d00d95b0d5
• Sesame Disk, Apr 2026, LLM Architecture Gallery 2026: Top Model Designs Explained, https://sesamedisk.com/llm-architecture-gallery-2026/
• vLLM, Apr 2026, v0.20.0 Latest: vLLM v0.20.0 Highlights https://github.com/vllm-project/vllm/releases#vlm0200

More AI Research Topics

Read more about:

• 500+ LLM Inference Optimization Techniques
• What's Hot in LLM Inference Optimization in 2025?
• Inference Optimization Research
• « Research Home