Mixture of Experts is a neural network architecture that divides a large model into dozens of specialized sub-networks called experts, then uses a lightweight router to activate only the top-scoring ones for each input token rather than running the full network every time. This selective activation mimics how the brain routes tasks to specialized regions. The result: a model can hold far more total knowledge than a dense network of equivalent compute budget.
Every model in the top ten of the Artificial Analysis open-source intelligence leaderboard uses this architecture. That list includes DeepSeek-R1, Kimi K2 Thinking, Mistral Large 3, and OpenAI’s gpt-oss-120B. According to Wikipedia’s MoE architecture overview, the core appeal is that total model capacity scales independently from per-token compute. You can build a smarter model without making it slower to run.
The hardware angle accelerated adoption. NVIDIA reported in December 2025 that MoE models run 10x faster on its GB200 NVL72 systems versus the previous HGX H200 generation. That speed gain collapses cost per token to roughly one-tenth of prior pricing. For API providers, it’s a structural economics shift: faster inference plus lower cost per token reshapes what’s commercially viable at scale.
For developers, the practical upside is direct. You can deploy a model with 600 billion total parameters while activating only 40 billion per token. Training costs drop. Inference costs drop. The tradeoff is more engineering complexity around load balancing, but every major AI lab now runs an MoE program because the cost math is too large to ignore.
Inside an MoE model, a router sits at each relevant transformer layer. It scores every available expert for the current token, then forwards that token to the top-k highest scorers: usually the best 2 out of 8, or 2 out of 64, depending on model design. Only those experts compute an output. The rest skip that token entirely. The foundational 2017 sparse gating paper by Shazeer et al. demonstrated this approach scales to hundreds of billions of parameters without proportional compute overhead.
The router itself is a small neural network, trained alongside the experts via standard backpropagation. Its biggest failure mode is lopsided routing: if a handful of experts get all the traffic, the ignored ones never develop useful representations. Training fixes this with an auxiliary loss term that penalizes unequal expert utilization, forcing load to spread across the full pool.
Dense models run every parameter for every token. MoE runs a fraction. A 600B-parameter MoE model activating 40B parameters per token is computationally equivalent to a 40B dense model, but carries 15x more stored knowledge on standby. That gap between total parameters and active parameters is the entire value proposition.
The concept dates to 1991. Robert Jacobs, Michael Jordan, Steven Nowlan, and Geoffrey Hinton published “Adaptive Mixtures of Local Experts,” introducing the routing idea that all modern MoE systems descend from. For nearly two decades, it stayed academic — the hardware just wasn’t there. That changed in 2017 when Noam Shazeer and colleagues at Google published the sparsely-gated MoE paper, demonstrating practical scaling to 137 billion parameters. Google followed in 2022 with the Switch Transformer, simplifying routing enough for production deployment.
The commercial wave hit in late 2023. Mistral AI released Mixtral 8x7B, the first widely-used open MoE model, followed by DeepSeek-MoE in early 2024. DeepSeek-R1 launched in January 2025 and topped the open-source leaderboard, making the architecture impossible to sideline. By early 2026, the question had shifted from whether it works to how to configure expert count and routing strategy for each deployment.
A router network at each model layer scores all available expert sub-networks for the current token, then sends that token to the top-scoring ones (usually 2 out of 8 or 64). Only those experts compute; the rest are skipped entirely. Their outputs are weighted and combined before the result passes to the next layer. The router and experts train together end-to-end using standard backpropagation plus a load-balancing auxiliary loss.
It’s a large language model built on MoE architecture rather than a standard dense transformer. Mixtral 8x7B, DeepSeek-R1, and Kimi K2 Thinking are all examples. These models carry high total parameter counts but activate only a subset per token, making inference faster and cheaper than a dense model of equivalent output quality.
Speed, cost, and scale. MoE activates a fraction of total parameters per token, cutting inference compute directly. Capacity scales by adding experts without a proportional compute increase. The real tradeoff: all experts must stay loaded in memory simultaneously, and routing adds engineering overhead. For large-scale deployment, the cost advantage outweighs both concerns by a wide margin.
These terms appear frequently alongside MoE discussions and fill out the broader picture.