When DeepSeek published the training costs for V3 — $5.576 million in compute for a 671-billion-parameter model — the AI industry did not know quite how to react. The figure was simultaneously too small to be credible and too large to dismiss. The architecture behind it, a Mixture of Experts (MoE) design that activates only 37 billion parameters per forward pass, represents not an incremental improvement but a genuine inflection point in how frontier AI models are built and deployed.
Understanding why requires unpacking what MoE actually changes — and what it does not.
The Dense Model Problem
Traditional transformer architectures — GPT-4, Claude 2, Llama 2 in its full form — are “dense” models. Every parameter participates in processing every token. A 70-billion-parameter dense model uses all 70 billion parameters for every word it generates. This creates a clean scaling relationship: double the parameters, roughly double the compute per inference, with predictable (if diminishing) returns on capability.
The problem with dense scaling is that it is brutally expensive at inference time. A 100-billion-parameter dense model requires roughly 200GB of GPU memory just to load the weights, before processing a single token. Running it at scale demands either expensive high-memory GPUs or complex tensor parallelism across multiple chips. The economics work for cloud providers with massive infrastructure investments; they do not work for most organizations.
What MoE Changes
Mixture of Experts solves this by making parameter usage conditional. The model is divided into “experts” — specialized subnetworks — and a “router” that decides, for each token, which small subset of experts to activate. DeepSeek V3 has 671 billion total parameters organized into experts, but the router activates only 37 billion of them per token. The rest of the parameters exist in memory but are not computing anything for that inference.
This creates a counterintuitive situation: a 671-billion-parameter model that computes like a 37-billion-parameter model. The capability comes from the total parameter count (which the router can draw on across a full context); the inference cost comes from the active parameter count. DeepSeek V3 benchmarks comparably to GPT-4o on many tasks while costing roughly one-seventh as much per token to run.
The training cost advantage is similarly significant. Dense model training requires every parameter to receive gradient updates at every step. MoE training only updates the activated experts for each token, which reduces the compute per step. The $5.57 million figure is remarkable not because it is cheap in absolute terms — few organizations can spend that casually — but because it is roughly 10-15x cheaper than equivalent dense model training. The next generation of MoE models, trained with this cost structure, will be built by a much wider set of actors than could afford dense frontier training.
The Deployment Arithmetic
For operators deploying models, the MoE arithmetic changes the feasibility calculation for on-premises deployment. A 671B MoE model at 4-bit quantization requires roughly 340GB of GPU memory to load the full weights, but only needs memory bandwidth for the 37B active parameters during inference. This makes it deployable on configurations like eight H100 80GB GPUs — expensive, but within reach of enterprises with serious AI ambitions, not just the cloud hyperscalers.
Compare this to running a dense 70B model at the same quality level: the dense model would require the same or more memory for comparable capability, with worse throughput because all 70B parameters participate in every token generation. MoE’s sparse activation pattern can be exploited to improve throughput — idle experts mean idle compute that can be used for batch processing or parallel requests.
What MoE Does Not Fix
Mixture of Experts has real limitations that enthusiasts often elide. Expert load balancing is a persistent challenge: if the router consistently sends certain token types to the same experts, those experts become bottlenecks while others sit idle. DeepSeek’s training process includes auxiliary losses to encourage balanced expert utilization, but load imbalance at inference time, particularly for specialized domains, remains a practical concern.
Memory bandwidth, not compute, is often the actual bottleneck in LLM inference. Even though MoE models activate fewer parameters per token, loading those parameters from GPU memory still dominates inference latency for small batch sizes. The advantage of MoE shows up most clearly at large batch sizes, where the compute savings from sparse activation outweigh the memory bandwidth overhead.
Training stability is harder with MoE. The router can collapse — learning to ignore most experts and route everything to a few — or oscillate, causing experts to specialize and unspecialize in ways that destabilize the gradient landscape. DeepSeek’s training required careful tuning of auxiliary losses and learning rate schedules that are not yet well-characterized in published literature. Replicating their results is not straightforward.
The Competitive Landscape Shift
The more significant implication of DeepSeek V3 is not the architecture itself — MoE has existed since the 2017 “Outrageously Large Neural Networks” paper — but the demonstration that aggressive MoE scaling can match dense frontier models at a fraction of the training cost. This changes who can participate in frontier AI development.
For the past three years, the economics of frontier AI strongly favored organizations with access to thousands of high-end GPUs and billions of dollars in training budget: OpenAI, Google, Anthropic, Meta. DeepSeek V3 demonstrates that a well-resourced but not hyperscaler-scale team can reach frontier capability with the right architectural choices. The moat of compute access is narrowing.
This has downstream effects on the enterprise AI market. If frontier-quality models can be trained and deployed at MoE efficiency levels, the price pressure on API inference will intensify. It also accelerates the development of specialized expert models — MoE architectures naturally support fine-tuning specific experts for domain specialization without catastrophic forgetting of general capabilities.
The efficiency inflection point is real. The question is not whether MoE will become the dominant architecture for large-scale AI — it likely will — but how quickly the tooling, training recipes, and deployment infrastructure catch up to support it at the scale the market is demanding.