Qwen3 Next 80B A3B Instruct

Qwen3-Next employs a highly sparse MoE architecture: 80 billion total parameters, but only approximately 3 billion are activated per inference step. Experiments show that, with global load balancing, increasing the total number of expert parameters while keeping the number of activated experts fixed steadily reduces training loss. Compared to Qwen3’s MoE (128 total experts, 8 routed), Qwen3-Next expands to 512 total experts, combining 10 routed experts and 1 shared expert—maximizing resource usage without compromising performance. The Qwen3-Next-80B-A3B-Instruct performs comparably to our flagship model Qwen3-235B-A22B-Instruct-2507, and demonstrates clear advantages in tasks requiring ultra-long context (up to 256K tokens).