Huawei’s AI research team has unveiled a new training method designed to improve the performance and efficiency of large language models (LLMs), aiming to outpace major Chinese and global rivals. In a newly published research paper, the team introduced a “mixture of grouped experts” (MoGE) technique—an evolution of the mixture of experts (MoE) approach popularized by DeepSeek.
MoGE: A Balanced and Scalable AI Model Architecture
Unlike traditional MoE setups that can suffer from imbalanced computing loads across devices, Huawei’s MoGE architecture introduces expert grouping and balanced token assignment. This, according to the research, significantly enhances training efficiency and inference speed, especially when deployed on Huawei’s proprietary Ascend Neural Processing Units (NPUs).
The study claims MoGE offers better expert load balancing and supports efficient parallel execution, solving a key limitation that held back earlier MoE models. These advancements make the model more scalable and less dependent on ultra-high-end chips, an essential feature amid ongoing US sanctions on advanced chip imports.
Pangu Pro MoE
At the core of this breakthrough is Huawei’s Pangu Pro MoE model, boasting 72 billion parameters. It sits within the sub-100B parameter class but is already being benchmarked against major open-source models such as GLM-Z1-32B (by Z.ai) and Qwen3-32B (by Alibaba). According to Huawei researchers, Pangu Pro MoE outperforms both, establishing it as one of the strongest large models developed within China.
Huawei’s investment in large AI models dates back to 2021 with the launch of the Pangu series, tailored for sectors like telecom, weather forecasting, and drug discovery.
Huawei’s AI Hardware Strategy
MoGE’s efficiency gains are directly tied to Ascend NPUs, Huawei’s in-house designed AI chips. These chips enable massive parallelism, allowing Huawei to maintain performance parity with top-tier global hardware, even under US trade restrictions.
While Huawei was banned from importing advanced chips in 2019, the company and the broader Chinese tech ecosystem have responded by doubling down on local innovation. Recently, Nvidia CEO Jensen Huang acknowledged that the performance gap between Chinese and US chips is narrowing, with Huawei’s latest chips reportedly rivaling the Nvidia H200.
Conclusion
Huawei’s MoGE breakthrough highlights how Chinese AI firms are adapting and innovating under external pressure. By combining novel AI model design with in-house chip engineering, Huawei is not only staying competitive but also challenging the dominance of global LLM leaders.
With the AI race accelerating and geopolitical dynamics intensifying, Huawei’s Pangu Pro MoE and MoGE architecture could set a new benchmark for efficient, scalable AI training in restricted environments.
About The Author
