ascend-tribe/pangu-pro-moe

Pangu-Pro-MoE 72B-A16B is a sparse large language model with 72 billion parameters and 16 billion activated parameters. It is based on the Grouped Mixture of Experts (MoGE) architecture, which groups experts during the expert selection phase and constrains tokens to activate an equal number of experts within each group, thereby achieving expert load balancing and significantly improving the model's deployment efficiency on the Ascend platform.