The architecture of open-weight LLMs: MoE, hybrid layers, ternary weights

16 points | by silasyl 2 days ago