Flattening Every Memory Peak in Long-Context Mixture-of-Experts Training

2 points | by gmays 11 hours ago