Simple, zero overhead way to compress model, KV cache via Low-Rank Decomposition

1 points | by thw20 an hour ago