Real-time LLM Inference on Standard GPUs (3k tokens/s per request)

7 points | by morgangiraud 6 hours ago