Speculative Decoding: The Free Speed Toggle Your Local LLM Is Probably Not Using

3 points | by ermantrout 11 hours ago