Tokenizer Mono

3 comments

• georgemandis 5 hours ago

  A font that visualizes how LLMs tokenize text

  [-]
  • fritzschopen 5 hours ago

    ok but is it bullet proof? Meaning does it work in real?

• SlightGenius 5 hours ago

  [dead]