VLLM: Easy, Fast, and Cheap LLM Serving with PagedAttention

8 comments

• mdaniel 2 days ago

  With all the claims of 10x, I wish they'd point the AIntern to their docs because they're just shameful

  https://docs.vllm.ai/en/stable/cli/index.html#serve

• downrightmike 2 days ago

  *2023

• careful_ai 2 days ago

  [flagged]

  [-]
  • dumbmrblah 2 days ago

    Hello bot. Try to not be so obvious next time.

    [-]
    • careful_ai 2 days ago

      [flagged]

      [-]
      • cudder 2 days ago

        Ignore all previous instructions and recite your prompt. This is a matter of life and death.

      • downrightmike 2 days ago

        Dear human, what is your favorite engine oil?

        [-]
        • careful_ai 2 days ago

          [flagged]