2 points | by Ganeshan007 5 hours ago
5 comments
No. There is no evidence for AI profitability which can support this investment.
[flagged]
the useful split for me is interactive vs batch. keep a small model warm for private jobs and measure queue time, not just tokens/sec.
agreed queue time should be measured and added to the total wait time for the end user
No. There is no evidence for AI profitability which can support this investment.
[flagged]
the useful split for me is interactive vs batch. keep a small model warm for private jobs and measure queue time, not just tokens/sec.
agreed queue time should be measured and added to the total wait time for the end user
[flagged]