The readme has a lot of words but still does not clearly explain what this does or what you'd use it for.
The best I can gather is that this captures telemetry while you're running some computationally-expensive job, and checks that the resource usage roughly matches what it's supposed to be. But what does "Tamper-evident" mean? I assume you sign with your own private key? What exactly prevents the data from being faked?
> "A run of committed code, on attested hardware at time T, reading a model with hash m and a dataset with hash d, produced score S.” If d is a public benchmark, they also confirm it is the real benchmark, without you exposing it. The score is also cross-checked by the compute-cost certificate, which confirms the evaluation consumed the compute a real forward pass over the data requires.
They should really put that front and center. Still not a complete enough description, but something at least.
It didn't sound very convincing. It doesn't really prove anything other than the author signed the report. Unless the auditing is running from a secure enclave/core that the user has no access to, it can't do what is claimed on the tin.
There alternatives to a secure enclave like building in a risc0 VM that produces a mathematical proof of each computational step that is very expensive to generate and very cheap to verify. THAT is proof.
But this seems to just be collecting evidence and signing it?
This is exactly what it is, just slop. I do agree with the intent we need someway to verify and validate agentic coding output that doesn't rely on codereviews; but this project is just sloppy.
The readme has a lot of words but still does not clearly explain what this does or what you'd use it for.
The best I can gather is that this captures telemetry while you're running some computationally-expensive job, and checks that the resource usage roughly matches what it's supposed to be. But what does "Tamper-evident" mean? I assume you sign with your own private key? What exactly prevents the data from being faked?
Found in the docs:
https://kveritas.org/docs/benchmarks - What a verifier learns:
> "A run of committed code, on attested hardware at time T, reading a model with hash m and a dataset with hash d, produced score S.” If d is a public benchmark, they also confirm it is the real benchmark, without you exposing it. The score is also cross-checked by the compute-cost certificate, which confirms the evaluation consumed the compute a real forward pass over the data requires.
They should really put that front and center. Still not a complete enough description, but something at least.
It didn't sound very convincing. It doesn't really prove anything other than the author signed the report. Unless the auditing is running from a secure enclave/core that the user has no access to, it can't do what is claimed on the tin.
There alternatives to a secure enclave like building in a risc0 VM that produces a mathematical proof of each computational step that is very expensive to generate and very cheap to verify. THAT is proof.
But this seems to just be collecting evidence and signing it?
It's because it's Claude-slop, the wording betrays it.
Punchy phrase. It does A, never B. It Xs, Ys and Zs.
These are the simple tells but the general tell is what you describe - a lot of words that say nothing.
This is exactly what it is, just slop. I do agree with the intent we need someway to verify and validate agentic coding output that doesn't rely on codereviews; but this project is just sloppy.