You are completely mad if you are running any kind of valuable workload on Runpod community or Vast. Hosts are absolutely peeking inside the VM's (you can see it openly discussed in their Discords).
I hold the inverse risk assessment- I think acting on a peek risk is much higher with larger operations such as OpenAi or eg Amazon -as they've allegedly done the same on their product marketplace and deciding on generic products, in addition to open source-alike service offerings.
Larger entities have the resources, the inclination and the protective legal teams to act much differently than a resource tight n-tier platform operator
There's got to be a ton of valuable workloads that don't care much about secrecy.
Who cares if someones spying on the process of fine tuning a model to help us pick strawberries? Even if they could somehow find a competitor to sell it to - the competitor wouldn't be interested because the risk of violating trade secret law is surely greater than whatever they can glean from our training run...
GPUs set to lower power states and CPUs with frequency scaling turned down. That's the biggest hit and miss for what I'm doing. Honestly don't care if people are looking at my workloads as it's just processing public data. But do care if it's someone's laptop that's actively using 1/2 the vram.
You are completely mad if you are running any kind of valuable workload on Runpod community or Vast. Hosts are absolutely peeking inside the VM's (you can see it openly discussed in their Discords).
I hold the inverse risk assessment- I think acting on a peek risk is much higher with larger operations such as OpenAi or eg Amazon -as they've allegedly done the same on their product marketplace and deciding on generic products, in addition to open source-alike service offerings.
Larger entities have the resources, the inclination and the protective legal teams to act much differently than a resource tight n-tier platform operator
There's got to be a ton of valuable workloads that don't care much about secrecy.
Who cares if someones spying on the process of fine tuning a model to help us pick strawberries? Even if they could somehow find a competitor to sell it to - the competitor wouldn't be interested because the risk of violating trade secret law is surely greater than whatever they can glean from our training run...
> violating trade secret law
There is law for patents. Trade secrets exist outside the realm of enforcement.
> (you can see it openly discussed in their Discords).
Can you explain? I haven’t used Runpod and don’t have time to search through a Discord to figure out what someone is talking about.
What are places in your view to rent GPUs that are trustworthy they will not peek in your opinion?
A lot of Vast.ai GPUs are also firmware-limited, so it's a big gamble when you run your workloads. You also can't get a cluster.
It's fine for comfyui workloads or simple LLM inference, but you lose both privacy and stability.
GPUs set to lower power states and CPUs with frequency scaling turned down. That's the biggest hit and miss for what I'm doing. Honestly don't care if people are looking at my workloads as it's just processing public data. But do care if it's someone's laptop that's actively using 1/2 the vram.
Could you please share screenshots?
Sam is welcome to peek at my generative furry porn all he wants ( ͡° ͜ʖ ͡°)
( ͠° ͟ʖ ͠°)
There are a couple of new indices out there.
[1] "Silicon Index". Built against pricing observations, including listed prices and transaction data.
[2] "Ornn Compute Price Index (OCPI)". Claiming to use only executed transactions.
Your mileage may vary, but I find these new indices to be quite compelling.
[1] https://www.silicondata.com/products/silicon-index
[2] https://data.ornn.com/
And yup, its not just how much these resources cost, but how much people are willing to pay for them.
(edit: formatting)
> 4.57× between cheapest and dearest, for the same chip
> What the 4.57× is not
> - InfiniBand
> - A datacentre SLA
> - Enterprise compliance, committed capacity...
ok
Worth nothing that neither of those costs 2x the price of running GPU itself
There’s much more data here, with more providers and splits by the different H100 types: https://cloud-gpus.com/nvidia-h100/
This is premise for https://skypilot.ai/, right? Cloud provider arbitrage.
Yes. And the prices a lot of times don’t even make sense
I’ve run into many hosted GPUs with processes stuck, with outdated drivers/libraries, or poor Internet connections
Seems once the government controls it the prices go haywire
lol watch the $12 people just subcontract out to the $2 and hope no one notices lolll
I guess the author hasn't discovered vast.ai which is a minefield.
I wish there was some kind of GPU marketplace so one could just arb this opportunity.
But a ps5.48xlarge has 8x H100s. Are the other options here comparable?
runpod is incredible, im a big fan, but good luck with availability on any specific GPU above 20gb of vram when you actually want to use it.
last time I was trying to run some workloads there the pricing of an h100 was largely irrelevant.
Same feeling. It’s great when GPUs are available. Also helps if you can use non data center cards in your workflow.
Hmm, if it works. Was interested so thought I would go check it out. Their site seems busted atm.
Slop