I’d look for a job, not because AI is so important for development that you have to have it, but because your company doesn’t have enough revenue opportunities (or the leadership to find them) that employee productivity doesn’t matter to them.
Companies doing well and growing pass on crazy revenue opportunities because the ones they are focused on are even higher roi. They don’t get bogged down in incremental costs.
If they are cost cutting on dev tooling some bright boss is going to realize the real savings is in cutting down on devs. If you are an American dev team your health insurance costs smoke any amount of token spend.
With Anthropic's prices it soon will be cheper to hire real people then "hire" their models. If you use it for codding I'm pretty sure you will not be able to find any model which can be hosted on local machine and produce more or less acceptable results. Options - use something more cheaper, Deepseek for example, or if company big enough think about hosting some open source on you private DC with GPUs, I know that some companies develop their own solutions, but I don't have good enough expirience to tell how mach it may cost
My own company hired a young goon of a man to spearhead their AI initiative. Lots of smiles and arrogance from him. Fast forward 2 months and reality has hit. Weekly meetings asking for feedback draw blank stares as employees explain that Claude can’t do shit to help their workload. This kid is starting to sweat. I bet he’ll be gone by the summer. Hilarious.
Unfortunately at my company leads have no insight into employees claude code caps, and no one has ever complained until now. Apparently some people were basically running with insane caps on CC (25k+), if you asked for it you were approved. Which lead to some people doing insane things on CC for no purpose.
Just setting up better SOPs around using AI for coding is going to help them a ton. They can chalk off the sunk cost to a "learning phase", with now being the time to use the lesson learnt to formulate some future-looking standard operating procedures. No need to suddenly go cold-turkey on AI. My 2 cents.
Specifically: to explore your opensource options with compute limitations, ask the community at r/LocalLLaMA on reddit. That's where the current SOTA opensource text-to-text models live.
Yeah i was looking there earlier, its just we thankfully mostly have macbooks, but i recently found out new devs are getting the smaller 8gb ram macbooks as well. Which is going to be even more frusturating.
Since my team is mostly remote running LLM on a cluster in the office is not really viable short term.
This is totally going to suck, but here's one option I was just suggested a few mins ago: https://www.reddit.com/r/LocalLLaMA/comments/1th1mqx/comment... For context, I was asking about running anything OpenClaw-friendly on my RTX4060 8GB VRAM. I know yours is a more involved use-case, but there's still some optionality here.
Given our field we cannot really use anything not approved by management. Pretty much if it doesn't leave our machine we can use its just i don't find anything good. We even have some new devs on the macbook neos, and i can't even find anything for them.
I was considering having something run locally within out building but the time when something like that would be avaliable is not near term so i am trying to make the best of what i can do.
I’d look for a job, not because AI is so important for development that you have to have it, but because your company doesn’t have enough revenue opportunities (or the leadership to find them) that employee productivity doesn’t matter to them.
Companies doing well and growing pass on crazy revenue opportunities because the ones they are focused on are even higher roi. They don’t get bogged down in incremental costs.
If they are cost cutting on dev tooling some bright boss is going to realize the real savings is in cutting down on devs. If you are an American dev team your health insurance costs smoke any amount of token spend.
With Anthropic's prices it soon will be cheper to hire real people then "hire" their models. If you use it for codding I'm pretty sure you will not be able to find any model which can be hosted on local machine and produce more or less acceptable results. Options - use something more cheaper, Deepseek for example, or if company big enough think about hosting some open source on you private DC with GPUs, I know that some companies develop their own solutions, but I don't have good enough expirience to tell how mach it may cost
My own company hired a young goon of a man to spearhead their AI initiative. Lots of smiles and arrogance from him. Fast forward 2 months and reality has hit. Weekly meetings asking for feedback draw blank stares as employees explain that Claude can’t do shit to help their workload. This kid is starting to sweat. I bet he’ll be gone by the summer. Hilarious.
Unfortunately at my company leads have no insight into employees claude code caps, and no one has ever complained until now. Apparently some people were basically running with insane caps on CC (25k+), if you asked for it you were approved. Which lead to some people doing insane things on CC for no purpose.
Just setting up better SOPs around using AI for coding is going to help them a ton. They can chalk off the sunk cost to a "learning phase", with now being the time to use the lesson learnt to formulate some future-looking standard operating procedures. No need to suddenly go cold-turkey on AI. My 2 cents.
Long term approach is probably to use AI as a tool that boosts productivity
Specifically: to explore your opensource options with compute limitations, ask the community at r/LocalLLaMA on reddit. That's where the current SOTA opensource text-to-text models live.
Yeah i was looking there earlier, its just we thankfully mostly have macbooks, but i recently found out new devs are getting the smaller 8gb ram macbooks as well. Which is going to be even more frusturating.
Since my team is mostly remote running LLM on a cluster in the office is not really viable short term.
This is totally going to suck, but here's one option I was just suggested a few mins ago: https://www.reddit.com/r/LocalLLaMA/comments/1th1mqx/comment... For context, I was asking about running anything OpenClaw-friendly on my RTX4060 8GB VRAM. I know yours is a more involved use-case, but there's still some optionality here.
The 16GB of RAM will really limit you, what about trying OpenRouter and using the cheaper models such as Kimi instead of running them locally?
Given our field we cannot really use anything not approved by management. Pretty much if it doesn't leave our machine we can use its just i don't find anything good. We even have some new devs on the macbook neos, and i can't even find anything for them.
I was considering having something run locally within out building but the time when something like that would be avaliable is not near term so i am trying to make the best of what i can do.