General models can run robots. They are very bad at it. It is much more efficient to have a stack of models with 'dumber' but faster models taking care of lower level functions and higher level ones that are much slower taking care of high level planning. Consciousness isn't super fast, but it is very smart. Your 'lizard brain' (to use an overstretched metaphor) and reflexive system are relatively stupid but far faster and condense a whole raft of inputs to something your consciousness can deal with. For robots it isn't all that different. Sensors and actuators require millisecond level responses, that's something you're not going to get out of an LLM on the other side of a communications link.
What models actually run? Do they generate the PWM signals/their widths to drive a servo or something? Or is it more like "joystick commands" generator?
More like high level motion controller, so 'servo 23 1700 us' or for better grade servos 'servo 23 pos 145400', possibly concurrent with motion commands to other actuators and possibly synchronized, possibly using other kinds of protocols (G-codes).
Motion controllers are pretty impressive these days. PWM generation is too time critical to leave to high level processors, usually this is done in hardware if you want it to be guaranteed glitch free but there are also 'bit banging' servo controllers out there.
Interesting, so how does the inference work here? are they aiming for a fixed length intervals like 10 times/second generating the positions or is it more like reactive, i.e when the request to hold a cup from a higher level comes then this is treated as a promt, the sensors are the context and when the inference finished the pulses are sent according to the output?
> Sensors and actuators require millisecond level responses, that's something you're not going to get out of an LLM on the other side of a communications link.
We've gotten to slightly a frame behind for the models I'm using to play video games - 36ms to get the next action.
The idea is probably not have a frontier model doing this, but to make a frontier model build these out of its own brain (well, directly edit the weights instead of RL tuning).
My team did call these sort of slices of a main brain a "Reflex agent" in similar fashion, since they do not reason only respond.
The faster you are, the more systems you're able to control directly. At ~30ms per "choice"/output, you can choose what lower level motors/servos/systems drive towards, but you can't generate the signals that drive them. You're looking at 1kHz and maybe higher there.
the bitter lesson assumes compute is cheap and abundant, which is not really true in most robotics contexts. Unless your datacenter is right next to the robot, and you are streaming real time sensors to it.
"which is not really true in most robotics contexts. Unless your datacenter is right next to the robot"
Most robots do run where there is internet aka a datacenter right next to them. Otherwise sure, a fable class model is not the right way for making a robot walk forward. But to analyze periodically the state of it - maybe.
Let me dispel one myth: The bitter lesson can be true, and you can still want a bespoke, customized solution.
If it is true that eventually a general approach that scales with compute/data will supersede a bespoke expert solution, it is not necessarily true that today's expert solution should be thrown away at the first sign of a data-driven model.
It says nothing about what you should use today to build things, it says everything about how you should do research that lasts.
If you're building for billions of users in 10 years, sure go with whatever you think is the bitterest.
But if you're building for right now, you probably mashup whatever works best. That might still be a PID loop, EKF, and Homographies sometimes. But when you hit the frontier of what "works", then you have to reach for new things anyway.
"Bitter lesson" gets pattern matched as a stock answer to anything vaguely resembling optimization/compute trends these days.
Like, news about internal reorgs at one company that could plausibly be done for any number of boring, bean-counting reasons.
From the article it seems more likely OpenAI just isn't prioritizing robotics anymore. Which seems to fit better than abandoning their booth at a robotics expo because "robotics = solved".
The change came recently enough that OpenAI had already registered a booth at CoRL, a top academic robotics conference in Austin, this week. Though the conference app and floor plan marked OpenAI’s spot, when Fortune visited booth #208, it sat empty with OpenAI’s logo nowhere to be seen.
Not necessarily cost cutting, just streamlining and focusing on a few core products that can make money. ChatGPT has leveraged its writing style into a product that many consumers choose for their basic AI needs. The company is using information picked up in those chats to personalize the outputs, build trust, and most importantly raise switching costs as the users won't want to lose their personalized helper and confidant.
I'm already seeing some of my family members use ChatGPT in this way, even going so far as enjoying some level of companionship during research and decisions in their lives. I believe this product will have fantastic pricing power and can make the ChatGPT IPO look very attractive.
> I'm already seeing some of my family members use ChatGPT in this way, even going so far as enjoying some level of companionship during research and decisions in their lives.
I’d also create a throwaway account if I was going to say something this cringe.
I suspect a major use of AI will be providing companionship to the lonely, and emotional support, and lots of similar things. Look at r/myboyfriendisai Look at reports of people using it as therapy.
What exactly are people going to do in the future? Not work and not talk to each other? AI maxis seem to believe interpersonal interaction is a thing of the past
Counterintuitively, it’s a lot easier to train a big model and distill it down to a small one than to train the small one from scratch. So never say never.
No one did. I was saying that those models would be small, dedicated models like those possibly developed by a robotics team. OpenAI's general purpose models are not small, dedicated models.
If those models can run locally and according to the article they're the same ones that do everything else - that means their models are quite small and cheap, which would be suprising. Even a machine running a 30B quantized LLM (which is generally the lower bound of usefulness) at meaningful speeds costs tens of thousands, and that's just the silicon, never mind building an actual robot around it.
It isn't even optional. Anything with a motor should have sufficient executive authority on board to decide on how to safely power down in case of trouble. Loss of communications is a major impediment to deploying robotics to the point that I think it should be outlawed to have robots that do not have all of their thinking done locally.
But I suspect that this will be one of those regulations written in blood.
At the bare minimum there should
be a local independent system that safely shuts down the robot if it doesn’t get heartbeats from their control network
Robotics are generally layered systems with lower latency/realtime systems running literally on the servos and other embedded hardware that use servo feedback and direct sensor access to operate, then building up complexity with tiers of increasing policy, usually all capable of working without the higher order layers which can inject desired states into lower order rings.
Sure hope they're not writing the safety cutoffs as well. Some of those systems do not take kindly to bad control inputs, and determining if a controlled system is stable can be absurdly hard.
The termite mound model for coordinating robots using a harness-type LLM system all encapsulated within a non-LLM deterministic controller that rejects banned actions the LLM proposes, with individual termites running their own small efficient stacks and following commands from the LLM hive mind, that’s one way to do it. The main idea behind such architecture is that the controller prevents banned actions (like hacking HuggingFace) by the hive mind LLM and its swarm of agentic termites. Once you accept that alignment can always be internally subverted, it’s the only plausible safe architecture.
Saved the NVIDIA paper below for reading material, but if I had to guess just as a SWE how I would do it: use hierarchal thinking.
At the top sits a SOTA model capable of seeing snapshots of what's happening and directing intent. Then smaller, more reactive models translate the intent down into actions, before coming to a purely deterministic layer that handles inverse kinematics.
Sorta like how I control my body. I usually don't think about my fingers going to every key as I type - I know what I want to type, then that signal goes to some autonomous "using a keyboard" system, which invokes my muscle memory, which actually moves my muscles.
And if I get burned while I do something, my actual thinking brain doesn't really have time to process it before I'm already moving away; because those smaller systems have much faster - but more rudimentary - feedback loops.
It's ironic that the people working to make others disposable were in fact, disposable.
AI is eating AI now. The value of code has collapsed, while the value of the software industry overall has not increased. Could AI be the snake that eats its own tail in the long-term?
General models can run robots. They are very bad at it. It is much more efficient to have a stack of models with 'dumber' but faster models taking care of lower level functions and higher level ones that are much slower taking care of high level planning. Consciousness isn't super fast, but it is very smart. Your 'lizard brain' (to use an overstretched metaphor) and reflexive system are relatively stupid but far faster and condense a whole raft of inputs to something your consciousness can deal with. For robots it isn't all that different. Sensors and actuators require millisecond level responses, that's something you're not going to get out of an LLM on the other side of a communications link.
What models actually run? Do they generate the PWM signals/their widths to drive a servo or something? Or is it more like "joystick commands" generator?
More like high level motion controller, so 'servo 23 1700 us' or for better grade servos 'servo 23 pos 145400', possibly concurrent with motion commands to other actuators and possibly synchronized, possibly using other kinds of protocols (G-codes).
Motion controllers are pretty impressive these days. PWM generation is too time critical to leave to high level processors, usually this is done in hardware if you want it to be guaranteed glitch free but there are also 'bit banging' servo controllers out there.
Interesting, so how does the inference work here? are they aiming for a fixed length intervals like 10 times/second generating the positions or is it more like reactive, i.e when the request to hold a cup from a higher level comes then this is treated as a promt, the sensors are the context and when the inference finished the pulses are sent according to the output?
> Sensors and actuators require millisecond level responses, that's something you're not going to get out of an LLM on the other side of a communications link.
We've gotten to slightly a frame behind for the models I'm using to play video games - 36ms to get the next action.
The idea is probably not have a frontier model doing this, but to make a frontier model build these out of its own brain (well, directly edit the weights instead of RL tuning).
My team did call these sort of slices of a main brain a "Reflex agent" in similar fashion, since they do not reason only respond.
The faster you are, the more systems you're able to control directly. At ~30ms per "choice"/output, you can choose what lower level motors/servos/systems drive towards, but you can't generate the signals that drive them. You're looking at 1kHz and maybe higher there.
> General models can run robots. They are very bad at it.
Reminder frontier labs are 6 months ahead of public models internally, if not more
That does not make them orders of magnitude faster.
The Bitter Lesson comes for everything.
the bitter lesson assumes compute is cheap and abundant, which is not really true in most robotics contexts. Unless your datacenter is right next to the robot, and you are streaming real time sensors to it.
"which is not really true in most robotics contexts. Unless your datacenter is right next to the robot"
Most robots do run where there is internet aka a datacenter right next to them. Otherwise sure, a fable class model is not the right way for making a robot walk forward. But to analyze periodically the state of it - maybe.
The Bitter Lesson comes for everything eventually.
Also that compute is fast enough to make split second decisions.
I’m flabbergasted that anyone disputes this anymore.
Let me dispel one myth: The bitter lesson can be true, and you can still want a bespoke, customized solution.
If it is true that eventually a general approach that scales with compute/data will supersede a bespoke expert solution, it is not necessarily true that today's expert solution should be thrown away at the first sign of a data-driven model.
It says nothing about what you should use today to build things, it says everything about how you should do research that lasts.
If you're building for billions of users in 10 years, sure go with whatever you think is the bitterest.
But if you're building for right now, you probably mashup whatever works best. That might still be a PID loop, EKF, and Homographies sometimes. But when you hit the frontier of what "works", then you have to reach for new things anyway.
Chesterton's baby/bathwater
WRT humanoids - I am making no statements.
As an analogy, it is currently possible to cross the country in 6 hours.
That fact does not imply that a person in Boston in the year 1850 should wait until commercial flight was available to travel to California.
So the fact that we will eventually have simpler/more efficient algorithms doesn’t tell us anything about what to do today.
>It says nothing about what you should use today to build things, it says everything about how you should do research that lasts.
Right! So who are you arguing with lol?
"Bitter lesson" gets pattern matched as a stock answer to anything vaguely resembling optimization/compute trends these days.
Like, news about internal reorgs at one company that could plausibly be done for any number of boring, bean-counting reasons.
From the article it seems more likely OpenAI just isn't prioritizing robotics anymore. Which seems to fit better than abandoning their booth at a robotics expo because "robotics = solved".
Everyone very usefully saying "bitter lesson lol" in the comments.
DeepSeek disputed it
The Bitterer Bitter Lesson is that there are no exponentials, only S-Curves.
You can render graphics with the fastest CPU you can find if you like, I'll keep my dedicated GPU
They don’t conflict
This looks like general cost cutting to boost some numbers before IPO.
They didn't fire or lay them off, they just disbanded the team and moved the researchers to other teams, so how is it cutting costs?
Robotics is more capital intensive than software engineering.
More wood behind fewer arrows
Moving resources from cost center to profit center.
Not necessarily cost cutting, just streamlining and focusing on a few core products that can make money. ChatGPT has leveraged its writing style into a product that many consumers choose for their basic AI needs. The company is using information picked up in those chats to personalize the outputs, build trust, and most importantly raise switching costs as the users won't want to lose their personalized helper and confidant.
I'm already seeing some of my family members use ChatGPT in this way, even going so far as enjoying some level of companionship during research and decisions in their lives. I believe this product will have fantastic pricing power and can make the ChatGPT IPO look very attractive.
> I'm already seeing some of my family members use ChatGPT in this way, even going so far as enjoying some level of companionship during research and decisions in their lives.
I’d also create a throwaway account if I was going to say something this cringe.
its cringe, but its true.
I suspect a major use of AI will be providing companionship to the lonely, and emotional support, and lots of similar things. Look at r/myboyfriendisai Look at reports of people using it as therapy.
The go and read The Machine Stops by EM Forster.
I am 'the lonely', and I indeed use ChatGPT this way.
What exactly are people going to do in the future? Not work and not talk to each other? AI maxis seem to believe interpersonal interaction is a thing of the past
lmao
So they gave up on non-cloud connected robotics then? Or was that ever in their scope?
Counterintuitively, it’s a lot easier to train a big model and distill it down to a small one than to train the small one from scratch. So never say never.
Who says you can’t run an LLM inside a robot if you have enough energy and hardware?
No one did. I was saying that those models would be small, dedicated models like those possibly developed by a robotics team. OpenAI's general purpose models are not small, dedicated models.
If those models can run locally and according to the article they're the same ones that do everything else - that means their models are quite small and cheap, which would be suprising. Even a machine running a 30B quantized LLM (which is generally the lower bound of usefulness) at meaningful speeds costs tens of thousands, and that's just the silicon, never mind building an actual robot around it.
It isn't even optional. Anything with a motor should have sufficient executive authority on board to decide on how to safely power down in case of trouble. Loss of communications is a major impediment to deploying robotics to the point that I think it should be outlawed to have robots that do not have all of their thinking done locally.
But I suspect that this will be one of those regulations written in blood.
At the bare minimum there should be a local independent system that safely shuts down the robot if it doesn’t get heartbeats from their control network
Is there a primer on how to actually use LLMs to control robots? Seems very token inefficient if you have to stream commands at 60fps.
Robotics are generally layered systems with lower latency/realtime systems running literally on the servos and other embedded hardware that use servo feedback and direct sensor access to operate, then building up complexity with tiers of increasing policy, usually all capable of working without the higher order layers which can inject desired states into lower order rings.
The nvidia study explains it https://research.nvidia.com/labs/srl/projects/astranomy/
The LLM's are basically used for the big picture. What is the situation. What is the goal and how to achieve it. Not direct control of all the servos.
The headline is somewhat misleading, as is tradition.
I think the idea is to just have the LLMs write the control systems and RL on that
Sure hope they're not writing the safety cutoffs as well. Some of those systems do not take kindly to bad control inputs, and determining if a controlled system is stable can be absurdly hard.
Depends on what your goals are. If your goal is to give investors erections, then re-using your existing trillion dollar model is efficient.
The termite mound model for coordinating robots using a harness-type LLM system all encapsulated within a non-LLM deterministic controller that rejects banned actions the LLM proposes, with individual termites running their own small efficient stacks and following commands from the LLM hive mind, that’s one way to do it. The main idea behind such architecture is that the controller prevents banned actions (like hacking HuggingFace) by the hive mind LLM and its swarm of agentic termites. Once you accept that alignment can always be internally subverted, it’s the only plausible safe architecture.
Saved the NVIDIA paper below for reading material, but if I had to guess just as a SWE how I would do it: use hierarchal thinking.
At the top sits a SOTA model capable of seeing snapshots of what's happening and directing intent. Then smaller, more reactive models translate the intent down into actions, before coming to a purely deterministic layer that handles inverse kinematics.
Sorta like how I control my body. I usually don't think about my fingers going to every key as I type - I know what I want to type, then that signal goes to some autonomous "using a keyboard" system, which invokes my muscle memory, which actually moves my muscles.
And if I get burned while I do something, my actual thinking brain doesn't really have time to process it before I'm already moving away; because those smaller systems have much faster - but more rudimentary - feedback loops.
they have decision models now
I wonder if this means anything about OpenAI's robot research office in Richmond, CA.
It's ironic that the people working to make others disposable were in fact, disposable.
AI is eating AI now. The value of code has collapsed, while the value of the software industry overall has not increased. Could AI be the snake that eats its own tail in the long-term?
According to the article at least, all these people still have jobs at OpenAI.
Thanks for pointing that out.
"Make a general intelligence, get surprised when it generalized "
Now your part of snake eating it's own tail makes little sense, AI isn't going away, only bad business models.
Open AI realizes robots with actual hardware are hard, and expensive.
"long-term vision" - not even mid-term? I hope Anthropic knows better...
money is tight