Can AI take control of a robot?
The AI researchers at Andon Labs, the people who gave Anthropic Claude an office vending machine to run, and hilarity ensued, have published the results of a new AI experiment.
They wanted to see if LLMs were technically capable of functioning as a robot’s brain, that is, connecting their “thinking” (textual decision-making) with real sensors and movement.
The result:
➡️ LLMs are not yet ready to safely and reliably control robots. – even the best models scored below 40% accuracy, humans reached 95%.
➡️ They made funny but also dangerous mistakes (falling down stairs, misinterpreting the environment, or reacting chaotically).
So I wonder how long it will take to achieve something like this, when Elon Musk wanted to present his robots as autonomous units? (and even then, it turned out that they were not completely independent)
How do you see the future of AI integrated into hardware robots?
Replies
Cal ID
None are actually good at it yet. In real-world tasks, these bots struggled, even with the best model. Often making unsafe or silly mistakes that a human wouldn’t.
So for now, fully autonomous robot brains are still far off, and AI needs to get a lot better at sensing, moving, and staying safe in the real world. Maybe with more training data it might evolve over time.
minimalist phone: creating folders
@sanskarix Maybe we'll see it in 2030. Maybe :)
For humanoids I’d say it’s going to be phased. Have you seen Joanna Stern’s interview with the 1X CEO about Neo? He basically says it’s still human-assisted and they’re using real homes to train it. The NVIDIAs keynote on physical AI was the similar direction, building the stack first. So near term it’s AI + human in the loop. full home autonomy comes later.
For other robots i think it’ll move faster especially when the environments are more structured like in warehouses.
minimalist phone: creating folders
@alecsorenpedersen IMO, China has already trained those robots quite well, but understand that the rest of the world is still tapping the water.
lol, vendor machine maybe yes...but right now, I doubt anyone would really want to pay 499 per month or 20k one time, just to get big chunks of iron that embedded with some chimps and numerous amount of cables that fails doing most of housework + a 24hr turned-on tele-monitoring feature.(yes, I'm talking about NEO )
This NEO thing really reminds me of Builder.ai , one of the biggest scam ever about AI(so far)
Though good side is at least Onyx Robotics being honest to public about this tele-cooperation feature on NEO. Clap for that.
minimalist phone: creating folders
@rickyguo I think that builder ai website cannot be open :D I think that China is training their robots much better than the US. Especially innovation and development seem to be faster there.
@busmark_w_nika BuilderAI has run off :)
Not sure which stage humanoid robotics development is currently in. From what I learned in uni back in the days it was using prolog. I'm more of a software sides tho, and I could imagine it would be really hard to train sth based on that since the core codes for building that were mostly hard-corded to recognise "things" and move....
Feels like we’re still in the “AI toddler with scissors” phase — lots of potential, zero motor control. LLMs can reason in text, but embodiment needs perception, feedback loops, and real-time adaptation and things language alone can’t do yet. My guess: a decade before “autonomous” actually means unsupervised.
minimalist phone: creating folders
@twinkal_shah1 When do you think we will have robots that are not so clumsy in their physical movements and can be managed by AI without any serious or harmful problems?
@busmark_w_nika I think it will still takes decade at least for this to happen and not until we've mature AI models to handle it for sure.
minimalist phone: creating folders
@twinkal_shah1 Maybe 2050 :D
@busmark_w_nika hahaah maybe or maybe sooner than we're expecting.
IXORD
Robots will only continue to improve over time. What we have now is actually a good starting point for this industry. Although some might not say the same when they stumble across YouTube Shorts xd
minimalist phone: creating folders
@ixord Hahaha, YouTube shorts only convince me that we are not there yet :D
Interesting take @busmark_w_nika! I see a lot going on about robotics, but hardly seing them in as super capable. Even with LLMs, latency make them weird...
minimalist phone: creating folders
@efren_canibe IMO, the advantage is that when you train one unit, you can pass this knowledge to hundreds of units. But to train a robot is not easy.
@busmark_w_nika definitely not an easy thing! A lot of things going on 🤯
minimalist phone: creating folders
@efren_canibe But doable (resources consuming) but doable :D
It’s fascinating but also a bit scary how unpredictable LLMs can be when given physical control.
minimalist phone: creating folders
@laylabell Especially scary to see some cases from China LOL.
I was born in a village without electricity. Literally saw the transformation and advancement of tech worlds. Now seeing another transformation through AI era. What a time to be alive!
minimalist phone: creating folders
@murad_hoque Wow, where have you been raised?
@busmark_w_nika I was born in a village near a small 3 tier town in Bangladesh. back in those days electricity was available at tiar 1 and tier 2 towns and only in big towns. when I was 4yrs old, My parents moved in to a town where there was electricity.
minimalist phone: creating folders
@murad_hoque Oh, when you talk about that like this, electricity is literally a privilege.
Fascinating results, really shows how far we still are from true robot autonomy. My guess is we’ll need a mix of specialized models and sensor-native AI before LLMs can safely control movement.
minimalist phone: creating folders
@mohammed_maaz3 They still seem to be clumsy when it comes to mechanical movements.
I actually believe this transition - AI controlling real-world robots - will happen faster than most anticipate.
Web and desktop AI are relatively easy: publish, iterate, shut down if things go wrong. With hardware it’s different. It’s more expensive, you must ship higher quality, deal with physical safety & logistics.
That means we’re still in a quiet phase. But when the break-through happens, I think we’ll see a flood of market releases - many hardware+AI solutions launching in parallel.
So yes, it may seem “quiet now”, but that’s exactly what precedes the moment when it all scales.
minimalist phone: creating folders
@shahar_shalev In how many years do you think AI will control hardware robots?