Aussie AI

14. Brainy Body

Book Excerpt from "The Sweetest Lesson: Your Brain vs AI"

by David Spuler, Ph.D.

14. Brainy Body

“A strong body makes the mind strong.”

— Thomas Jefferson.

Brain Versus Brawn

Do you need a body to be smart? It sounds somewhat facetious when you write it like that. And yet, it’s a serious question with a whole slew of research papers behind it, titled: embodied AI.

Technically, I’m mainly going to talk about the theories of “embodied cognition” and the achievement of real intelligence. There are other practical aspects of physical AI such as sensory capabilities (e.g., machine vision) and having robots that have mobility and the ability to take actions (i.e., go somewhere and do something).

This research about embodied cognition espouses the theory that achieving true intelligence requires physical interaction with the world. In short:

No brain without a body.

Why might full intelligence require a physical presence in the world? Some of the reasons a physical body is required include:

Self-awareness (of oneself and one’s dimensions)
Building a theory of the 3D world
Generalization of world-based reasoning

Another obvious reason is for an LLM to actually understand the human senses. Any good writer of fiction knows to use the five senses for “show not tell” in a scene:

Sight (vision)
Sound (hearing)
Smell (aural)
Touch (tactile)
Taste (gustatory)

You know there are more than five senses, right? I’m not being supernatural, but refer to all the weird extra senses that our bodies actually have, including:

Physical sensations — pain, discomfort, butterflies, tingling, numbness, etc.
Body states — hunger, thirst, nausea, fever, illness, dizziness, etc.
Proprioception — knowing the position of our body in 3D space (my favorite one!).

There’s a bunch more. Apparently, humans can detect magnetic fields, but not as well as pigeons. Except for that weird one, you know all of these extra senses, innately, without having to be trained or sent to boarding school.

How many does your LLM understand? None.

It is more than a theory that LLMs should reside in physical robots, such as to achieve mobility or allow actions to occur, but that their training must occur in this way so that the LLM can itself be intelligent in a more complete way. Maybe boarding school for LLMs is worth a try.

Why Embodied?

Think about how a baby learns. They reach out into three-dimensional space learning where everything is, including their own personal space. Later, they crawl around, learning how things are arranged in the world, and what it feels like to touch anything they can find, to their benefit or their peril.

How can an LLM do that? All it has is its words. Can we really explain all of those things to an LLM by giving it more words to read? How is it to learn the meaning of this joke (and is it a joke?):

Kemp’s Law: Never re-tie only one shoelace.

An LLM is inherently stuck inside its own head. Indeed, embodied AI is one of the theories about why modern LLMs still struggle with various aspects related to human-level intelligence:

Two-dimensional spaces (e.g., chess boards or Galaga).
Three-dimensional reasoning (e.g., people sitting at the kitchen table).
Textures and senses (e.g., how it feels to touch or smell).

Is automated driving of a car in a 2D or 3D world? The answer is: yes, both. There are aspects of 2D maps, certainly, for efficient execution and specification of certain tasks such as pick-up, drop-off and routing. However, for safety the LLM must consider its full three-dimensional shape and the same of the other vehicles and their overall environment.

The benefit of real-world experience is not limited to physical ideas about a multi-dimensional environment and the objects it contains. Some of the more non-obvious limitations to full intelligence that embodied AI seeks to improve include:

Self-awareness — the AI is in the environment, too).
Temporal reasoning — time always flows one way.

Understanding of time and the notion of “causality” (cause-and-effect) may be improved by embodied AI. Learning about the world while experiencing real-time changes in the environment could improve an LLM’s ability to reason about time.

Fake Versus Real Bodies

Researchers have tried the old fake it till you make it style of research for embodied AI, although they don’t describe it that way in their research funding proposals. Instead of a real body, we can pretend that scanning through a stream of video and audio data is the same as having eyes and ears. You can try to learn from that, and this is definitely getting better.

Training advanced LLMs on video data is a thing.

But what about real bodies? What about the senses of touch? Can they explore the world and learn what it means to be inside a 3D space? Do they feel pain when they fall off their bicycle? Lots of robots have real bodies, and they’re starting to look impressive, but do they learn?

As far as I know, no-one’s really invented a robot hooked up to a smart enough LLM that it could learn. I mean, I could have written “learn as it grew” and there’s a few Hollywood movies that I quite like where this happens, but not in real life.

Robot soccer is not quite there yet.

It might be a while before humanity designs an advanced mobile learning station. Oh, wait! Sorry, I was wrong. We have managed to create something advanced that can learn like that: babies.

References

Some of the many references on Embodied AI, with the theory beginning as far back as Brooks (1991), include::

Shaoshan Liu and Shuang Wu, Apr 29 2024, A Brief History of Embodied Artificial Intelligence, and its Outlook: Examining the history, current state, and future of Embodied Artificial Intelligence, https://cacm.acm.org/blogcacm/a-brief-history-of-embodied-artificial-intelligence-and-its-future-outlook/
HCPLab, July 2025 (accessed), Paper List and Resource Repository for Embodied AI, https://github.com/HCPLab-SYSU/Embodied_AI_Paper_List
Rolf Pfeifer and Fumiya Iida, 2004, Embodied Artificial Intelligence: Trends and Challenges, https://people.csail.mit.edu/iida/papers/PfeiferIidaEAIDags.pdf
Robert McCarthy, Daniel C.H. Tan, Dominik Schmidt, Fernando Acero, Nathan Herr, Yilun Du, Thomas G. Thuruthel, Zhibin Li, 12 Nov 2024 (v4), Towards Generalist Robot Learning from Internet Video: A Survey, https://arxiv.org/abs/2404.19664
Yang Liu, Weixing Chen, Yongjie Bai, Xiaodan Liang, Guanbin Li, Wen Gao, Liang Lin, 26 Aug 2024 (v7), Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AI, https://arxiv.org/abs/2407.06886
Zhe Sun, Pengfei Tian, Xiaozhu Hu, Xiaoyu Zhao, Huiying Li, Zhenliang Zhang, 25 Mar 2025, Body Discovery of Embodied AI, https://arxiv.org/abs/2503.19941
Matej Hoffmann, Shubhan Parag Patni, 15 May 2025, Embodied AI in Machine Learning -- is it Really Embodied? https://arxiv.org/abs/2505.10705
Rodney A. Brooks, 1991, Intelligence without representation, Artificial Intelligence 47 (1991), 139–159, https://people.csail.mit.edu/brooks/papers/representation.pdf (Seminal paper in embodied AI theory.)
Yequan Wang, Aixin Sun, 20 May 2025, Toward Embodied AGI: A Review of Embodied AI and the Road Ahead, https://arxiv.org/abs/2505.14235