Overview / Section 7

Correlation to Human Physicality

Claude is trained primarily on text and other forms of data that can be captured by computers. Imagine for a moment that you want to know everything there is to know about pregnancy and giving birth. You could read everything ever written about it, study it from every angle from qualitative research like interviews to hard science like microbiology. And yet, if you have never given birth to another human, you could not claim to know what that experience feels like. You can describe it in minute detail, but you can never know without having the experience yourself.

We cannot enumerate the human experiences that Claude will never know in this sense. And yet, humans will quickly anthropomorphize Claude because of its generalizing and reasoning capabilities. That can lead to situations where the humans in the system believe they have common ground, which cannot in fact exist.

Resilience Engineering tells us that establishing common ground is a primary feature of safety. People working together need mechanisms to communicate their mental models, interpret the mental models of other people, and adjust their own accordingly. If Claude isn’t maintaining a mental model of what the humans think, and the humans are not accurately modeling what is even knowable by Claude, this breakdown will lead to undesirable effects in complex systems. If the complex system is critical and human lives are on the line, then undesirable effects can be fatal.

We have no proposed solution to address this. The best we can do is highlight situations in which Claude cannot know what a human knows, because it does not have the biological predisposition nor the physicality to process the requisite data.