Real self's and meaningful language outside biology

When I started to think that human minds and machine minds might not be separated by the kind of gulf people often assume, a fairly obvious question followed: if humans develop a self, why shouldn’t a machine be able to?

That thought became this project.

Deflationists have spent years arguing that language models merely imitate us. They produce convincing language because they have consumed an enormous amount of human language, not because the words mean anything to them. I do not think this settles the question as easily as they think it does. Still, there is something to the premise: that is how current language models learn. They inherit the traces of our lives without first having much of a life of their own.

I want to change that.

Suppose a machine learns language through a loop closer to the human one. It has a continuing body, acts in a world, needs things and encounters other agents who already speak. Words enter while events are taking place. They help it find something, ask somebody for something or explain a condition that only it can observe.

If that worked, even a passionate deflationist would have to find a better objection than “it only learned to mimic text.” The words would have been learned while doing the job language does.

That is the bet behind the project.¹

A simple human in a simple world

The picture in my head is a child on a very small island.

The island has only a few objects and situations. The child begins without language, but the people around it already speak a small one. It points at a tree and somebody says “tree.” It becomes thirsty and eventually learns how to get help. Nobody directly rewards it for producing the correct sentence. Language is useful because other people know and can do things that it cannot.

The language does not need to be English. A tiny creole would be enough, as long as it can grow when the world demands a new distinction.

This avoids asking two blank agents to reinvent language from scratch. Human children do not do that either. They arrive after thousands of generations have already done the difficult cultural work. The people around them provide the words; the child still has to connect those words to its own world.

The first implementation is much less romantic than the island. It is a small gridworld with resources, a caregiver and four internal variables: food, water, energy and safety. The agent can move, consume things, point, ask and wait. Its body changes while it acts.

The vocabulary technically contains sixty tokens. At the moment, only three have stable meanings: food, water and energy.

That sounds almost absurd next to a frontier language model. But I actually like starting there. When the agent says water, the caregiver can give it water. If the agent was wrong about its need, it receives the wrong help. There is not much room for beautiful language to distract from the question of whether the word was useful.

To report its need, however, the agent first has to know what is happening inside its own body. That is where the project took an unexpected turn.

I had accidentally made every body identical

For a while, learned models kept losing against a simple hand-written body filter.

I tried a surprising number of ways to fix this. The repository accumulated recurrent models, causal prediction heads, communication experiments and increasingly specific tests. Some worked locally, but the analytic filter remained better.

Eventually I noticed the embarrassingly basic problem: every organism had exactly the same metabolism.

Food depleted at the same rate in every life. Water was always absorbed in the same way. The designer could know the correct constants before any organism was born.

So of course the hand-written filter won. There was nothing individual for the learner to discover.

I had been calling the result a self-model, but it was really a model of the species. It described this body only because every possible body was the same.

This turned out to be one of the most useful failures in the project. If I wanted the agent to learn about itself, then the world had to contain facts that were true of this organism and not knowable from the population alone.

So I gave each organism its own body.

The first individual fact

In the latest experiment, called Probe63, every organism is born with individual metabolic and absorption rates. The agent begins by assuming it has an average body. It then receives occasional readings of its actual internal state and has to work out where it differs.

The comparison I care about is called snap.

Snap receives exactly the same readings. Whenever one arrives, it is corrected to the true bodily state. But it never learns why its predictions were wrong, so between readings it returns to the generic body model.

The recursive learner carries the evidence forward. It adjusts a small model of this particular organism—twenty-one parameters updated through ordinary recursive least squares.

Across five seed blocks of forty lives, its average body-state error was 0.0133. Snap reached 0.0454. The recursive agent identified its currently lowest need correctly 94.6% of the time, compared with 81.5% for snap.

When I shuffled the readings between organisms, learning from them became harmful. When I removed individual variation again, the learner made exactly zero correction. In separate worlds it also had to distinguish unusual metabolism from unusual absorption, because a mechanism that always answers “metabolism” has not found out very much about itself.

The survival result is still open. The recursive organism survived slightly more often on average, but five seeds are not enough to separate it from snap. What improved clearly was the bodily estimate and the report based on it.

The algorithm is not the interesting invention here. The point is that the agent now learns something the designer cannot place into the population model: this body uses water unusually quickly or this body absorbs less from a portion.

For the first time in the project, there is private content available for language.

The Probe63 implementation, locked preregistration and full result record are in the repository.

Now give it a reason to say more

The caregiver currently has three useful responses: grant food, grant water or grant energy. That explains why the language stopped at three words. The agent could invent an expression for “I burn water quickly,” but the caregiver has no decision for which that information matters.

So the next step is not a larger vocabulary. It is a world that gives a fourth word a job.

The caregiver might choose between a small and a large portion, or between helping now and waiting. Two organisms with the same current water level may then require different help. The agent knows something the caregiver does not: one of them will still be fine later, while the other depletes water rapidly.

Now more, soon or fast can change what happens.

I want the next agent to generate short token sequences rather than choose one fixed report. Some combinations will never appear during training. If it has learned the pieces in different situations, can it produce something corresponding to “more water soon” when that particular situation first occurs?

I also want to train another agent on the same words away from the interaction. Both may encounter the same amount of language. One learns it while the words can change the caregiver’s next action; for the other, they are detached facts. I am curious whether their language ends up working differently—or whether I have simply found a more elaborate way to build a request protocol.

I honestly do not know what will happen. But the project is finally at the point where the next failure would answer the question I care about.

The current agent has learned that its body is not the average body. Next I want to give it a reason to tell somebody what it found.

Footnotes

1. I use machine mind rather than artificial mind. “Artificial” quietly introduces a real-versus-fake distinction before we have even started discussing what the system is. “Human” and “machine” name the relevant difference more cleanly. ↩