An LLM’s Interpretation of Reality

The next frontier in AI is ontology.

There are several reasons for this, but the shortest explanation is that AI is getting better at reasoning faster than we are getting better at defining reality for it.

I think of all this as Reality Interpretation.

What this means practically is that, as models become better at reasoning, more of the bottleneck moves into the environment around them: that is, the entities that exist, the relationships between them, the states they can occupy, the measurements that matter and the rules that determine whether an outcome is acceptable.

Many of us already operate this layer manually, running model/agent evals, designing strict schemas, semantic layers, policy engines, state machines, and increasingly precise definitions of what an agent is allowed to believe and do. These may look like separate engineering problems, but underpinning them is the same frontier problem: a capable machine still needs a world whose meaning does not change underneath it.

What is Reality?

For an autonomous system to work reliably, its interpretation of reality has to match ours.

Consider: when I say a task is done or a customer is high value, the model and I need to mean the same thing. Our definitions must resolve to the same underlying facts, otherwise the model can reason perfectly inside the wrong reality.

Lack of sound Reality Interpretation is what causes model hallucinations. The model constructs something coherent that simply does not correspond to the world outside it.

But it need not be hallucinations at all. Outdated business logic can create the same problem; the facts gathered by the agent may all be correct while the real-world interpretation is wrong. This is why words like good, bad, fast, slow and important become consequential once machines begin acting on them.

(Coincidentally, I think this ambiguity is also a major source of human inefficiency inside organisations. Managers leave concepts fuzzy because they can resolve the ambiguity themselves, while everyone else develops a slightly different definition of acceptable, good or great)

The way out is to make the underlying reality explicit.

You define the things that exist (eg a customer), then their properties, relationships, possible states, and the events that move them between those states. You also decide which source is authoritative for each fact.

This is the ontological work that defines reality.

What is Truth?

Now that you’ve defined Reality, the next question is how do you determine whether a claim about that reality is actually true?

Suppose you say someone is tall. The observable fact might be that they are 180cm. But that number tells us very little on its own; we first had to decide that height was the relevant property to begin with, then decide what threshold constitutes tall.

The measurement must be objective, mathematically verifiable and measurable truth. The choice of what to measure, and what bar to apply to it, is the Ontology.

So how does this translate to agentic systems?

An example: a service can have 120ms latency, but whether that is fast depends on which latency we care about, over what period, and against what threshold. Or, in another example, a customer can have spent $10,000 and still not be high value if the business only cares about gross profit or retention.

Once those are defined in an ontology, verification should be as deterministic as possible. Truth, in the engineering sense, should be objectively, repeatably measurable and verifiable.

Many of us do this instinctively when using coding agents; we’ll let claude code or codex take completely different routes through a repository, but before pushing to production we still run hardcoded tests.

Thus, if two agents inspect the same system and one concludes that it is healthy while the other concludes that it is not, either the ontology is incomplete, the measurement is ambiguous, or the standard has not been defined properly.

There will always be outcomes that require judgment and therefore probabilistic evaluation (eg whether a piece of research was insightful or a response was empathetic may never collapse neatly into a deterministic test). But where the ontology gives us something objectively measurable, there is little reason to rely on another model for verification.

TLDR: the model should be free to interpret how to get somewhere, but it should have much less freedom to decide whether it’s arrived.

The obvious response to all of this is better evals, right?

As agents become more capable, we build increasingly elaborate benchmarks and graders around them to measure whether the work was good, whether the right tools were used, whether the final answer was correct, and where the harness failed.

This is useful. But most evals answer a different question from the one an autonomous system ultimately needs answered.

Most evals tell us how the system performed after the fact. A failed eval can improve the next run, but it doesn’t really protect the present run that’s being evaluated. If an agent has already sent the money or deleted a database record, learning afterwards that the run scored poorly is not especially helpful.

Your Ontology Is Your Moat

This has consequences for our collective 20 year obsession with “proprietary data”, mainly because structured information becomes a huge moat.

Two competing companies can have 10 years of customer transactions, support tickets, operational logs and purchasing history and still be in very different positions. One has the records, the other has also encoded what those records mean: which customers matter, which products substitute for one another, what constitutes a failed close, where exceptions apply, which source is authoritative, and which outcomes the business actually cares about.

That difference becomes more valuable as the model layer becomes more commoditised, especially because the model used for a given task may change several times over the life of a system. The company’s interpretation of its own business is much more persistent.

The most valuable thing you can do as a business owner, then, is to think very carefully and methodically about what data you own and have access to, and what that means for your business.

TLDR: proprietary data tells the machine what happened. The harder asset to reproduce is the structure that tells it what any of it means.

Where I Think This Goes

An ontology is ultimately an opinion about reality.

What you choose to represent gains weight, while what you leave out becomes harder for the machine to see. Once agents begin acting against that representation, these choices heavily influence what the system notices, values and ultimately optimises.

I don’t think the end state is a giant static schema maintained by humans forever, though.

Humans may define the broad structure, but the machines we develop should increasingly be able to notice where that structure fails and requires dynamic adaptation. For instance, where a metric stops explaining outcomes well or if recurring exceptions suggest that its current model of the world is missing something, those examples warrant an ontological update.

If that happens, the ontology itself becomes part of the self-recursive learning loop. The system is no longer only improving how it acts inside a fixed interpretation of reality; it is also improving its own interpretation.

Of course, that raises even harder questions: how much authority should a machine have to revise the reality it uses to judge itself? and how can objectivity be maintained?

And even if we eventually trust machines to discover that our definitions were wrong, the harder part will be knowing whether and when they are right.

It’s not clear to me what all this looks like in practice yet, but it’s an interesting area I’m actively looking into at both Kamino and Mission Systems.