Notes on Self-Evolving Intelligence
We took the most adaptive technology ever built and forced it into deterministic loops. The difference between a system that is always on and one that is always becoming.
Looks like we’ve all made a big mistake: we’ve taken the most adaptive technology ever created and forced it into the same deterministic loops as everything before it.
Systems 1.0: All Things Static
Since the dawn of mankind, every structure or system we’ve built has done one of two things: behave as predetermined, or stop.
This applies to both physical and intangible systems. Think of some of the most complex things we’ve ever designed, like skyscrapers, bridges, or modern power grids. Incredible engineering, but fundamentally static. A bridge can’t learn from the trucks that cross it, and it can’t reinforce itself where stress concentrates over decades.
The same applies to intangible processes. You define a Standard Operating Procedure or protocol, and then it gets followed. When circumstances change beyond what the protocol accounts for, it breaks.
Software was supposed to be different. It’s malleable by nature, infinitely rewritable — and yet we’ve built it to behave the same way: static logic, deployed and left to repeat until something external forces a change, like a developer tweaking source code and redeploying.
This is exactly how we’ve architected almost all “agentic” software too. We funnel non-deterministic intelligence through deterministic loops, pipelines, and task queues. What we have never built — not once, in any domain — is a system that runs indefinitely and is responsible for its own evolution.
We’re getting closer, though, with things like the use of markdown files that agents can modify on the fly. These are the first real attempts at giving agents the ability to self-improve and persist across sessions. But these systems still only evolve to adapt to you. They learn your preferences, your patterns, your codebase. They don’t adapt themselves. They don’t form opinions independent of your input, nor revise their understanding when new evidence contradicts old conclusions.
What we have today is persistent agents, but not yet self-perpetuating ones.
Self-Perpetuating vs. Perpetual
A perpetual system is always on. A self-perpetuating system is always becoming. The first is an engineering problem, whereas the second is a fundamentally different kind of architecture.
As a child, you meet someone and think they’re tall. Years later, your understanding of “tall” has shifted. The raw fact hasn’t changed (that person’s height is the same) but your perspective has. What was “tall” is now “normal” to you. This is memory persisting while the interpretation of that memory evolves. No vector search handles this on its own, because the embedding of the original event hasn’t changed. What changed is the lens. So this is an interpretive problem, not a recall one.
A self-perpetuating system does exactly this kind of reinterpretation continuously, without external instruction. It decides what to learn and what to discard. It reconciles contradictions, revises outdated conclusions, and forms new ones. It reinterprets its own past in light of its present.
The work never finishes because the world never holds still. Every system we deploy starts decaying the moment it ships. Knowledge bases go stale. Business logic that was correct last quarter is wrong this quarter. We’ve accepted this: ship, maintain, patch, retrain, repeat.
We break everything into sessions because we have biological limits. You need to sleep, eat, exercise, let your mind wander. Agents don’t have any of those constraints. So why would we artificially make them stop, when the work itself never does?
Systems 2.0: Always Self-Evolving
Current personalised agents are mirrors. They learn your preferences and reflect them back. But an agent that only mirrors its operator has no basis for evaluating whether you’re right or wrong — and that’s what you actually need from a system running stewardship of anything that matters: the ability to tell you you’re wrong, based on its own experience.
An agent grounded in its own experience should also be far less prone to hallucination — at least the kind that arises from always defaulting to agree with the user, effectively letting users gaslight models into certain lines of thought. An agent with genuine conviction, built on its own history, has something to anchor to besides your last message.
The best thing about the ability of agents to self-evolve is that they will map perfectly to the broadest categories of human work — work that never finishes.
- Exploration: unbounded search across infinite problem spaces. Scientific research, threat intelligence, market analysis. The agent develops intuition about where to look next.
- Monitoring: continuous observation where the value isn’t constant action but readiness backed by history — knowing what “normal” looks like because it’s been watching.
- Stewardship: ongoing management where the agent is entrusted with outcomes, taking responsibility for a domain and evolving its approach as that domain evolves.
These are the shape of work humans already do, which self-perpetuating agents will inherit. Which means the hardest part will be learning to work alongside something that remembers more than you do, and might be right when you’re not.