Philipp Hölke

Library

Your Agent Needs to Sleep. Here’s Why & How

How to build an AI sleep cycle that curates your agent’s memories — because long-term memory isn’t solved when retrieval works; without maintenance it turns into a landfill.

A high-contrast stippled monochrome of a woman’s face in profile, head tilted back and eyes closed, at rest against black.

I keep seeing teams add “long-term memory” to agents and treating it like a solved problem once embeddings and retrieval are in place. Vectors go in, you can query them later, job done.

But that’s not really memory. It’s just storage that never cleans itself up.

Human memory doesn’t work like that. We don’t just accumulate experiences forever and hope retrieval magically stays useful. Things fade. Similar experiences blend together. Vague impressions get clarified later or quietly dropped. And a lot of that maintenance happens when we’re not actively doing anything, mostly during sleep.

AI memory systems skip that step entirely. Early on the memory store is small, everything feels relevant, retrieval looks impressive. But over time things start to get noisy in ways that are hard to notice until they’re already baked in. Old preferences stick around even when they’re no longer true. Redundant memories pile up. Low-confidence summaries never get revisited. Contradictions don’t resolve, they just coexist — and, like humans, cognitive dissonance is not something agents handle well.

What’s missing is a maintenance loop: something that periodically looks at memory and asks whether it still deserves to be there in its current form. The system hasn’t learned anything wrong exactly, it’s just never had a chance to tidy up.

The mental model that’s worked best for me is thinking of this as a sleep cycle for an AI system. You separate cheap, mechanical checks from slower, more reflective decisions, and you run the whole thing on a schedule rather than in response to every event (just like a human circadian rhythm).

Here’s the minimal version that’s actually useful.

Detection

This part should be boring and deterministic. No models involved. You score memories for decay based on time or lack of reinforcement. You flag things that haven’t been touched in a while. You group similar memories together using embeddings or even simple text overlap. You mark summaries that are too short or clearly low confidence. All this step does is narrow down the set of memories that might be worth thinking about. Nothing changes yet.

Decision

This is where you slow down and allow semantic judgment. You look at small batches of candidate memories and decide what to do with them. Some get merged because they’re clearly talking about the same thing. Some get split because one memory actually contains two ideas that shouldn’t live together. Some get rewritten because the summary is vague and not very useful. Some get dropped because they’re just noise. The important part is that the default is to keep things as they are unless there’s a clear improvement. Memory systems should be conservative by default.

Applying Changes

After that you apply changes in a controlled way. No silent overwrites. If a memory is merged or split, the originals stick around in some form. If something is deleted, it’s a soft delete. If a summary changes, embeddings get regenerated. Every mutation goes through a single place in the codebase so you always know how memory is allowed to change.

Logging

Finally, you log everything. Not just what changed, but what it looked like before and why the change happened. That log isn’t just for debugging. It’s how you keep trust in a system that edits its own past. If you ever can’t explain how a memory ended up the way it is, that’s a design failure. This is the real context graph everyone’s been posting about for the past fortnight.


One thing that’s easy to miss here is that memory decisions aren’t neutral. The same memory shouldn’t age the same way for every system. What matters depends on identity. Preferences might be critical for a personal assistant and mostly irrelevant for a research agent. Old goals should fade when priorities change. So memory maintenance needs to be filtered through who the system is becoming, not just what it once observed.

Most teams don’t run into this problem immediately. It shows up later, once agents run longer, act more independently, and start making decisions based on months of accumulated context. At that point it’s hard to retrofit because memory has already turned into a landfill.

If you’re building agents that are meant to stick around, it’s worth thinking about this early. Not as a grand theory of mind, just as a small, boring maintenance job that runs in the background and keeps things usable.

And now, off to sleep.