Your AI Doesn’t Actually Remember You

When you’ve been working with an AI coding agent for a while, it starts to feel like it knows your project. It knows what you’re building, what you’ve changed and what you discussed earlier.

That sense of memory comes from the system around the model. It brings useful information from earlier in the conversation into the next request. Let’s look at how that works using three terms: context, context windows and memory.

The model is stateless

Stateless means the model doesn’t automatically keep a memory of the conversation between requests. It works with the information available each time. The harness, the software around the model, carries relevant details forward.

Imagine this conversation:

You: My project uses Next.js 16. AI: Got it. You: Which router should I use?

It feels like the model remembered your stack. Conceptually, though, the harness can include the earlier messages in the next request:

The model can answer using that earlier information because it has been provided again. It doesn’t need to have stored the conversation itself.

So what is context?

Context is the information available to the model when it generates a response. That can include instructions, conversation history, files, code, tool definitions and results from earlier actions.

As you work, more information becomes available. You send messages, the model responds, a tool reads a file and another runs a command. The harness decides which parts to include in the next request.

What is a context window?

There’s a limit to how much information the model can work with at once. That limit is its context window. It’s measured in tokens, which are small pieces of text, sometimes a word and sometimes part of one. The response may need room in that window too.

Think of it like a desk with limited space. Instructions, earlier messages and files all take up room. A bigger desk holds more, but you still need to choose what belongs on it. As we saw with “Hello”, the harness may put information there before you type anything.

More context isn’t always better

A larger window gives us room to include more information, but deciding what belongs in it still matters. For a particular problem, the model may only need a few relevant files, earlier decisions and the latest tool results.

Including everything adds tokens to process, which can increase costs. Outdated decisions and unrelated information can also make useful details harder to pick out.

This is why context management matters. The harness needs to decide what to carry forward, what can be summarised and what can be left out. A larger window gives it more room, but it still needs to use that space thoughtfully.

What about memory?

For information from today’s session to be available tomorrow, the system needs to save it somewhere. Project instructions might live in AGENTS.md, architecture decisions in documentation, and progress in a plan or issue. A future session can read those records when it needs them.

I find this a useful distinction: the model processes the request, the harness carries the session, and the environment can preserve information beyond it.

The model doesn’t need every detail from every conversation. It needs the right context for the next request. That’s what makes managing context such an important part of building useful AI tools.