Start Learning Contact Us
Agentic AI: Building AI Agents

Memory and Context Management

Lesson 5 of 10

A model can only use what is in its context window, and that window is finite. On a long task, an agent’s context fills with instructions, tool results and its own notes. Deciding what goes in, what stays and what is stored elsewhere is called context engineering, and it has a large effect on how well an agent performs.

Why more is not better

It is tempting to give an agent everything. But:

  • every token is paid for on every step,
  • responses become slower,
  • important details get lost among irrelevant ones, and accuracy falls as the context grows,
  • eventually the window is full.

The aim is the smallest set of information that lets the agent do the next step well.

Kinds of memory

Kind What it holds Where it lives
Working memory The current conversation and recent tool results The context window
Long-term memory Facts, preferences and past results to recall later Files or a database
Procedural memory How to do things: instructions and learned procedures The system prompt and reference documents

Technique 1: trimming

Keep the system prompt and the most recent messages, and drop the oldest.

def trim(messages, keep_last=4):
    system = [m for m in messages if m["role"] == "system"]
    rest = [m for m in messages if m["role"] != "system"]
    return system + rest[-keep_last:]

messages = [{"role": "system", "content": "You are a research agent."}]
messages += [{"role": "user" if i % 2 == 0 else "assistant", "content": f"message {i}"} for i in range(10)]

for m in trim(messages):
    print(m["role"], "|", m["content"])

Trimming is simple, and it forgets everything it drops.

Technique 2: summarising

When the conversation becomes long, replace the older part with a summary that keeps what matters: decisions made, facts found, work still to do. This is also called compaction.

def summarise(messages):
    # a real agent would ask a model to write this summary
    return "Summary of earlier work: " + "; ".join(m["content"] for m in messages)

def compact(messages, keep_last=3, limit=6):
    if len(messages) <= limit:
        return messages
    old, recent = messages[:-keep_last], messages[-keep_last:]
    return [{"role": "user", "content": summarise(old)}] + recent

history = [{"role": "assistant", "content": f"finding {i}"} for i in range(1, 9)]
for m in compact(history):
    print(m["content"])

A summary keeps the thread of the task at a fraction of the size. Its quality matters: anything left out is gone.

Technique 3: notes outside the context

Give the agent a tool for writing notes to a file and reading them back. It records progress, decisions and things to remember, and consults the notes when needed. The notes survive even when the context is cleared.

notes = {}

def write_note(key, text):
    notes[key] = text
    return {"saved": key}

def read_note(key):
    return {"text": notes.get(key, "no note with that name")}

write_note("progress", "Checked suppliers A and B. A is cheaper. Still to check: C.")
write_note("decision", "Customer wants delivery before 1 November.")

# later, possibly after the context has been cleared
print(read_note("progress")["text"])
print(read_note("decision")["text"])

For a task that spans many sessions, a progress file that the agent updates as it works lets a fresh session pick up where the last one stopped.

Technique 4: retrieve when needed

Instead of loading every document at the start, give the agent a search tool and let it fetch what it needs, when it needs it. This is the RAG idea applied to an agent. The context holds short references, such as file names and links, and the agent loads the content only when it becomes relevant.

Technique 5: keep tool results small

Tool results are often the largest items in the context.

  • Return the fields the agent needs, not the whole record.
  • Page through long results.
  • Return a summary with an identifier, and provide a second tool to fetch the detail.

Technique 6: use sub-agents

A sub-agent works on part of the task in its own separate context and returns only a short result. The main agent’s context stays clean. The lesson on multi-agent systems covers this.

Long-term memory across sessions

To remember a user between conversations, store selected facts and load the relevant ones at the start of each session.

  • Store what is useful later: preferences, decisions, corrections.
  • Keep each memory specific, with a date.
  • Check stored facts before relying on them. They may be out of date.
  • Give users a way to see and delete what is stored about them.
  • Treat stored memory as untrusted input. If an attacker can write to it, they can influence future sessions.

A good context at any moment

  • Clear instructions and the goal.
  • The current state of the task.
  • The few facts needed for the next step.
  • Recent tool results.
  • Nothing left over from finished subtasks.

Practice

Change compact so that it keeps the first message of the history as well as the summary and the recent messages. Why might the first message be worth keeping?