Factored

Context Management for LLMs

## About This Course

Large language model agents are limited by a fixed context window: they can only process a finite amount of information at once. That constraint becomes a real problem in long-running conversations, where important details from earlier exchanges are silently dropped. This hands-on lab introduces two practical context-engineering techniques—long-term memory and rolling summaries—to help agents retain and recall information over time.

You will extend a LangGraph-based task-tracking agent that monitors conversations, manages action items, and maintains a running summary of prior interactions. When the agent’s active context window fills up, older conversations are evicted and stored in a ChromaDB vector database. Later, when a user references something from the distant past, the agent retrieves relevant history through semantic similarity search rather than relying on whatever still fits in the window.

Throughout the lab, you will implement the storage layer, build a retrieval tool, wire up eviction logic that connects memory to the agent graph, and refine the system prompt so the agent knows when and how to use its new long-term memory. Automated tests guide your progress at each step, so you get immediate feedback as you work.

By the end of this course, you will understand how vector databases support agent memory, how sliding-window eviction interacts with semantic retrieval, and how prompt design enables an LLM to use memory tools effectively. You will also have optional extensions to explore—tuning retrieval parameters, swapping in local embedding models, or adding new vector collections for past tasks.

---

## Requirements

To succeed in this lab, you should be comfortable with the following:

<ul>

<li>Python programming — You can read and modify Python modules, work with functions and classes, and run scripts from the command line.</li>

<li>Basic LLM concepts — You understand what a context window is, how prompts guide model behavior, and what embeddings represent at a high level.</li>

<li>Familiarity with LangChain or LangGraph — Prior exposure to agent graphs, tools, and state management is helpful but not strictly required; the starter code provides the structure you will extend.</li>

<li>Command-line and environment setup — You can create a <code>.env</code> file, install dependencies with <code>uv</code> or <code>pip</code>, and run <code>make test</code> to verify your work.</li>

<li>OpenAI API access — An active <code>OPENAI_API_KEY</code> is required for embeddings and agent calls unless you complete the optional task of switching to a local embedding model.</li>

</ul>

No prior experience with ChromaDB or vector databases is required; the lab introduces those concepts as you implement them.

---

## Course Staff

Juan Jose Muñoz

Juan Jose Muñoz is a Machine Learning Engineer at Factored, where he designs and ships production-grade AI systems for enterprise clients. His work spans LLM applications, agent architectures, and the practical challenges of deploying models that must perform reliably over long, multi-turn interactions. He brings a hands-on engineering perspective to this lab, focusing on patterns that translate directly from coursework to real-world agent development.

Jairo Nicolas Gomez

Jairo Nicolas Gomez is a Machine Learning Engineer at Factored with expertise in context engineering, retrieval-augmented workflows, and building intelligent agents on top of modern LLM frameworks. He has worked on systems that combine structured state management with semantic search, and he co-developed this lab to give learners practical experience extending LangGraph agents with vector-backed long-term memory.

---

## Frequently Asked Questions

What web browser should I use?

The Open edX platform works best with current versions of Chrome, Edge, Firefox, or Safari.

See our <a href="https://edx.readthedocs.io/projects/open-edx-learner-guide/en/latest/front_matter/browsers.html" target="_blank" rel="noopener">list of supported browsers</a> for the most up-to-date information.

Do I need an OpenAI API key to complete the lab?

Yes. By default, this lab uses OpenAI embeddings and LLM calls. Before you begin, create a <code>.env</code> file at the project root and add your key:

<code>OPENAI_API_KEY=your-key-here</code>

Keep your key private and never commit it to version control. If you prefer not to use a paid API, the lab includes an optional extension to replace OpenAI Embeddings with a local model such as <code>sentence-transformers</code>, though that path requires additional setup on your machine.

How do I check whether my implementation is correct?

After completing each homework section (marked with <code>## homework:replace:on</code> / <code>## homework:replace:off</code> comments in the source code), run the automated test suite from the project root:

Tests validate your vector store, retrieval tool, eviction logic, and related behavior. Read the failure messages carefully—they point to what still needs to be implemented or corrected before you move on to the next task.