This summary is produced by the author, and not by AI.
The most important task for you during agentic development is to create, gather, and curate context. Context is any information that the model ingests to produce a more relevant and useful response. Providing the right context at the right time helps agents produce better results; it’s about more than just writing markdown files:
In this article we give some guidance on how you can curate context for agentic development in business intelligence.
When working with AI agents, common everyday words take on specific technical meanings. Before we begin, it’s worth very quickly defining a few of them:
In short, there’s different types of text that LLMs work with, which it might ingest and use in different ways.
As mentioned in an earlier blog post about LLM fundamentals, LLMs work by taking in and outputting text. They do this one token, or chunk, at a time. Even AI agents work this way when they are calling tools such as reading a file or running a command in the terminal. An agent harness reads the output text, intercepts it, and runs the tool calls on behalf of the agent.
As a result, LLMs have no memory by default. They operate purely based on the input text and the numerical details of the language model, iterating in a loop. You can think of this like calling a function in a programming language. Because the model is dependent on the input text, this makes managing those inputs important for quality results. Memory functionality is sometimes added later via specific tools or saved markdown files, but even that is added into the context.
The key part to understand is that LLMs have amnesia (metaphorically). They wake up with no memory or history and have to work based on whatever little scraps of paper you slide over to them. You are charged for every word on those scraps of paper, so you want to keep them small.
The sum total of all the text inputs given to an LLM is context. This includes the system prompt, the user prompt, any tool descriptions, the conversation history, and so on. Much of this is invisible to you as a user, but it’s still important to manage. Finally, the context window is the maximum amount of text, measured in tokens, that an LLM can process.
If you use a coding agent like Claude Code or GitHub Copilot, you can run /context to get a visual representation of how full your context window is. A distinction that will become important later on in the article is context that is loaded each and every session and context that is specific to a session. The former is important to manage because it represents a tax on each and every session:
The problem with too much context is threefold:
The more input text you provide an agent, the worse its performance is, even for simple tasks like recalling what was said. This is often called context rot. So, even though your model may support a million tokens of context, you will often get much better results with just 100-250k of context. Think of the window size like the maximum safe weight on an elevator. Just because you can fill it to the brim doesn’t mean you should.
It’s not just about volume, but what the context is. Agents are prone to focusing on irrelevant context; they’re easily distracted.
LLM providers charge based on input and output tokens. So, the more irrelevant context you provide, the more expensive the operation. All of this is to say managing the context an agent receives, also called context engineering, dramatically affects price and efficiency of AI agents.
LLM providers often offer a significant discount (50-90%) for cached input tokens. Coding agents will write to the cache automatically but some APIs may require you to manually do so and charge you extra for cache writes. For long-running work, the benefit of input caching is huge.
This has an effect on how you want to order your context, typically. Cache hits match based on the prefix, or beginning, of a conversation. This means that you want consistent items, like your system prompt, to be at the beginning and variable items, like the current time, to be in the end of your context window. See Anthropic’s documentation for more info.
In a previous article about AI readiness, we describe the context window as more like a context budget. Your goal is to find that sweet spot in the middle where you give it enough information to guide it; nothing more.
With AI skills or MCP servers, it can be tempting to add as many as possible. Unfortunately, however, this adds a constant drag to your agents in the same way as having a heavy load on the back of a bicycle makes it hard to steer and stay upright. So, what type of context should you add and how should you manage it?
First, you should keep always-loaded context to a bare minimum. Depending on your agent harness, such as GitHub Copilot or Claude Code, files like AGENTS.md or CLAUDE.md will be loaded into every session. This should be kept very short, with pointers to reference files for occasionally needed information. Pretend that you are being charged 5 cents for every word you write in here.
Splitting the context into core context and loadable references illustrates a concept known as progressive disclosure. Again, imagine you are being charged 1 cent for every single word that is read by the agent. You wouldn’t want it to read the full instructions every single time. You would want to progressively share, or disclose, more and more of the information the agent needs.
AI Skills are optimized for progressive disclosure. They contain a YAML frontmatter that describes what the skill does. This is often all an AI agent ever sees. If the user or the agent decides to invoke a skill, then the agent reads the rest of the skill file, which is written in Markdown. Then, the AI agent may decide to read bundled reference files, if any.
MCP servers are somewhere in between. Originally, many MCP servers were very token-inefficient because their tool descriptions were always loaded into context. Different agent harnesses have different workarounds for this. Claude Code has a tool search tool (a bit meta, right?) while GitHub Copilot groups different tools together as virtual tools.
While these optimizations help, we recommend installing skills and MCP servers per-project where it makes sense and trying to keep as few user-level skills and MCP servers as possible.
So, what happens if you try to exceed the context window? In many agent harnesses, it will automatically compact, or summarize, the entirety of the agent session. Then it will start a fresh context window based on that summary. You can often manually run a /compact command to do this yourself.
Now, imagine if you had a two-hour meeting and unexpectedly you were asked to summarize it in a few sentences. Let’s say this would then be used as the agenda for next week. You would likely forget important details or even misremember certain details. Compacting is the same. It’s a lossy process and if done more than once, it starts to quickly degrade the quality of a session.
Instead, you should regularly start a fresh session or use a command like /clear to clear the session context. This is also where subagents are very useful, if your harness supports them. Because a subagent will have its own context, it can do work like exploring a codebase or reviewing code commits and then return the result to the main agent without polluting the main agent’s context.
One of the reasons why we don’t recommend letting an AI document your data model for you is because large language models are verbose. Their whole job is to generate text, so it’s not surprising that they generate a lot of it.
The same advice applies to writing AI skills and documentation that go back into an agent’s context. It can be tempting to tell the agent to write a skill for a given task. Unfortunately, in addition to being verbose, agents will include details that are mildly relevant but can be easily rediscovered if needed. It’s reasonable to use AI to help you with a first draft if you are feeling overwhelmed. But ideally, you should be writing the skills and documents that go into your context by hand.
In both cases, even if AI was good at writing concisely and clearly, there are likely specifics to your organization that you want to be putting into the guidance and documentation an AI agent will read.
Human-written context is better than AI-generated context largely because you’re documenting implicit information and making it explicit. In contrast, what an AI generates is just summarization of what’s already there, or an imperfect representation of your discussions and documents that you’ve given it. Relying on AI for context curation is like playing a game of telephone, where the meaning and intent of the information – often which is quite subtle and nuanced – gets quickly buried under idiosyncratic AI writing.
Managing context for agents isn’t a one-off, monolithic task. You can spend a lot of effort gathering and setting up the context, but you’ll need to keep updating it over time as your business, tools, preferences, and project evolves. If you only add, though, then the context quickly gets too much, and the outdated information becomes irrelevant.
To avoid this, you need to think about curating the context, like a diligent librarian would the books on the shelf. Things need to be organized, clean, and in the right place. Managing context is dull, hygienic work that keeps a workflow healthy.
A simple way to get started:
LLMs are heavily dependent on the context given to them. Much of this can be controlled by you in terms of the files, skills, and MCP servers you provide to an agent. The key thing is to find the Goldilocks zone of enough context to do the task but not distracting or irrelevant information.
Take your semantic models further with Tabular Editor.
Give Tabular Editor a spin