The Ollama Context Window Trap: Why Your Local Agent Describes Code Instead of Reading It

Ten videos about running a coding agent on your own machine have pulled roughly 1.9 million views this quarter. The one video titled "Stop Using Default Ollama Settings for Coding" has 111.

That gap is the whole problem. The setup guides are excellent and the demand is real, but the Ollama context window, the single setting that decides whether any of it works, gets almost no attention. And when it is wrong, the failure does not look like a configuration error. It looks like a stupid model.

What the Ollama context window default actually breaks

Your agent stops being able to call tools, so it starts describing work instead of doing it.

That is the tell, and it is worth recognising because it is so easy to misread. Devsplainers put it in one sentence in a video with 142,619 views: a developer pointed a big cloud coding agent at a small local model on a 32 GB laptop and "watched the model confidently describe his code without ever once opening a file."

The mechanism is not mysterious. A tool-calling turn has to fit, inside the context window, all of this at once: the system prompt, the definitions of every tool the harness exposes, your request, the tool's response, and the model's formatted call. Squeeze the window and the tool definitions are what fall out. The model still answers; it just answers from imagination, because it never actually got to read anything.

And the numbers you need are not agreed on anywhere:

Four different numbers across four primary sources, and not one page that reconciles them. That is why the SERP for this is held by GitHub issues rather than by a guide: nobody owns the answer.

Why people set it too low in the first place

Because the context window is not free: it rents space from the same memory the model does.

This is the part the config tutorials skip, and it explains the behaviour better than "people didn't read the docs." Devsplainers again, on the memory math: budget roughly half a gigabyte per billion parameters, so a 32B model wants about 20 GB just to sit there before it reads a line of your code. And then: "your context window rents space from the same pool. A long one can eat 10 gigs by itself."

So the person who trimmed the window was not being careless. They were trying to make the model fit on a 24 GB card, the number everyone converges on, with 16 GB described as fine for autocomplete and not much more.

Here is the twist that makes the whole trade-off collapse, and I have not seen it stated anywhere else. In a walkthrough with 135,227 views, Callum at Wanderloots builds a 64k variant of Gemma 4 E4B and shows ollama ps before and after. The model goes from 3.3 GB to 3.4 GB. Doubling the working memory cost him about a hundred megabytes.

The fear is real at 32B. At the sizes most people actually run, the window is nearly free and the default is costing them the entire agent.

Four ways to set it, and the one command that tells you the truth

Start with the diagnostic, because most people have never checked what they are actually running: ollama ps prints the context window of the loaded model. Until you run it, you are guessing.

Then pick a lever:

A model variant. Pull the base model, set the context, ollama create a new name. This is what Wanderloots demonstrates, and the important property is that a variant does not duplicate the weights: "it doesn't take up another 10 gigs." You get gemma4e-64k sitting beside the original for the cost of a config.

A Modelfile parameter. PARAMETER num_ctx 32768, then ollama create. Same idea, written down in a file you can commit.

The service environment variable. OLLAMA_NUM_CTX=32768 on the Ollama service, if you want it applied without minting variants.

The application setting. Newer Ollama exposes a context length in Settings, which forces every model to launch at that window. Blunt, but it is the one-click version if you only ever run agents.

There is a fifth path worth knowing for a one-off: ollama run <model>, then /set parameter num_ctx 131072, then /save <model>-128k.

Wiring it up afterwards is the easy part. Ollama exposes a native endpoint, and appending /v1 makes it OpenAI-compatible, which is what Hermes, Claude Code, Codex and OpenClaw all speak. As Callum puts it, that is why these things are called harnesses: the model is the engine, the harness is what gives it hands.

The other context problem, and why people conflate them

The window bites twice, and only the first bite is a setting.

The second one is that the window fills up while you work, and this happens on every agent, local or frontier, cheap or expensive. Claude Code's own context window docs make the starting position explicit: CLAUDE.md, auto memory, MCP tool names and skill descriptions are all loaded before you type a single character. You begin every session already spending.

These two problems get discussed as one thing because they share a name, and that costs people time. If your local agent never reads a file, that is the setting. If your agent read forty files and has now started forgetting the first ten, that is the discipline. Fixing the first one takes a command. Fixing the second one takes a habit.

How we handle the second one

Briefly, because this is our practice rather than the argument.

I watch the context constantly and I clear early. The part that matters is what happens immediately before the clear: I ask the agent to prepare for the reset first: write down where we are, what is decided, what is still open. Only then wipe. Clearing without that step throws away the reasoning and keeps the mess.

The docs explain exactly why that works, and it is worth reading the table rather than trusting instinct. On compaction, CLAUDE.md, auto memory and the plan from plan mode are re-read from disk. The conversation is not. Claude Code re-reads at most five of the files touched in the session, most recently modified first, and any file over 5,000 tokens comes back as a path reference without its content. So the rule falls out on its own: anything that must survive a reset has to be on disk before you trigger it.

Two refinements from the same page that I did not know before writing this. /compact accepts an instruction (/compact focus on the auth bug fix), so you can choose what the summary keeps instead of letting the automatic pass guess. And re-injected skills are capped at 5,000 tokens each and 25,000 total, with truncation keeping the start of the file. That last detail is a direct instruction for anyone writing skills: put the load-bearing rules at the top of SKILL.md and push detail into references. We had built our article skill that way by instinct; it turns out to be the documented requirement.

Where this does not save you

A bigger window will not fix a model that is too small. Reliable tool calling shows up somewhere around 27–35B parameters. Below that, per Devsplainers, "older, small models faked it — they'd happily describe code they'd never opened," and no num_ctx value changes that.

Nor will it fix over-compression. Quantise too hard and the damage is visible in exactly the places you care about: "broken tool calls, malformed JSON, switching language halfway through a function." The working rule is a sharp 27B over a lobotomised 70B.

Do not point a frontier-scale agent at a 7B and expect it to hold. The big cloud harnesses assume the frontier's flavour of tool calling and a huge window; they faceplant on small models regardless of configuration.

And local is not automatically the cheaper answer. The honest framing I found is hybrid: let a frontier model plan the messy thing, let the local model grind the well-specified pieces for free, escalate the hard 20%. Own what you can, rent what you must. Mitchell Hashimoto's verdict still stands for the hardest work: on a $5,000 machine, local is not there yet.

What I take from this

The interesting thing is not that a default is wrong. Defaults are wrong all the time. It is that this particular wrong default produces a failure that looks like a different problem entirely, and the entire content shelf is pointed at the promise rather than the mechanism. 1.9 million views on "run it locally," 111 on "stop using the defaults."

If you are debugging a local agent that seems confidently useless, check ollama ps before you change models, change quantisation, or conclude that open weights are not there yet. It is a thirty-second check, and on the model sizes most people run, the fix costs about a hundred megabytes.

Everything after that is the second problem, and that one you cannot buy your way out of.

FAQ

What context window does an AI agent actually need with Ollama? Depends on the harness, and they disagree: OpenClaw's error names 16000 as its minimum, Hermes asks for 64000 to give the agent tools. 64k is the value most advice converges on. Check the harness's own docs before picking a number.

How do I see the context window my model is currently using? ollama ps. It prints the loaded model with its actual window, which is the only way to know whether your setting took effect.

Does a bigger context window use more memory? Yes. It shares the same pool as the model weights, and a very long window can cost gigabytes on a large model. But at small sizes it is nearly free: a 64k variant of a 3.3 GB model measured 3.4 GB.

Why does my local agent talk about my files instead of reading them? Almost always because the tool definitions do not fit in the context window, so no tool call is made and the model answers from its own assumptions. Raise num_ctx first; only then suspect the model.

Is /clear or /compact better in Claude Code? Different jobs. /clear is for switching to unrelated work. /compact keeps a summary, and it takes an instruction (/compact focus on X), so you control what survives. Either way, write anything important to disk first, because the conversation does not survive and files on disk do.

---

We build AI agents and the infrastructure around them as an agency. See Next Estate, the property SaaS we run with 900+ residential complexes and 110,000+ apartments, and CashYou, a Telegram-based exchange platform we built end to end. More on what we do and who we are.