field notes / local inference

Seventy Minutes of Nothing

What actually happens when you unplug the coding agent from the cloud.

Apple M1 Pro · 32 GB Ollama 0.32.15 qwen3.8:27b · 16.8 GB pi 0.84.3 offline · 2 days

I wanted to know if a coding agent running entirely on my own laptop could build something real. Not a to-do list. Not a fizzbuzz. Something with state, timing, rendering, input handling, and a UI — a thing you could actually play.

So I built Tetris. No frameworks, plain HTML/CSS/JavaScript. It works. It's 1,086 lines across three files, and it took four and a half hours of session time across two days.

The Tetris '89 title screen: a cyan TETRIS wordmark on a dark blue well, flanked by orange-bordered LEVEL, SCORE, TOP SCORE and control panels.
The title screen. Cabinet chrome, NES palette, and a beveled-block treatment the model chose on its own.

The interesting part isn't that it worked. It's how it failed on the way there — and the fact that the single most expensive failure had nothing to do with the model's intelligence.

The stack

Everything ran on one machine, offline. The agent had the usual tools — read, write, edit, bash. It could see the filesystem and run commands. Nothing left the laptop.

The config that mattered, and that I didn't think about hard enough:

~/.pi/agent/models.json
{
  "id": "qwen3.8:27b",
  "contextWindow": 32768,
  "maxTokens": 8192
}

Hold onto that 32768.

The first fifteen minutes were great

I opened with one prompt:

I want to build a web version of the 1989 NES classic game "Tetris". No frameworks please, plain html, css, javascript.

Six minutes later it had written index.html. A few minutes after that, style.css — 200 lines of NES-flavored CSS with a beveled cabinet UI, a proper color palette, and a score panel. It even caught its own mistake mid-stream:

The file content is actually different from what I wrote — interestingly, the write tool's result reported 2785 bytes, but the file shows a different version of the title art… Whatever — I'll replace it now using the exact text that exists in the file.

That's real agentic behavior. It wrote a file, re-read it, noticed the mismatch, and repaired it without being asked. I was, at this point, feeling pretty good.

Then it said: "Now the game logic in tetris.js:"

And the wheels came off.

Seventy minutes, thirteen failures, zero progress

Here is the actual sequence, pulled from the session logs. Every bar is one attempt to write the game engine. Every one died the instant the model tried to emit it as a single tool call.

Session 1 · consecutive turns · stop=error err=terminated · in=0 out=0
22:51:40terminated
23:01:29terminated
23:08:27terminated
23:15:04terminated
23:15:57terminated
23:16:21me: "I had to restart the service, please continue."
23:23:15terminated
23:28:55terminated
23:34:05terminated
23:39:18terminated
23:42:08me: "Try again"
23:47:21terminated
23:47:50terminated
23:54:41terminated
00:01:36terminated
00:04:28me: "Continue"
Every terminated error in the entire two-day project occurred inside this one 70-minute window. All thirteen, in a row, all doing the same thing.

in=0 out=0 — no tokens billed, no usage recorded. The stream just died.

The really maddening part was watching the model's reasoning each time. Because each retry started clean, it re-derived the same design decisions from scratch, over and over:

Ghost piece? It's not in the NES version, but it's handy to have. Add a subtle ghost at low alpha — or maybe skip it to keep it 1989.

Ghost piece (optional; NES added ghost in 1990, but let's include a subtle ghost)

ghost? The NES '89 version didn't have a ghost piece… keep the classic: no ghost. That said, I'll add a subtle ghost block for usability

Three retries, three different answers to the same question. It re-litigated the scoring table too, landing on 40/100/300/1200 one round and 100/300/500/800 the next. It was doing an hour of thinking and shipping none of it.

What was actually wrong

At the time I assumed I was running out of memory. I closed every other app on the machine — Chrome especially — and things did seem to get better, so I filed it under "27B is too big for 32 GB" and moved on.

Going back through the server logs, that diagnosis was wrong. There is not a single out-of-memory error in any of them. Free system memory at every model load sat between 21.3 and 27.4 GiB, against a 16.8 GB model. Closing Chrome was real — the first load of the night had the least headroom of any, and free memory climbed 5–6 GiB once I cleared the decks — but memory was never what killed those thirteen turns.

It wasn't the model either. It was a number mismatch I created and never noticed.

My agent config advertised a 32,768-token context window. But Ollama's server was being restarted with a different ceiling, and I kept changing it while I flailed:

OLLAMA_CONTEXT_LENGTH across server restarts · local time
TimeCeiling enforcedAgent believed
18:0116,38432,768
18:1532,76832,768
18:4716,38432,768
19:048,19232,768
19:1816,38432,768
21:1132,76832,768

The agent never knew. It happily planned an 800-line file to write in one shot, because as far as it was concerned it had 32k of headroom. When the real ceiling was 8k, the generation ran into the wall mid-tool-call and the connection was severed. No error message the agent could reason about. Just terminated.

And there was a second, nastier detail buried in llama.cpp's startup output:

~/.ollama/logs/server.log
cmn  common_init_: KV cache shifting is not supported for this context,
                    disabling KV cache shifting

Ollama had launched the server with --context-shift, but the model's architecture didn't support it, so it was silently switched off. That's what turned a soft limit into a hard one. With context shifting, running out of room means the window slides and you lose old tokens. Without it, running out of room means the request dies.

Once I knew what to look for, the fingerprints were everywhere. Later in the project, once things were "working," I still saw this:

Truncations pinned to whatever ceiling was live at the time
— while the server was running -c 8192 —
stop=length   in= 8074   out=114
stop=length   in= 8080   out=108
stop=length   in= 8120   out= 71

— after I raised it to -c 16384 —
stop=length   in=16243   out=140
stop=length   in=16247   out=137
stop=length   in=16330   out= 54

The agent's UI showed none of this. It just looked like the model kept trailing off mid-sentence.

When you self-host, the number your agent believes and the number your server enforces are two different numbers — and nothing checks that they agree.

The workaround: make the model plan around its own limits

At 00:17 I gave up on brute force and typed this:

We are running out of context. We need to work in smaller modular chunks. What overall pieces do you need to write? I'll prompt for one at a time.

That turned out to be the whole ballgame. It came back with a seven-item build plan:

  1. Core model + piece definitions — grid constants, 7 tetromino shapes, NES palette, rotation, collision
  2. Game loop & gravity — spawn/lock cycle, requestAnimationFrame, fall-speed-per-level, game over
  3. Line clearing + scoring — scoring table, line counter, level progression, high score
  4. Rendering — beveled blocks, ghost piece, NEXT preview, HUD
  5. Input handling — keyboard and touch, soft drop, hard drop
  6. States & overlays — ready screen, pause, game over, restart
  7. Sound — WebAudio square-wave beeps with a mute toggle

I saved that list to a text file. And from then on, my workflow was: start a brand new session, paste the plan, say which piece we're on, let it work, quit before the context fills.

Six of my eight sessions open with almost exactly the same sentence — "Here's the status: index.html and style.css are done. tetris.js is in progress. We're building Piece 4 from the list below."

This is the part I'd tell anyone else doing this. A cloud agent lets you be lazy about state because the window is big enough to hold the whole project's history. Locally, the window is the scarcest thing you have, and the fix isn't a bigger window — it's not needing one.

The state lives in your prompt, not in the model's context. A short, re-pasteable status header is worth more than 24k extra tokens.

The session data backs this up. My first session — the brute-force one — ran 2 hours 11 minutes for 37 assistant turns and produced two of three files. The five "one piece at a time" sessions that followed averaged about 22 minutes each, and each one landed its piece.

What the small model was surprisingly good at

I want to be fair here, because the failures are more quotable than the wins.

It tested its own code. Unprompted. There's no browser in the loop, so it built itself a headless harness — stubbing out localStorage and evaluating the game source inside Node to exercise the logic:

A test harness the model wrote for itself
node -e "
global.localStorage = { _v:{}, getItem(k){return this._v[k]||null},
                        setItem(k,v){this._v[k]=v} };
const src = require('fs').readFileSync('tetris.js','utf8');
const fn  = new Function(src + '; return {SHAPES, rotateShape};')();
let t = fn.SHAPES.T;
for (let i = 1; i <= 4; i++) { t = fn.rotateShape(t); console.log(JSON.stringify(t)); }
"

It used that harness to catch genuine bugs — rotation mutating the shared shape constants, a collides() call indexing off the end of the grid, updateHud() blowing up when a HUD element was missing. It also learned, over several tries, that node --check tetris.js was a cheap way to catch its own syntax errors before running anything.

That is real engineering judgment from a 27B model on a laptop. I did not expect it.

Where it was consistently weak

Across the whole project, the tool-call error rate tells the story.

bash 30 / 18
edit 23 / 3
read 22 / 0
write 5 / 0
succeeded failed bash failure rate: 37%

Shell quoting

Repeatedly. Its favorite move was embedding $(cat tetris.js) inside a double-quoted node -e "…", which lets the shell expand the file's contents into the command line and produces gibberish:

[eval]:2 $(cat tetris.js)
             ^^^
SyntaxError: missing ) after argument list

It took several tries to converge on reading the file from inside Node instead of interpolating it from the shell.

Editing the wrong file

More than once, an edit call named one path and carried content belonging to a completely different file — CSS rules sent to index.html, HTML sent to tetris.js. To its credit, it usually noticed:

I carelessly executed an edit call to the wrong file with the wrong edit (I was intending to edit tetris.js, but ended up putting in the contents of index.html — this was a mistake).

Self-corrupting edits

My favorite. In a single call it introduced a variable called hudComplete, then in the very next edit of the same call referenced hudCOMPLETE, and then tacked on a third edit replacing hudComplete with hudComplete — a no-op that failed the whole batch. Because the edit tool is all-or-nothing, one junk edit threw away two good ones.

Design drift — and this one shipped

I asked for the 1989 NES game, and specifically prompted for NES scoring: 40/100/300/1200 per 1/2/3/4 lines. What's actually in the code is:

tetris.js:264 — shipped
const SCORE_TABLE = { 1: 100, 2: 300, 3: 500, 4: 800 };
// 7-bag randomizer, like modern Tetris

That's the modern Tetris Guideline table, not the NES one. Same story with the randomizer — the 1989 game used a famously streaky pseudo-random picker; the code uses a 7-bag, with a comment that says the quiet part out loud.

The model debated this with itself in the transcripts and repeatedly chose "better as a player experience" over "matches what he asked for." Then it wrote a README asserting "NES scoring rules" next to the wrong numbers.

It's not a bug — the game plays fine — but it's a spec deviation that survived all the way into the documentation, which is exactly the kind of thing that slips past you when you're reviewing a 70-minute session for whether it ran.

The numbers

8sessions
125assistant turns
1,169,580input tokens
57,410output tokens
13turns terminated
6turns truncated
1,086lines shipped

Input-to-output ratio: about 20:1. That's the number that reframes local agents for me. Agentic coding is overwhelmingly a reading workload — every turn re-sends the conversation, the file contents, the tool results. Your prompt-processing speed matters more than your generation speed, and prompt processing is exactly what degrades as context fills:

One 5,900-token prompt, ingested
n_tokens = 1024,  progress = 0.17,  t =  8.98 s / 114.01 tok/s
n_tokens = 2048,  progress = 0.35,  t = 25.09 s /  81.63 tok/s
n_tokens = 3072,  progress = 0.52,  t = 41.64 s /  73.78 tok/s
n_tokens = 5120,  progress = 0.87,  t = 75.14 s /  68.14 tok/s

Ninety seconds just to read the prompt, losing 40% of its throughput on the way. Generation ran 5–8 tokens/sec once context was deep. A response a hosted model returns in four seconds took two to seven minutes.

Four and a half hours of session time, and the overwhelming majority of it was the laptop reading.

Worth being precise about that number. Adding up the time between the first and last model response in each of the eight sessions gives 4 hours 28 minutes of session time. Subtract the dead zone and roughly 3 hours 20 minutes of that was forward progress. One stall, in one session, ate a quarter of the project.

Mid-game Tetris board with a stack of colored tetrominoes, a hollow outlined ghost piece near the top, an O piece in the NEXT panel, and a score of 000386.
Mid-game. Ghost piece outlined near the top, NEXT preview populated, score earned from hard drops — all of it written one numbered piece at a time.

What I'd do differently

Pin the context ceiling once, in both places, and verify it. Set OLLAMA_CONTEXT_LENGTH, set the agent's contextWindow to the same number, then confirm with ollama ps before writing a line of code. My seventy wasted minutes were entirely this.

Check whether your model supports KV cache shifting. If it doesn't, hitting the ceiling is fatal rather than lossy. Know which failure mode you're in before you're in it.

Never ask a local model for a big file in one call. Give it a skeleton and have it fill in one function at a time. The failure isn't that it can't write 800 lines — it's that an 800-line generation is a single point of failure with a five-minute fuse and no partial credit.

Make the model write your build plan, then own that plan yourself. Keep it in a file. Re-paste it every session. Treat the model's context as scratch space, not memory.

Review for spec compliance, not just for "does it run." The scoring table is the tell. Everything was green — it ran, it was tested, the README was polished — and it still quietly wasn't the thing I asked for.

Was it worth it?

For a two-day weekend project on a laptop, with no API bill and no network: yes, genuinely. The game works. It's playable. A 27B model wrote 800 lines of stateful game code, built its own test harness, and caught real bugs.

But the honest framing is that I spent most of those hours being the model's context manager. A hosted agent does that work invisibly, and the value of "invisibly" is much higher than I would have guessed before I had to do it by hand.

The model was never the bottleneck. The plumbing was.

The gap is harness work, not model work

That's the part I keep coming back to, because it's the optimistic reading. Everything that cost me time was a job a tool could do — and mostly a job a tool could do today, with no bigger model and no more RAM.

Here's what I'd want from pi, or from any local agent harness, in rough order of how much time each would have saved me.

  1. Handshake the real context limit at startup. My config said 32,768. The server was serving 8,192. Nothing checked. Ollama will tell you what it's actually serving — ask on connect, compare, and refuse to start on a mismatch. That one check turns a seventy-minute dead zone into an error message before the first prompt. It's maybe ten lines of code.
  2. Show a context gauge. Every hosted agent shows how full the window is. Locally it matters more, because overflow is fatal rather than lossy — and I had no readout at all. I was flying with the fuel gauge painted over.
  3. Budget the output, not just the input. My config asked for up to 8,192 output tokens. With a prompt already 8,100 tokens into an 8,192 ceiling, that request was arithmetically impossible, and the harness sent it anyway. n_ctx − prompt_tokens is the real output budget. If it's smaller than the task needs, say so instead of dying.
  4. Know whether overflow is survivable. llama.cpp announced KV cache shifting is not supported at load and nobody was listening. With shifting, hitting the ceiling costs old tokens. Without it, hitting the ceiling costs the request. The harness should read that line and compact early when there's no safety net.
  5. Automate the workaround I did by hand. Ask the model for a build plan, save it to a file, start a fresh session per task, re-paste the plan as a status header — that is a mechanical procedure. A harness could own it end to end: keep a durable project-state file on disk, watch the gauge, and when it crosses a threshold, summarize, checkpoint, and re-seed a clean context. Compaction is table stakes in cloud agents. It's worth more locally, and it's largely absent.
  6. Treat tool output as the main expense. A 20:1 input-to-output ratio isn't conversation — it's the same file contents and command output re-sent turn after turn. Elide large tool results by default, keep a handle so the model can re-read a slice, and drop results once superseded. This is where the tokens actually go.
  7. Make big writes structurally impossible to lose. An 800-line single-shot write is a five-minute generation with no partial credit. Cap write size and expose an append/patch tool, and it becomes six bounded calls that fail independently. The model doesn't need to be smart enough to chunk its output if the tool won't let it do otherwise.
  8. Classify failures instead of showing a blank turn. Thirteen times I got an empty response with no explanation, and it took reading raw JSONL two days later to learn they were all the same error. A dead stream is an overflow, a crash, or a cancel — the harness can usually tell which. Say so.

None of that is speculative research. It's product work on the scaffolding, and it's the difference between "a 27B model on a laptop is a toy" and "a 27B model on a laptop is a junior pair-programmer that never sends your code anywhere."

The models are already past the bar. The tooling around them is what's still catching up — and unlike waiting for the next model or the next Mac, that's the part anyone can go fix this week.

Playable → nickhirras.github.io/tetris-web
Source → github.com/nickhirras/tetris-web

← All field notes