I tested 17 local models with my coding agent. I kept 2
More than 500 runs to find out which local model can do the work of a coding agent: the method, the surprises and the two that were left.
I had downloaded too many models. That was the problem. There were so many on the disk that each one I tried looked great one day and useless the next, and I could no longer tell whether I was choosing or just trying. So I stopped guessing and started measuring.
I use a coding agent I wrote myself, donk. It reads files, searches text, edits and runs commands in a repository on behalf of whichever model sits behind it. The point of having built it is being able to swap that model: a cloud one when the task calls for it, a local one when the code is mine and I don't want it to leave the machine. What I wanted to know was which local model can handle the job.
I ran everything on a PC with a Ryzen AI 9 HX 470 and 64 GB of memory. It has three places where a model can run: the CPU, the integrated graphics (a Radeon 890M) and an NPU, the AI chip that now comes in every other laptop. The models were served by Lemonade.
How I measured
No reading an answer and deciding it looked right. Every task had an automatic check at the end. If the job was to fix a bug, the project's tests had to pass, and the model could not have touched the tests. If the job was to find a function, the answer had to be exactly file:line. If it was configuration, a script read the file and checked every detail that had been asked for.
Each run started from a clean copy of the project. I recorded hits, time and the model's writing speed, in tokens per second.
I did it as a funnel, because I wasn't going to spend an afternoon on a model that had already failed the basics. First: does the model load and manage to use the agent's tools? Of the 17, 14 did. Then five code tasks in a test project I built just for this: find a function, implement another, rename a name across nine files, fix a bug, find a value in a huge table. Nine were left, by the rule that anyone who got fewer than 8 of 10 and also took more than 100 seconds per hit was out. I picked three champions for the next stage, with system configuration, architecture questions and bugs planted in three of my projects. One of them, Qwen3-14B, dropped out there, and I brought Qwen3.5-9B back to compare. The final tiebreak, with 13 tasks, three times each, was between Qwen3.6-35B, Ornith-9B and Qwen3.5-9B. Counting everything, it came to more than 500 runs (the second phase alone was 160, and the tiebreak 117).
The NPU stayed out
It was the part I cared about most and the one that delivered least. Five of the models that run on it simply don't use the agent's tools. I suspected a bad installation, so I compared the text that reaches the model with and without the list of tools. It had the same size. The NPU works; the model just ignores the tools, and an agent with no tools can neither read nor edit anything.
One NPU model did work. It got 8 of 10 tasks right, but it took 381 seconds per hit. Ornith-9B, the fastest on the graphics card, took 40 in that same phase. For now, the NPU waits.
Thinking more didn't help
Some models have a reasoning mode, where they write a draft before answering. I had been running all of them with it off and started to suspect I might be hurting the ones built to think. I tested with and without on five models.
For those that were already getting it right, nothing changed; it just used more text and more time. Qwen3-14B got worse: it fell from 6 hits out of 6 to 4, and became almost five times slower, because it got stuck in a loop repeating the same edit. I kept it off.
The best at code was the worst at configuration
Qwen3-14B got all 10 code runs of the second phase right (five tasks, twice each). I expected it to be a finalist. In system configuration it got 4 of the 10 runs.
The way it failed is what bothered me most. I asked for a systemd service to restart by itself, run as another user and have a protection turned on. It made a single edit and answered that everything was done. I went to check the session and it was one piece only.
On the architecture questions it was similar. I asked whether a service with a single instance should use Redis or a cache inside the process itself, and it recommended Redis, without noticing that this puts a whole server in the way to gain almost nothing. It left the list. What I took from that sounds obvious, but I had only now understood it for real: being good at one category of task says nothing about the others, and whoever tests only code finds that out late.
Who stayed
| Qwen3.6-35B | Ornith-9B | |
|---|---|---|
| Hits in the tiebreak (out of 39) | 38 | 32 |
| Time per hit | 48 s | 31 s |
| Writing speed | 12.6 tok/s | 16.9 tok/s |
| Architecture questions (score from 0 to 1) | 0.79 | 0.85 |
| Bugs in real projects (out of 18) | 14 | 13 |
| Disk space | 21.7 GB | 6.2 GB |
Qwen3.6-35B became the default. It's the one that gets the most right, and the only one that fixed the hardest bug I planted in one of my projects. The other two finalists, Ornith-9B and Qwen3.5-9B (which got 33 hits out of 39, one more than Ornith, but left the list for being the slowest of the three and always missing the big tables), tried up to the 20-turn limit and never found the cause. Across the real projects the difference from Ornith is one hit (14 against 13), but it hides the hard bug: 3 of 3 against 0 of 3. Ornith missed that bug in all three tries and two more lookups. On the other hand Qwen3.6-35B is slow at renaming a name across many files, about 200 seconds, and at an SSH configuration task, about 80.
Ornith-9B stays for the small things: finding a function, tweaking a configuration file, answering a question. It's smaller, faster and was the best on the architecture questions. But it can't rename a name across nine files. It got zero out of three, where the 35B got three out of three. It has its blind spot, and now I know what it is.
If you want to see where each one hits or misses, on the 13 tasks of the tiebreak:
| Task | Ornith-9B | Qwen3.6-35B |
|---|---|---|
| Find a function (two tasks) | 6 of 6 | 6 of 6 |
| Implement a function | 3 of 3 | 3 of 3 |
| Fix a simple bug | 3 of 3 | 3 of 3 |
| Rename across 9 files | 0 of 3 | 3 of 3 |
| Find a value in a huge table (three sizes) | 8 of 9 | 8 of 9 |
| SSH configuration | 1 of 3 | 3 of 3 |
| Four other configuration tasks | 11 of 12 | 12 of 12 |
The times per hit in the earlier table come from the tiebreak. The ones from the code phase, like the 40 seconds above, belong to a different set of tasks, so the numbers don't repeat.
One thing that helps explain the speeds: on integrated graphics, what limits you is memory, not arithmetic. That's why an MoE model, which uses only part of its parameters for each word, does well. Qwen3.6-35B, with 35 billion parameters, writes at about 13 tokens per second on the code tasks. A dense 14-billion model, much smaller, came in at 4, on the same tasks and with reasoning off.
Running both at the same time
I felt like keeping both loaded and choosing per task, so I tested it.
Ornith-9B on the CPU works. In a separate round of 15 configuration runs it got all of them right on the CPU and all of them on the graphics card, writing at 11 tokens per second against 18. Slower, same hits. (It did better on SSH there than in the tiebreak, where it got 1 of 3: one more reminder that few repetitions mean noise.)
One on the graphics card and the other on the CPU also works, but they get in each other's way. The memory is the same for both, so each lost about a third of its speed. Not a single hit fewer.
Both on the graphics card took down my session. Twice. The second time I looked at the system log, and both crashes had the same line, seconds before the session dropped: the graphics card saying it had rejected a command for lack of memory. My reading is that the integrated card shares memory with the rest of the system, and two large models plus the graphical interface don't fit together. I don't know if that's all of it, but I wrote it down and went back to one model at a time.
Before that I also lost part of the work. The test program and the partial results were in /tmp, which goes away when the machine restarts. The important numbers were already written down, but I now keep everything on disk.
What I don't know
Each task ran two or three times per model. A difference of one hit, like 32 against 33, is noise, and I was careful not to draw conclusions from it. I measured only what I use: Rust code, configuration files and design questions. I don't know how these models do in Python, in frontend work or in projects with hundreds of files. The speed numbers are valid for this machine.
I tested reasoning on five models. On a model really built to think, it may help more than I saw. And I eliminated some that may come back: a few NPU ones I haven't tried yet, and one that fell out because of a problem in its prompt format and could return fixed.
If you want to repeat something like this, the path is simpler than it looks. Pick a few tasks that really matter for your use, write a check for each one, and start with too many models, cutting at every phase. What took me a few hours to find out fit in a two-column table.