Build it from scratch. Understanding fundamentals of how agentic coding harnesses is a must though if you gonna go that route. I think everyone should take time and learn these things, maybe reverse engineer Codex Cli or something like that as a starter. That info is very valuable in this day and age.
Can you say more about Codex? I'm using GPT-5.5 in my own harness and it's not liking it very well, so I'm thinking I ought to make it more Codexy so it's more ergonomic for it. (edit format, tool calls etc.) But haven't gotten around to it yet.
In short its a good idea to have tool calling be closely representative to what the model expects as these models are tuned to their own preferred way of doing things, it will surely save you lots of time. The disadvantage is that now your harness system is not as model agnostic as you would like and also you will have to keep up in changing landscape by adapting the tool calling structure with major updates for best results. Its a personal decision you will have to make for yourself. Personally my harness system uses its own way of doing tool calling as I am trying to experiment with simpler tool schema's that also work for smaller less intelligent models but I have yet to do enough A/B testing to say that is a smart approach. As time goes on I think the smart thing to do might be to set up an adapter type of module that changes its tool schema's based on underlying model used for the agent. This preserves optimal behavior patterns with little investment from me. You might have to adjust system prompt in some minor ways as well so keep that in mind. As far as codex i prefer it as i like the way Open Ai does things in that harness system (the spirit if you will), there's interesting tidbits I always find and while I don't usually use them for my own harness system they are inspirational in other ways. you can gather what the devs were trying to achieve with certain implementations.
Thanks. Yeah I noticed that models react very differently to the same harness. I've had small models that do great and big ones that fail in weird ways.
Generally the smaller ones are less flexible, but they're much cheaper and faster so, I try to design for the smaller models.
One thing I noticed is that if you get the loop fast enough, the work starts to feel different. It becomes real time and interactive. As opposed to the usual feeling of, having to wait a few minutes every time.
Another thing I noticed is that even very small models are capable of making many edits simultaneously. But most harnesses don't seem to design for that. Even editing one file they have to do a whole bunch of turns.
Whereas you can just edit several files in a single LLM call and it just works fine. So you get an order of magnitude speed up, as well as a reduction in costs.
Not the comment author, but I use pi and customize it with my own extensions. Pi automatically tells models how to customize itself, so it's a pretty easy process.
I started mine from scratch in 2023 because I wanted to use LLMs from a terminal and there was nothing else compelling at the time (nowadays there is pi and opencode)
Harnesses are/can be incredibly simple things, not much more than a HTTP client that renders things in a way that suites your taste.
It’s not that difficult, it’s just a system prompt and a set of basic file edit/bash/etc tools.
Me, personally, I didn’t build it from scratch but I ported original CC from published sources into Python and extended it to match my own requirements.
I use GLM in my custom harness. It completes the same tasks at the same level of quality, except 8x faster and 8x cheaper. (Same goes for GPT!)
I'm not sure how that's possible. I expected to get increased correctness for that order of magnitude (something something test-time compute!) but I am not getting it.
