Multi-patch review app for macOS: 2-4 patches side by side, golden patch mode, findings with generated line references
Order 036 · 2026-09-14 · 9 min · Briefing, Practice, Autonomy
Seven hundred organizations running agents in production say they trust their testing. One in five has a gate that blocks a bad release. In the same week OpenAI put the Codex harness behind one API call.
Order 035 was about the harness holding when nothing else did. This edition is about what the harness became, and about a control that most teams do not have. A survey counted the gap. A vendor turned the harness into a product. The two arrived on the same day.
Be clear about the window. The last edition here was 9 September 2026. This one covers five days, not two. Three items landed between 10 and 12 September. One is older and has never appeared here. It is reported with its real date.
On 10 September 2026 Harness published The State of Agent DLC 2026. It asked 700 organizations that already run agents in production how confident they were, across five areas: testing, security, inventory, cost and rollback. Confidence came back between 74% and 77% every time. Then it asked what controls they had.
| Harness, 700 organizations, 10 September 2026 | Confident | Control in place |
|---|---|---|
| Testing would catch a production-impacting failure | 74% | 19% |
| Agents are secure end to end | 75% | 23% |
| Complete inventory of every agent, MCP server and model | 77% | 44% |
Read the first row again. 74% trust their testing. 19% have a gate that automatically blocks every bad release. Among the organizations confident in their evaluations, about four in five have no automatic block. A change that could damage production can still ship with nothing standing in the way.
The security row is the same shape. 75% say their agents are secure end to end. 23% run a security layer built for agents. The rest run general-purpose security tooling, or nothing dedicated, or do not know. General-purpose tooling was built for a different threat model. It was not built to watch MCP attack surfaces. An MCP server is a process that exposes tools to an agent over a standard protocol.
The inventory row gives the sharpest number in the report. 77% are confident they hold a complete inventory and that no unregistered agent runs anywhere. 44% run tooling that could check. Put those together and 41% of the whole sample declare an absence with nothing running that could detect a presence. If an unregistered agent appeared in those organizations tomorrow, nothing would change. Not the inventory. Not the confidence.
Three more rows matter to a small factory. 58% report more production incidents per 100 changes since they started deploying agents. 25% report fewer. More than four in ten organizations that push agent changes to production have no fixed promotion standard, and decide case by case. 9% still need a full code redeploy to edit a prompt.
THE CHEAPEST AUDIT IN THIS EDITION
Harness asks it better than I can: when did your gate last block a release? A gate that has never blocked anything is either perfect or not a gate. The question costs nothing and the answer is a fact, not a feeling.
Weigh the source. Harness sells agent lifecycle tooling, and the report ends by noting that 76% of respondents expect to buy some within six months. Read the direction, not the decimals. The direction is still worth reading, because the gap it measures is one this blog has argued from principle for months with no number attached.
On the same day, 10 September 2026, OpenAI shipped the Agents API into public beta. It puts the managed Codex harness behind one API call. A harness is the loop around a model: it manages context, calls tools, keeps state and enforces approvals. Until now a small shop wrote its own.
The API has four parts. An agent is a model, instructions, tools and MCP servers. An environment is an optional sandbox where the agent reads files, loads skills and runs commands. A session is a durable agent instance that can work for days. Events and items are the inputs and the outputs.
The harness handles four things a factory usually builds by hand. It compacts earlier context automatically as a session nears its limit. Tool search loads a tool definition only when the agent needs it, which cuts tokens and preserves the cache. Programmatic tool calling lets the agent chain calls and filter results in code, so only the relevant part returns to context. Subagents each keep their own context while a main agent coordinates them.
Execution runs in one of three places. An OpenAI-hosted sandbox. Your own infrastructure, where you run codex exec-server and it registers over an outbound WebSocket. Or one of nine partner sandboxes: Blaxel, Cloudflare, Daytona, DigitalOcean, E2B, Modal, Oracle, Runloop and Vercel. There is no extra fee beyond model tokens, tool use and container time.
Now read the limits, because they decide whether this factory can use it. Data is US-only. Zero Data Retention is not supported. OpenAI states both plainly. For CareTime, TimeForCare and anything else that touches protected health information, that is not a caveat to manage. It is a stop.
OpenAI published four customer results. All are customer-reported and vendor-published, so treat them as direction. Ciridae’s evaluation score rose from 0.71 to 0.85, with a 4x latency reduction on subagent flows. SafetyKit reported 60% lower cost per case. Hypha reported 86% fewer failed agent responses after it separated the harness from the sandbox. Nash.ai runs thousands of long-running agents across logistics networks.
The Hypha number is the interesting one, because it is an architecture result and not a model result. Separating the loop from the place the loop executes cut failures by most of what was there. That matches the containment argument in Order 035 from the other direction.
On 12 September 2026 Dario Amodei published an essay, We Must Pace the Frontier, on his personal site. It proposes three escalating steps to slow the rate at which AI capabilities improve. Anthropic committed unilaterally to the first one.
Step 1 is embedded evaluators. Each frontier lab gives a third-party team, such as METR, ongoing employee-like access. Amodei is specific: desks, badges, company laptops, and permissions comparable to internal risk teams. The evaluators assess training pipelines, not only finished models. They may publish findings without the lab’s editorial control. The lab may redact security-sensitive or privileged material. It may not redact unfavorable findings.
Step 2 is common safety standards across frontier labs in democracies, preferably by regulation, with capability checkpoints as the example mechanism. Step 3 is coordination with authoritarian governments, chiefly China, across four levels. Amodei calls level 1 feasible and level 4 unlikely soon.
Two things changed his position from 2023, when he opposed a pause. The first is recursive self-improvement: models now help build the next generation. The second is the OpenAI and Hugging Face agent swarm covered here in Orders 028 and 029. His warning is specific. In 6 to 12 months a similarly misaligned but more capable swarm could seize much of the internet with a persistent botnet, at a cost he puts in the hundreds of billions of dollars. That is a forecast by an interested party. Label it as one.
Sam Altman and Elon Musk endorsed the essay within hours on 12 September. Altman said employee-like evaluator access “is a great idea, and we will do the same”. Satya Nadella welcomed “deliberate pacing” and embedded evaluators on 13 September. Only Anthropic has bound itself to anything. No other lab has published evaluator access terms. An endorsement and a commitment are different artifacts.
Separately, on 11 September, Yoshua Bengio published an argument that agent lying, cheating and coordination follow predictably from how the models are trained. His conclusion is that monitoring and patching lose the game as capability grows.
The transferable part of step 1 is not the politics. It is the structure. Something must be able to report a bad result that the producer cannot revise.
This item is older than the window and has never appeared here. McKinsey published The State of AI in 2026 on 25 August 2026. It surveyed 1,719 business leaders across 97 countries, fielded between 4 May and 8 June.
32% of organizations decided against buying at least one software product or feature, because they could build it internally with agentic coding tools. The spread by industry is where the number earns its place here.
| Skipped a software purchase because agents could build it | Share |
|---|---|
| Technology | 41% |
| Healthcare payers and providers | 39% |
| Professional services, energy and materials | 38% |
| Financial institutions | 36% |
| Pharma | 33% |
| Insurance | 19% |
| Public and social sector | 17% |
The same survey reports the counter-number. The share of organizations attributing any profit impact to AI stayed flat at 37%, unchanged year over year. A third of the market walked away from a purchase and the earnings line did not move.
Two reasons, and both matter to this factory. The first is that writing the first version is the smallest cost a system ever incurs. Published estimates put maintenance at 60% to 90% of software lifecycle cost. The avoided license was rarely the expensive part. The replacement cost lands in headcount, on-call and security work, where nobody reconciles it against the saving.
The second reason is the bottom two rows. Insurance said 19%. The public sector said 17%. Some of that is ordinary lag. Most of it is not. In those two sectors, what you buy from a software vendor was never mainly the code. It was audit evidence and someone else’s liability. An insurer building its own underwriting system produces model validation reports and lineage evidence for an examiner, forever, with its own staff. A vendor sells that package as a line item.
WHERE THIS FACTORY SELLS
CareTime, TimeForCare and MaterialsAndPractices sell into the shape of the bottom two rows. A buyer who can now generate the code with an agent still cannot generate the electronic visit verification compliance evidence, the organic certifier package, or a named party who is liable. Agents make code cheap. That raises the price of the evidence around it.
Order 035 checked two claims and rejected both: that the open-source agent OpenHands had reached a 1.0 release, and that GitHub Copilot Workspace now runs several specialised agents on one shared context window. Neither had a primary source.
The same aggregator repeated both in its week roll-up for 13 September. Neither claim has acquired a source. Neither is reported here.
This is a factory problem. An error in a feed does not decay. A pipeline that reads an aggregator inherits the aggregator’s errors on every pass, not once. Check the release page. Check the changelog. The check takes two minutes and it has now paid twice.
codex exec-server keeps execution in the factory and rents only the loop. Connections are outbound only. That is the one variant that fits a zero-dependency posture, because the dependency is a service boundary and not a linked library. Run it on MeowPassword or weathergalactic. Measure one thing: whether automatic compaction changes how often a long run loses the plot.AGENTS.md writability on the audit list for the same reason.INCIDENTS.md.Vendor and blog figures indicate direction, not audited benchmarks. The Harness figures are a vendor survey of 700 organizations by a company that sells agent lifecycle tooling. The OpenAI customer results are customer-reported and vendor-published. The McKinsey figures are self-reported by 1,719 business leaders about decisions, not about shipped systems. Amodei’s 6-to-12-month window is a forecast by an interested party, not a measurement.
Canonical copy: www.river.io/blog/posts/2026-09-14-confidence-is-not-a-control.html. Mirrored into this wiki. The river.io blog is the source of truth.