Building the Software Factory: How We Shipped Payment Links with Symphony

Harness engineering has changed the software engineer's work. Instead of supervising every agent turn, we designed a production system in which agents can implement, test, prove, review, and improve themselves, with humans setting direction and owning the final product that's delivered. Coinbase Business Payment Links and Products was our team's first serious test of an "autonomous software engineering" team on a Tier 1 service. We had "zero" full-time engineers on the project team, with a few people setting direction and the rest of the human team providing support with final reviews.

Part 1: Taking humans out of the day-to-day loop, while still owning the outcomes

Harness engineering is the engineering topic of 2026.

The frontier models are getting better every day, but it's also clear that the "layers of harnesses" that surround the model are as important, if not more. The teams getting the most leverage out of LLMs are building this machinery to manage durable context, tools, permissions, tests, review agents, state machines, and feedback loops. For our work, we were heavily inspired by OpenAI's writing on “Harness engineering”, which introduced the Symphony spec, and we ran an experiment with it on a real-world project.

The 2025 loop

The default 2025 workflow kept a human in the innermost execution loop:

1. A human prompts a task.
2. The agent goes and writes up a bunch of code.
3. The human reads the output, verifies the change manually, and points out what is wrong.
4. The agent iterates, and the process is repeated until the pull request is opened.
5. Automated PR review runs on the PR; the human asks the agent to address PR feedback.
6. The PR eventually gets reviewed by other humans, who tend to verify changes manually as well.

This workflow is structurally very limited since the engineer is still the scheduler, context store, test runner, reviewer, retry policy, and process supervisor. Every additional agent creates another chat window that demands attention. Engineers started flexing whether they can multi-Claude 4 or 8 or 12 sessions at a time, but harness engineering makes that flex moot by asking engineers to completely step out of most of the sessions.

The 2026 loop

The 2026 pattern moves the human out of the inner execution loop and into an outer control loop.

The human still owns:

• what is worth building;
• the product and technical direction;
• the acceptance criteria and risk boundaries;
• the final review and production accountability;
• exceptions where judgment, taste, or organizational context matter.

Inside those boundaries, a separate machine loop can run without continuous intervention:

implement → test → inspect failure → review → fix → re-review → publish evidence → wait for final acceptance

The goal is to minimize human participation in routine execution while increasing human leverage over system design, evaluation, and prioritization.

The chip-design analogy

As an aside, a useful analogy comes from chip design and production.

NVIDIA engineers design chips. They define architectures, build simulation and verification systems, prepare a tapeout package, and work with manufacturing partners on constraints. TSMC operates an extraordinarily sophisticated production process that turns designs into physical chips. NVIDIA’s chip designers do not personally stand beside every wafer or monitor the production of every individual GPU.

That separation is the source of leverage we're trying to build into the workflow of software engineers. The chip designers invest in the quality of the design, the interface to the factory, the verification process, and the feedback from yield and post-silicon validation. The factory invests in repeatable execution.

The analogy maps surprisingly well:

Chip system	Agentic software system
Product requirements and architecture	PRD, TDD, issue, and acceptance criteria
Design rules and process design kit	Repository conventions, workflow, skills, and tool contracts
Tapeout package	A dispatchable Linear issue with enough context to execute autonomously
Fab and manufacturing execution	Symphony plus the selected coding-agent CLI
In-line metrology and yield gates	Tests, CI, automated reviews, and evidence capture
Post-silicon validation	Human review, rollout, monitoring, and customer feedback

The analogy isn't perfect, but it works. The key takeaway is that "production" of software can now be (mostly) decoupled from the "design" of software: separate design and control from routine execution, and make the interface between them exceptionally good. Humans should design the factory, inspect its outputs, improve its process, and own the consequences. They should not have to pull every lever on every production run.

Part 2. What we shipped: Business Payment Links

Coinbase Business brings the best of Coinbase to businesses: trading, borrowing, lending, stablecoin payments, checkout APIs, and much more.

For this "software factory" experiment, our team wanted to pick something with real stakes, instead of a "toy project" that's used internally. Payment Links v2 was an unusually good project for testing this operating model.

First, it had clear business value. Coinbase Business customers use payment links to collect stablecoin payments without building a full checkout integration. The existing service is Tier 1 and processes material monthly payment volume.

Second, the implementation surface was large and reflective of any medium-sized project. The work spanned:

• a Tier 1 backend service and a GraphQL/API layer on the backend;
• authenticated merchant product and payment-link creation experiences on the frontend;
• the logged-out buyer payment page;
• migrations and backfills from Payment Links v1;
• feature flags, lifecycle semantics, validation, privacy, compliance, security testing, and observability.

Third, it sat in the sweet spot for the current frontier. At its center, much of the feature was recognizable product engineering: basic CRUD, filters, pagination, forms, UIs using our design system, etc. The difficult parts were maintaining consistency across layers, preserving compatibility, responding to PM and Design iteration, (safely) handling migration of real customer data, and repeatedly closing the gap between “the code works” and “the product feels right.”

We know that a toy CRUD app proves that an agent can write code, which is not even remotely interesting in mid 2026. A CRUD-shaped expansion inside a Tier 1 service tests whether an agentic system can participate in a real-world engineering team.

What we built in detail

The old Payment Links product model centered on a fixed-amount, one-time-use PaymentLink object that represented the thing a buyer would pay.

The new iteration, Payment Links v2, separated the system into three layers:

1. Product: the reusable catalog and pricing definition, including fixed or flexible pricing (useful for donation use cases).
2. PaymentLinkPage: the buyer-facing page, including reuse (single-use or multi-use, including limited uses) and the ability to collect buyer information, enabling this page to become a linkable or embeddable checkout page.
3. Payment: the individual payment object minted when a buyer uses the page. This object is the same as the minimal PaymentLink we had earlier.

This iteration unlocked catalog products, reusable links, variable amounts, buyer information collection, usage filters, collected totals, and clearer page-level lifecycle controls.

The Products surface lets merchants create and manage reusable catalog items with fixed or flexible pricing.

The Payment Links v2 flow connects a saved product or one-time item to single-use, multi-use, or limited-use rules, optional buyer information, and a live buyer-facing preview.

It also created moderately interesting data migration opportunities: existing links had no backing Product or PaymentLinkPage, and existing Checkout API users (built on the same PaymentLink primitive) could not be silently reclassified as users of the new Coinbase Business UI.

Product direction in less than a week

We wanted to build this without an engineering team, with part-time support from an EM, a PM, and a Designer.

The product-design phase started around a frontend prototype built directly in the real frontend repository by our team's designer. This was deliberately disposable. The point was not to produce a mergeable PR; it was to make the product concrete enough that Design, Product, and Engineering could play with it and stress test the UX.

We used the prototype in multiple sessions and clicked around the new Products and Payment Links tabs, argued about the hierarchy, and discovered where the mental model did not hold. Our PM then synthesized all the discussion into a product requirements document (PRD). That PRD became the basis for a technical design document (TDD), which the engineering team reviewed together.

Within roughly a week, we had moved through:

working prototype → live product/design critique → PRD → TDD

At this stage, our goal was to build a real prototype that aligned the entire team and produce a final technical artifact (the TDD) that could serve as a "master plan" for generating future Linear issues to feed to our software factory.

Setting up the project footprint

We used the TDD to set up the Linear project in stages.

We tried to create about 30 issues at a time, but towards the end, the project contained 185 issues:

• 147 completed;
• 22 canceled;
• 10 in backlog;
• 3 started;
• 2 marked duplicate;
• 1 in triage.

The Linear project became the production queue: issue state controls dispatch, repository labels select the implementation lane, and linked pull requests keep progress visible.

These issues were created using a terminal-based coding agent (Claude Code + Opus), but we asked the agent to ensure that the blocking dependencies across issues were in place.

Blocking relationships keep the queue safe. Symphony watches the same dependency graph and leaves blocked Todo issues idle until their prerequisites finish.

Once a batch of issues was ready, we fed it into our own implementation of Symphony (more on that in the next section). Matching project issue IDs to symphony-labeled pull requests found 146 PRs covering 144 project issues across the three main repositories. Of those, 141 had merged, and 5 were closed without merging.

The merged PRs represented:

• 122,062 additions;
• 30,267 deletions;
• 1,271 changed-file instances;
• 700 commits;
• 1,342 inline review comments.

A note on data migrations

The project contained several smaller data migrations. One example was payment link ownership classification. The existing /payment-links external API was also used by our Checkouts API customers. Before migrating Coinbase Business GUI links, we needed to identify those customers, set new API-created links with a CHECKOUTS_API owner type, and backfill the migratable historical cohort. Only then could we safely normalize legacy GUI records from UNSPECIFIED to PAYMENT_LINKS (the GUI product).

The page migration then used a separate chain:

1. Create one shared helper that maps an existing link to an ad-hoc Product plus PaymentLinkPage.
2. Use that helper for a feature-flagged dual-write on new PAYMENT_LINKS links.
3. Run a one-time backfill for existing eligible links.
4. Reconcile the new Product/Page state with the parent link.

Part 3: Operating Symphony

Symphony is a spec from OpenAI that turns an issue tracker into a long-running orchestration surface. The open-source specification describes a service that polls for eligible work, creates an isolated workspace per issue, launches a coding agent, records its state, and retries or stops according to tracker state.

My own Rust desktop implementation supports Cursor CLI, Codex, Claude Code, and OpenCode. At Coinbase, we run a security-reviewed internal fork.

The overview is the operator's control panel: worker state, recovery queues, recent failures, provider limits, and token usage in one place.

Configure repositories as production lanes

Each implementation repository is registered with:

• a Git URL;
• an install/bootstrap command;
• optional team or project routing rules;
• a repository-specific workflow or the shared default workflow.

Linear labels route work to repositories. For this project, repo:core-service, repo:api-graph, and repo:merchant-web identified the repository. Symphony cloned the selected repository into a per-issue workspace and ran the repository’s setup command before dispatch.

Each production lane has an explicit clone URL, bootstrap command, routing labels, and workflow selection.

Conceptually, the configuration looked like this:

tracker:
  project: Payment Links v2
  assigned_to_me: true
  active_states: [Todo, In Progress, Rework, Merging]
  terminal_states: [Done, Canceled]

repositories:
  - name: core-service
    route_label: repo:core-service
    install: make gen
  - name: api-graph
    route_label: repo:api-graph
    install: make gen
  - name: merchant-web
    route_label: repo:merchant-web
    install: ./scripts/cleanInstall

agent:
  backend: use-local-selection
  model: use-local-default

This is pseudo-configuration; the desktop app stores these settings through its UI:

The settings surface binds a Linear project and its state machine to a set of registered repositories.

The workflow is inspectable and editable rather than hidden inside the orchestrator. Template variables expose tracker and repository context to each run.

Linear as long-term memory across runs

The key idea that Symphony introduces is to use Linear as the durable long-term memory across multiple runs across issues.

The issue state is the control plane:

Linear state	Symphony behavior
Backlog	Do nothing
Todo	Create or recover the workspace, initialize the workpad, move to In Progress, and execute
In Progress	Continue the existing work and address current feedback
In Review	Stop changing code; wait for a human decision
Rework	Close/reset the existing approach and start again from a clean branch
Merging	Run the landing procedure or reconcile an already merged PR
Done/Canceled	Terminal; stop and eventually clean up

Moving an issue to Todo is effectively unlocking work on it. It is a deliberate dispatch action that you take. More commonly, you put multiple issues into Todo — as long as they have blocking dependencies specified within Linear, Symphony respects those.

The agent creates a single persistent ## Symphony Workpad comment on the Linear issue. The workpad records the plan, acceptance criteria, validation, notes, confusions, workspace/commit stamp, and what previous attempts have already accomplished. On retries, the next agent run reads the workpad and repository state instead of reconstructing the task from a transient chat history.

The workpad is the durable handoff between runs: the plan, acceptance criteria, validation, notes, and unresolved confusion remain attached to the Linear issue.

This combination of the Linear issue state and the durable workpad serves as the memory of the system.

The Runs view pairs durable issue history with the complete log of a selected attempt, so an operator can inspect what happened without supervising it live.

Let the local agent configuration choose the model

We intentionally kept Symphony from becoming a model router with a centrally hard-coded “best model.” It launches the selected local agent CLI and, unless overridden, uses the model configured there.

That has several advantages:

• engineers can use the provider and model available in their environment;
• model upgrades do not require changing the orchestration protocol;
• different engineers can experiment without forking the workflow;
• the harness can be evaluated independently from the model.

Agent and runtime settings choose the local CLI while keeping model overrides, permissions, polling, and concurrency explicit.

Humans review outside the loop

When an agent believes the work is ready, it updates the workpad, verifies required checks, attaches evidence, moves the issue to In Review, and stops. By this point, our automated PR reviewers have already reviewed the PRs, and in our experience, we converge to a PR that's ready for a human review after a few rounds. Most importantly, the Symphony workflow requires the agent to add its proof of testing to make the PR reviewable at a glance. For frontend PRs, it means that screenshotting its work is mandatory.

The human "author" then reviews the PR manually. If there are changes:

1. Leave normal PR comments and move the issue back to In Progress.
2. Symphony resumes work on the issue and the agent sweeps top-level comments, inline threads, and automated review feedback.
3. It changes code or replies with a justified disagreement, re-runs validation, and returns the issue to In Review.

Once the human author is satisfied, another team member performs the final human review.

Review loops humans would not patiently perform

The system ran several independent forms of review: repository tests and CI, automated inline review, an adversarial “Brutally Honest Review,” feedback sweeps, and human-gated review.

The most interesting examples were not the one-shot successes, but the issues that required repeated iteration:

• One backend foundation PR introduced the PaymentLinkPage entity and merchant CRUD in the core service. Its PR accumulated 88 inline review comments, 14 commits, and 18 Symphony runs. One sequence of adversarial reviews found a wire-compatibility gap. The fix exposed a status-transition compare-and-swap race. Fixing that exposed a not-found-versus-mismatch ambiguity. The loop continued until the review was green.
• One frontend PR, which wired the Products list page to GraphQL, required 22 recorded Symphony runs. Its workpad reached “retry #21.” Across those runs, the agent addressed accessibility and routing feedback, stopped fabricating values for malformed data, corrected amount formatting, retried screenshot capture, handled a wrong session cookie, rejected screenshots that had redirected to the wrong page, embedded the eventual proof, and cleaned the screenshot commit out of the final branch. The PR accumulated 40 inline review comments and 10 commits.

It is tempting to call these superhuman persistence. Two humans are unlikely to exchange twelve rounds of reviews, but the agents will keep looping — you just have to ensure that your automated reviewer is fine-tuned enough to eventually converge.

The operating lesson for us was to continuously measure and improve the loop:

• distinguish implementation, review, environment recovery, and no-op attempts;
• cap retries and back off transient failures;
• short-circuit completed work;
• preserve a durable record so a retry does not start over;
• escalate true ambiguity instead of celebrating infinite patience;
• periodically update the workflow or skill that caused repeated confusion.

Some stats on this project: the local Symphony database recorded 759 runs across 127 project issues, totaling 129.4 hours of aggregate agent runtime. The median issue had four runs. Sixty-one issues had at least five runs, and nineteen had at least ten. Peak active-run concurrency was three, while the project later reached eleven simultaneously open PRs.

That is the shape of the leverage: a small bounded execution pool can maintain a much wider pipeline of work in implementation, CI, review, and product feedback.

Part 4: The economics: intelligence is becoming too cheap to meter

The early project work used Claude Opus 4.8 through Claude Code. Our preliminary blended spend was roughly $100 per merged PR, which seemed on the expensive side to be practical.

Later we moved much of the workflow to Cursor Grok 4.5 High. The preliminary observed cost was under $5 per merged PR. We are also experimenting with Luna Max after the price drop and other open-weight models once they're available.

When capable model inference becomes inexpensive, the economically possible workflow changes. We can afford to spend tokens on:

• adversarial review loops until convergence;
• generating screenshots and proof;
• analyzing failed trajectories;
• improving the harness itself.

“Intelligence too cheap to meter” is an exciting world to be moving toward.

Our biggest takeaway from running this over the past two months: the cost curve is falling fast enough that orchestration and evaluation quality matter more than token thrift. Cheap intelligence reminds us that the bottleneck is moving toward specifications, evaluators, integration, taste, and accountability.

5. The first rung of recursive self-improvement

“Harness Engineering for Self-Improvement” provides a solid framing for how this sort of harness engineering is crucial for recursive self-improvement. The central point is that near-term recursive self-improvement is unlikely to begin with a model directly rewriting its own weights. A more practical path is improving the machinery around the model: context, workflows, permissions, memory, tools, evaluation, and eventually the optimizer that changes those things.

The author describes the progression of the optimized object as:

instruction prompts → structured context → workflow → harness code → optimizer code

This maps directly to what we experienced. The first version of the workflow encoded what we knew. Real runs exposed missing tool instructions, confusing screenshot procedures, unsafe cleanup patterns, stale context, review-loop gaps, and conditions where agents stopped to ask humans for help. Those failures turned out to be training data for the harness.

The Retro tab

We added a Retro tab to the Rust implementation as a first practical layer of self-improvement.

The Retro turns execution traces into reviewable workflow proposals, each with an exact diff and an explicit accept or reject decision.

When you "Generate a retro", it:

1. Collects terminal runs and agent events from local SQLite.
2. Fetches the durable Symphony workpads from Linear.
3. Groups repeated failures, retries, approvals, tool confusion, and explicit ### Confusions entries by repository.
4. Maps findings to either the common workflow or a named repository skill.
5. Materializes an exact proposed diff.
6. Requires a human to accept or reject every proposal.
7. Applies accepted default-workflow changes locally or opens one implementation PR per repository for workflow and skill changes.

The core sequence is visible in src-tauri/src/retro.rs:

let runs = repo.list_retro_runs(&retro.since_at, &retro.until_at).await?;
let events = repo.events_for_run_ids(&run_ids).await?;
let workpads = tracker.fetch_workpads(&issue_ids).await?;

let report = analyze_retro(
    &retro,
    &runs,
    &events,
    &workpad_by_issue,
    &found_workpad_issue_ids,
);

let suggestions = materialize_suggestions(&retro, &report, &proposal_config).await;

The Retro UI makes the proposals reviewable: pending, accepted, rejected, and unavailable suggestions; exact before/after diffs; workflow application; and per-repository implementation PRs.

This is intentionally conservative. A run that needed approval becomes evidence that an unattended workflow or skill is incomplete. Repeated screenshot failures should improve the screenshot skill. Repeated review confusion should improve the feedback skill. A prompt change is versioned and diffed instead of silently appended to an ever-growing instruction blob.

What we have today is an observability-driven improvement loop with deterministic pattern extraction and a human approval gate. It proposes edits to the harness, but it does not yet prove that those edits improve future performance.

The next layer needs an evaluation protocol:

1. Baseline: record success, retries, runtime, review findings, rework, and human touches for a pre-change cohort.
2. Hypothesis: tie every proposed workflow/skill edit to a specific repeated failure mode.
3. Canary: apply the change to a bounded set of future issues or replay a held-out trajectory set.
4. External evaluation: keep tests, trace audits, and final human review outside the loop being optimized.
5. Promotion or rollback: retain changes only when the target failure improves without degrading quality elsewhere.
6. Memory hygiene: deduplicate or retire obsolete guidance rather than letting the workflow grow monotonically.
7. Negative results: preserve proposals that did not help so the next retro does not rediscover them.

That creates a ladder:

Layer	What improves	Status
0	Humans manually edit prompts and procedures	Established
1	Run/workpad evidence generates reviewable workflow and skill diffs	Retro today
2	Accepted changes are tested on canaries and measured against a baseline	Next
3	An optimizer searches multiple workflow/skill candidates under a held-out evaluator	Future
4	Harness and model-training systems co-improve with external safety and quality gates	Research

The software factory can observe its own production failures, propose a process change, route that change through review, and measure whether the next production run gets better.

Conclusion: move humans up the stack

This project demonstrated that as the execution becomes cheaper, the quality of engineering direction becomes more valuable.

On this project, humans supplied the product and design insights, turned a prototype into reference docs, chose the right architecture, designed the migrations, reviewed evidence, and decided when the PRs were acceptable.

Symphony supplied persistence. It kept the queue moving, preserved context, created isolated workspaces, executed changes, responded to comments, re-ran tests, retried failures, collected evidence, and garbage-collected workspaces, all without demanding that a human supervise every turn.

That is the shift from using an agent to operating an agentic production system, a "software factory" if we may.

The 2025 question was: How far can my agent get in just one turn?

The 2026 question is: How do I design a system that can repeatedly produce verified code, while humans spend their time on higher-leverage work?

We are still early. The loops are imperfect, the cost comparisons are entirely non-scientific and vibed, and recursive improvement needs real evaluators. But it is difficult not to be optimistic. When intelligence becomes cheap enough to spend freely on, one engineer can operate at a very different level of abstraction.

Sources

• OpenAI: Harness engineering—leveraging Codex in an agent-first world
• OpenAI: An open-source spec for orchestration—Symphony
• OpenAI Symphony repository and specification
• Lilian Weng: Harness Engineering for Self-Improvement