If you’re building AI agents, you’ve probably seen the same problem over and over. The agent tries one plan, writes some code, tests it, and then either gets stuck or throws everything away.

That’s exactly why “Harness-of-Harness” (HoH) keeps showing up in new research and demos. It uses a repeated plan-coding-testing loop and keeps the project state across runs. That means the agent can improve step by step, instead of resetting every time.

In this guide, I’ll explain what Harness-of-Harness is, why it works, and how you can copy the same idea in your own agent setups. We’ll also cover what to measure, how to avoid common failure modes, and a few practical patterns you can use today.

This is about real agent loops and real outcomes, not vague “multistep reasoning” claims. If you want agents that can build something you can actually run, Harness-of-Harness is one of the clearest recent signals on how to do it right.

What “Harness-of-Harness” (HoH) means in plain English

Most agent setups fall into one of these buckets:

  • A single run agent: it gets a goal, makes a plan, writes code, and stops.
  • A “tool use” agent: it can call tools, but the outer approach is still mostly one pass.
  • A multi-agent swarm: lots of voices, but not always a clear loop that keeps progress.

Harness-of-Harness (HoH) is different because it is built around an outer harness that keeps the whole project alive.

The core idea is simple:

  1. Plan how to do the task.
  2. Code the plan.
  3. Test the changes.
  4. Keep the project state and run the next iteration.
  5. Repeat many times until the task is done.

The “of Harness” part matters too. HoH is not just “an agent.” It is a test-driven environment where the harness runs cycles and preserves what happened before. That changes the agent’s behavior. The agent starts treating earlier work as a foundation, not as something temporary.

A well-known example from the search results says HoH used an iterative planning-coding-testing loop and demonstrated a 70-iteration case study where it built a playable first-person shooter game from scratch. You can read more in these sources:

Also, the search results mention benchmarks on FrontierSWE and ProgramBench with an average 52.25% gain in task success versus standard agent harnesses. Source shown in search results:

That’s the key. HoH isn’t just “more steps.” It is structured steps with state and feedback.

Why iteration works when agents usually fail

Let’s be honest. A lot of agent failures look like this:

  • The model writes code that “looks right.”
  • It only fails when compiled or run.
  • The agent gets new info from the error.
  • Then the agent either forgets earlier progress or restarts from an earlier draft.

So you end up paying the cost of a restart many times, and the agent never builds momentum.

Harness-of-Harness (HoH) helps because it creates momentum.

1) State preservation stops the “reset spiral”

In a state-preserving loop, the agent keeps the same repo. It can:

  • reuse existing files
  • keep previous fixes
  • refine functions instead of rewriting everything

That reduces churn. It also makes testing feedback more useful because errors refer to real code that already exists in that same workspace.

2) Feedback is forced into the loop

HoH uses testing as a required step. That means the model cannot skip the hard part.

If you just ask an agent to “write the app,” you get lots of output and little learning.

If you force a test between iterations, the agent gets a clear scoreboard:

  • did it pass
  • what failed
  • what changed

It is harder to “sound confident” when the code has to compile.

3) The loop lets the agent recover from wrong turns

Even good models make bad calls. The problem is when the setup has no route to recovery.

With Harness-of-Harness (HoH), a wrong turn is not fatal. It becomes a test failure. Then the next iteration can fix the specific problems.

This is one reason HoH is linked to better success rates on structured coding benchmarks, like the ones cited in the search results.

The HoH loop you can implement yourself

You don’t need the full HoH stack to copy the key pattern. You can build a “HoH-like” loop with your own agent orchestration.

Here’s a practical version you can implement.

Step 1: Create a workspace that persists

You need:

  • a repo directory that stays the same across iterations
  • a place to save logs and test outputs
  • a way to keep diffs, or at least keep the full working tree

In other words, your agent should not run in a clean folder every time.

Step 2: Add a planning pass that can be revised

On each iteration, the agent should produce:

  • what it plans to change this time
  • why that should fix the failing tests
  • what files it expects to touch

Important detail: plan output should be tied to test feedback from the last iteration.

Step 3: Implement and run tests every time

After code changes:

  • run your unit tests
  • run integration tests if you have them
  • compile or run a basic smoke test

Capture output.

If tests pass, you stop.
If tests fail, you provide the failure output back into the next iteration.

Step 4: Keep memory of failures

You need a “failure log” that the agent can read during the next loop.

This can be as simple as:

  • store the last 5 failures
  • store the error messages
  • store the relevant stack traces
  • store which files changed last time

This is how state becomes useful, not just preserved.

Step 5: Run a budget and stop condition

HoH shows 70 iterations in a case study. That does not mean you should always do 70.

Use:

  • a max iteration limit
  • a “no progress” limit (example: tests fail in the same file for 3 loops)
  • a time budget

This keeps costs and time controlled.

A mini blueprint for building a “HoH-style” coding agent

Let’s make it concrete. Imagine you want an agent that builds a small web app.

Your loop might look like this:

  1. Goal: “Build a small todo app with login and a dashboard.”
  2. Iteration 1:
    • plan entities and routes
    • write files
    • run tests and basic login checks
  3. Iteration 2:
    • read test output
    • fix auth bug
    • update types or schema
    • rerun tests
  4. Iteration 3:
    • add missing UI component
    • run lint + unit tests
  5. Continue until tests are green

This is not “magic.” It’s just making sure the agent can correct itself with each cycle.

And if you want to borrow tooling ideas, you can also look at how research setups keep harness state and repeatedly test. The sources in search results point you toward the original work:

What to measure (so you know it’s working)

Benchmarks like FrontierSWE and ProgramBench are great, but you also need internal metrics for your own build.

Here are practical measures that match the HoH philosophy:

1) Success rate per task

Track “done” vs “not done” for each task type.

Because stateful loops should increase “done” at the same attempt budget.

2) Tests pass rate over time

Plot:

  • iteration number on x-axis
  • test pass count on y-axis

This shows whether the loop is improving or stuck.

3) Change size and churn

If the agent makes huge rewrites each iteration, it may not be learning.

State-preserving loops should tend to reduce churn over time.

4) Failure locality

Look at which files keep failing.

If the same module fails repeatedly, you likely need:

  • better tests
  • a clearer spec
  • or improved tool constraints

Common failure modes in iteration loops (and how to fix them)

Even with Harness-of-Harness, you can still get stuck. Here are common ways it breaks and what to do.

Failure mode 1: The agent keeps changing the wrong thing

Symptom:

  • tests fail in the same area repeatedly
  • diffs are unrelated to the failure output

Fix:

  • force the plan to reference the exact failing test names
  • require the agent to quote the failing error lines in the plan

Failure mode 2: Tests are too weak

Symptom:

  • the agent “passes” tests but the app is broken in real usage
  • or the agent keeps reaching a fake success

Fix:

  • add smoke tests that run the app end-to-end
  • add at least one “real action” test (login, create item, query dashboard)

Failure mode 3: State grows messy

Symptom:

  • after many iterations, the repo becomes a patchwork
  • performance and behavior degrade

Fix:

  • run a cleanup step every N iterations
  • or implement “rebuild from last stable commit” after large churn

Failure mode 4: Cost rises too fast

Symptom:

  • every iteration reruns heavy tests and consumes time

Fix:

  • split tests into:
    • fast unit tests every loop
    • heavier tests every 3 loops
  • cache dependencies where possible

How to “route” better agent requests using intent

One more practical idea ties the HoH loop to modern agent routing.

In real apps, your agent may get different types of requests:

  • “Fix this bug I found”
  • “Add a feature”
  • “Refactor for performance”
  • “Explain why tests fail”

If the system can detect intent, you can route to different workflows.

That’s where router-style agent thinking helps. Neura AI’s Router Agents are designed to route requests based on user intent, using RAG plus reasoning and action. If you’re building agent systems that need the right loop style for each intent, it’s worth checking how Neura approaches routing in its product suite:

Even if you do not use Neura, the principle is the same: choose the loop method based on the type of task.

Where HoH fits in the real world of app building

So what does Harness-of-Harness (HoH) change for teams?

Here’s the biggest shift: you stop asking the agent to be perfect the first time.

Article supporting image

Instead, you build a system where:

  • the first output is a draft
  • the next iteration improves it
  • tests guide every step

This matches how humans work. We write, we run, we fix. We rarely do everything in one shot.

The case study showing a playable first-person shooter from scratch after many iterations might sound extreme. But the same idea scales down to:

  • small internal tools
  • bug fix cycles
  • feature additions that need repeated edits and verification
  • migrations where tests are clearly defined

It also gives teams something concrete to use in review. Each iteration has a trace:

  • plan
  • code changes
  • test output
  • updated behavior

That’s much safer than a single opaque “here’s the final code.”

Try it today: a simple checklist for your next agent build

If you want to get 80% of the HoH benefit quickly, use this checklist.

  • Do you keep the same workspace across iterations?
  • Do you run tests every loop, not just at the end?
  • Can the agent show how it used the last failure output?
  • Is there a clear stop condition?
  • Do you log failures and changes so the agent can refer back?
  • Are your tests fast enough to support many iterations?

If you can answer “yes” to these, you’re already much closer to Harness-of-Harness style behavior.

And if you’re starting from scratch, you can also read the primary research source cited by the search results:

Conclusion: Harness-of-Harness style loops are how you make agents dependable

Harness-of-Harness (HoH) is trending because it tackles a practical problem: agents need a way to improve without resetting and without skipping tests.

The big takeaway is this.

State-preserving iteration turns agent output into a real build process.

Instead of one attempt that may fail silently, HoH creates a loop where each cycle is guided by test feedback. That’s why it reports strong gains on coding benchmarks and demonstrates long-horizon project builds.

If you’re working on agent-based coding, your next step is not picking a new model. It’s building a better loop.

Give your agent a workspace that persists, force tests into the loop, and make failure output part of the next plan. That is the HoH lesson in the simplest form.

Neura AI related resources you may find useful

If you’re exploring how to make agent workflows more organized and intent-aware, you can also browse Neura AI’s platform and app set here:

You can also check Neura ACE as a content and research style agent example (useful patterns for structured outputs and iterative improvements):

####################################################################################################