code-evolve: PI agent swarms

Think agent swarms with guardrails, so no giant bills or unexpected behavior.

Here is the high-level loop, in five steps:

1. Start from a seed program and its score.
2. Pick a promising parent.
3. An agent edits a copy of it in its own worktree.
4. The evaluator scores the new candidate.
5. Keep the good ones and repeat.

It has explicit exploration and exploitation steps where more capable agents plan new approaches or tweak existing ones. If the resulting program gets a better score then it joins the island’s population and becomes more likely to be picked as a parent for the next round, so good ideas get built on. Weak programs drop out when the population is full.

An “island” is just an independent swarm of agents. Periodically (every few rounds, on a schedule you set) the best programs are shared between islands. Each island copies in the other islands’ top working programs, so a good idea found on one island can spread without every island collapsing onto the same solution.

The secret sauce is the score. The task needs to be measurable with a numerical value (an “eval”) so that we know when it is hill climbing. It does not need to be deterministic but the score does need to be consistent enough across runs.

The swarms do not see the tests behind the score. The agents only see the task, the score, and how the score is computed. Otherwise an agent will happily cheat (game the tests) instead of making the program better.

Running it

It runs inside Pi. The plugin lives at gitlab.com/robotandkid/code-evolve-pi-plugin. Set the models, number of islands, budget and limits once:

/evolve-settings

Then point it at a repo, its hidden tests, and a task:

/evolve start ~/code/count-primes ~/code/count-primes/tests \
  "Make countPrimes(n) as fast as possible while keeping it exactly correct."

The tests folder holds the evaluator, a script that runs the candidate and prints one line of JSON:

{"score": 12.4, "metrics": {"passed": 12, "total": 18}}

Watch it with /evolve-dashboard, and stop or resume with /evolve stop <run-id> and /evolve resume <run-id>.

Results

I’ve used it to optimize an LLM pipeline. It found fewer, more reliable LLM calls with smaller outputs, so the total cost went down 90% (ten times cheaper) while keeping accuracy. Total time stayed the same.