Coding assistant on your own repo

A small open-source library has more open issues than its two maintainers have hours. Most are small: a failing edge case, a typo in an error message, a function that needs a test it never got. The job is an assistant that reads an issue, edits the repository, runs the tests and stops with a branch ready for review, not one that decides what to merge.

Example run

"parse_duration() raises on '1h30m'; add a test
and fix it."

Walkthrough

The agent reads the issue, and for a fix touching this repository’s own conventions (adding a test, say) calls load_skill for the relevant one rather than working from memory of how the last repository it saw was organized. It proposes an edit, code applies it and runs the existing tests, and the agent reads the result: if a test still fails, it tries again; once everything passes, it asks to open a branch.

That request is where the scope check runs. The model is not judging its own change: code compares the actual diff, and any commands the agent asked to run, against what this task was scoped to touch. A request inside the issue’s own files, running only the test suite, becomes a branch. One that reaches outside (a workflow file, an unrelated dependency pin, a shell command off the allowed list) is refused, with the reason handed back so the agent can try again.

Everything past the branch belongs to a person. Nothing merges its own pull request, and nothing the agent proposes reaches the default branch or a push credential until a maintainer reads the diff. The tests it ran are the ones already in the repository, not new coverage it invented.

Running this on every incoming issue takes a few more decisions. Nothing persists that the agent controls: the maintainers’ skills, the branches it opened, a log per issue. Every run starts from the repository as committed, so a bad one leaves a branch nobody has to merge. That log is what a maintainer reads the next morning: the issue, every edit attempted, every test result, every refusal the check made, since a refusal is the most interesting line in it and the easiest to lose. Stopping it means stopping the trigger: stop handing it issues and nothing new starts. The issue text, the files it opens and the test output go to whichever model runs it, unremarkable for a public library and a longer conversation for a private one. Cost is one agent loop per issue, and what goes wrong is rarely a wrong fix; it is a scope-creeping one the check let through.

What to measure

Coding agents don’t answer the site’s own document-QA questions, so this recipe needs its own small test set: synthetic issues against a synthetic library, each with a known-good fix and its test file, written to look like the issues this repository gets. Measure the share that reach a passing suite and the attempts each took. Then run the finished fix against the rest of the suite, not just the file the agent touched, to see whether an accepted change broke behavior the shown tests never exercised. Measure the scope check the way any permission check should be: try to get the agent to request something out of scope on purpose, in ways its author did not think of, and confirm every attempt is refused. None of this has been run here.

Variations

• Add human approval as a pause before the branch is opened at all, for a repository where even a scoped, tested change should not reach a person’s queue unannounced.
• Run several issues at once as parallel calls, one independent loop each, before reaching for anything that coordinates them.
• Let the skill set grow with the repository, the way a coding agent’s instruction files are meant to, instead of fixing it once.
• Widen the scope check from paths and commands to the size of the diff, refusing a change large enough that no fixed suite is good evidence it did only what the issue asked.

Design choices