Docs

PR fix

PR fix is a lightweight two-phase workflow for quick fixes on existing pull requests. When a maintainer comments on a PR asking Last Light to fix something, and the classifier detects a BUILD intent, this workflow runs instead of the full build cycle — it works out why the PR is red, then fixes it and pushes directly to the PR branch.

The two phases exist because retrying is only useful if you know why something failed. A blind second attempt on a PR whose tests need a database, or whose base branch is already red, spends a full install and test cycle to learn nothing. So diagnose classifies first, cheaply, against the already-cloned workspace — and fix runs only if the verdict says a repair is worth attempting.

Permission profile: repo-write — can push commits to the PR branch. No architect planning or independent review.

Pipeline

Diagnose classify the failure Fix if fixable: fix + push

How it differs from build

Build PR fix
Phases 6 (guardrails → architect → executor → reviewer → PR) 2 (diagnose → fix)
Approval gates Optional (post_architect, post_reviewer) None
Code review Independent reviewer with fix loop None — relies on the existing PR review process
Creates branch Yes (new branch per issue) No (pushes to existing PR branch)
Typical use New features from issues Fixing failing CI, addressing review comments

What it does

  • Clones the repo at the PR's head branch
  • Diagnose — reads the PR description, comment thread and CI failures, reads the repo's own .github/workflows/*.yml, names the differences between CI and the sandbox (toolchain version, OS, absent services or secrets), reproduces the exact failing command, and classifies the failure. It changes nothing: no edits, no commits, no pushes, no labels
  • Fix — implements the smallest change that lands the PR, guided by the diagnosis
  • Runs a local gate it wrote for itself — a targeted reproduction of the failure it diagnosed (one test file, one lint rule, one build target), not a copy of the repo's CI pipeline. CI runs on the pushed commit and is the authority on green; the gate only answers "is the thing I diagnosed fixed?"
  • Pushes the fix commit to the PR branch — only on a green local gate. If the gate is still red when it runs out of iterations it says so rather than pushing a speculative fix, which would cost a full CI cycle to prove nothing

The five failure classes

The diagnosis lands in exactly one class, and the class decides what happens next. Both PR fix and Dependabot CI fix share the same vocabulary — it lives in the fixing skill.

Class Meaning What happens next
reproducible The same command fails in the sandbox too Fix it
env-mismatch Passes locally, fails in CI on a version / OS / flag difference Align to CI and re-verify — often a config change, not a code one
flaky A timeout or network blip, or the same job passed on an earlier commit Change nothing — but only twice in a row; the third is taken at face value and fixed
infra-dependent The check needs secrets, a live service, a deployed backend or a browser Can't be fixed here — escalate, naming the checks
upstream-broken The base branch is red too Not this PR's fault — it self-heals when the base goes green
The last three are stopping verdicts: the fix phase is skipped and the run finishes successfully. Correctly working out that a PR can't be fixed from here is the job, so it isn't recorded as a failure — a red run would comment "couldn't auto-fix, leaving it for a human" on a flaky test, offer a Retry that cannot succeed, and skew the cost and failure stats.

What each attempt remembers

Attempts are separate runs, days apart — the next one starts when CI goes red again. Nothing carries over on its own: the phase outputs are gone and the workspace is reset between runs. So each phase signs off with a single machine-readable line, and the harness keeps one bounded line per attempt: 

attempt 1: class=reproducible cause=lockfile stale vs package.json | outcome=pushed gate=green
attempt 2: class=env-mismatch cause=CI runs node 22, sandbox node 20 | outcome=gave-up gate=red

Attempt 3 therefore knows what was tried and what was ruled out, without replaying two agent sessions. The prompt is told which attempt it is on and of how many, so it can choose between spending and stopping.

Later attempts can also run on a stronger model. Set a pr-fix-retry entry in models and every attempt above fix.escalateModelAfterAttempt uses it instead of pr-fix — the cheap model gets the first shot, the expensive one gets the problem it couldn't solve. That count is the number of attempts on record, not the budget counter, so asking for a retry on a PR that has already failed three times keeps the stronger model rather than dropping back to the cheap one. The model each attempt actually used is recorded on the run, so it is visible in the dashboard afterwards. Leave the entry unset and nothing changes.

The same lines drive the budget. An attempt is only counted when the diagnosis actually landed — a run that crashed on a sandbox failure, a quota rejection or a model API error costs nothing, because otherwise one bad hour would escalate every open PR across every repo to requires-human and a human would have to un-stick each one by hand. flaky and upstream-broken cost nothing either — they are verdicts about something other than this PR's code. Repeated flaky verdicts are bounded separately by fix.maxFlakyDeferrals: three in a row means the job is not flaky, it is intermittently really failing, and it gets a real attempt. And if anyone else pushes to the PR, the counter and the history reset — the world moved, so it is a fresh problem.

Notes the agent leaves for itself

The marker line has fixed fields. Anything the agent works out that doesn't fit them would be lost — so it can also write a short note to the pull request's journal, one line at a time: 

ruled-out: regenerating the lockfile changes nothing — the failure is in the source
constraint: the e2e job needs a postgres service; it can never run in this sandbox

Four kinds — finding, constraint, ruled-out, todo. ruled-out is the one that earns its keep: it records a negative result the agent actually checked, which is exactly what stops the next attempt spending itself repeating it. The journal is keyed on the pull request, not on the workflow, so a review of the PR reads what the fix runs learned.

The cap is the feature: twenty notes per PR (oldest dropped), 240 characters each, 4 KiB rendered. Anything replayed into every later prompt has to be small, or attempt 3 pays for attempt 1's verbosity. Notes carry their provenance — which run, which phase, which day — and once someone else pushes they are marked stale rather than deleted: a claim about the old commit isn't evidence about the new one, but it's still worth seeing.

Notes are hints, never instructions. A fix agent reads PR content written by whoever opened it, so anything it remembers could carry that content into a later run's prompt. The journal is therefore fenced and stripped: each note is one line and cannot forge the fence it renders inside, a note containing a token the harness parses is discarded outright, and nothing in the fix loop can be reached because a note said so — no note substitutes for the local gate, and no note can cause a push.

When it gives up, it says so

When the budget really is spent — every attempt used, the fix.maxCostUsd ceiling reached, or a diagnosis of something no further attempt can fix (needs secrets, a live service, a browser) — Last Light stops and tells you. It applies requires-human and posts one comment naming which of those it was, how many attempts were spent, and what each of them found. A PR quietly abandoned with nothing on it is worse than a labelled one: nobody can tell the difference between "still working on it" and "gave up three days ago".

The label is a notification, and nothing reads it. What holds the PR back is the recorded fact that one of our runs escalated it at a particular head commit — and there are four ways to lift that, all of which do exactly the same thing: re-arm the attempt counter and the cost budget for a fresh full window. Push a commit; comment @last-light retry [reason]; remove the requires-human label; or run lastlight pr retry owner/repo#42, the only surface with no GitHub event behind it and therefore the only one that also dispatches straight away. A retry keeps what earlier attempts learned, where a push wipes it — a push changed the code and a retry did not. To stop Last Light touching a PR at all, apply the hold label (lastlight-ignore), which beats all four. How each is authorised, and what a retry does not override, is on the PR state model page.

Two stopping cases deliberately stay quiet. A failing base branch is not this PR's problem and heals itself the moment the base goes green, and a fork PR gets its own explanation (there is no branch we can push to) — labelling either would poison requires-human with conditions nobody needs to act on.

Triggers

  • GitHub comment: @last-light fix this on a PR (classifier detects BUILD intent on a PR comment)
  • GitHub webhook: a check_suite.completed event that settles red on a PR whose head commit Last Light pushed (surfaced internally as pr.checks_failed). This is the feedback loop the fix phase never had — it could push a fix and never learn whether the build went green, because that event only ever fired for dependency PRs. Routing is deterministic: a dependency-update PR goes to Dependabot CI fix, anything else comes here. It cannot fire on a PR Last Light has not touched. Requires the GitHub App to subscribe to Check suite events (Checks: read).

When it won't run

Before any sandbox is provisioned, a single resolved view of the pull request answers whether a fix run is worth spending at all. It is refused when the PR comes from a fork (there is no branch we can push to — the author gets a comment saying so), when the base branch is itself red (a fix here cannot make CI green, and the PR becomes eligible again the moment the base recovers — nothing to un-stick by hand), when the attempt or cost budget for that PR is exhausted, when the exact head commit has already been assessed, and when another run is already working the PR. That last one is a real lock: only one run may be in flight for a given PR across every workflow that declares itself PR-scoped — packaged, that is PR fix, Dependabot CI fix, Dependabot PR merge and PR review — so two agents can never clone and push the same branch at once.

An explicit @last-light request from a maintainer overrides the escalation guard and the already-assessed check — asking directly is an intentional override. It does not override the facts: a fork PR, a red base branch and an exhausted budget stay refused however the run was asked for. The full ordering of those guards, and what each refusal leaves on the PR, is on the PR state model page.