Docs

QA test

The qa-test workflow drives a flow. Given a CLI command or a locally-served app and a sequence of steps, it installs and runs the code in the sandbox, exercises each step, and reports step-level pass/fail with evidence. It never modifies code.

Permission profile: issues-write — can read repo contents and post a QA report comment; it never pushes code.

Pipeline

QA Test run flow + report steps

What it does

  • Defines the test steps (from the request, or inferred from a PR's diff) and their success criteria
  • Installs and runs the code, then exercises each step — a CLI directly, or a web service via its dev-server and curl
  • Records each step PASS / FAIL / BLOCKED with evidence (command, stdout/stderr, exit code)
  • Continues past a failure for maximum coverage — unless a step blocks everything downstream
  • Posts a structured report: a results table, issues found, and what wasn't covered
Text evidence by default, screenshots when available. The text phase uses bash, file read, and the GitHub tools — no browser. On a Docker host with the browser-QA image built, a second gated phase drives the locally-served app in a real headless Chromium and posts step-level screenshot evidence on top of the text report; everywhere else that phase is silently skipped and a rendered-UI step is marked BLOCKED.

Triggers

  • GitHub: a maintainer comment @last-light qa-test <target> [-- steps] on an issue or PR, or a natural-language request the classifier reads as a qa-test intent ("run through the signup flow and tell me what breaks")
  • Slack: a message classified as a qa-test intent against a managed repo
  • CLI: lastlight qa-test owner/repo#N -- "<steps>"

Skills

This workflow uses the qa-test skill (the flow procedure and report shape) plus the building skill (install + run the repo). The gated browser phase additionally uses the browser-qa skill (a bundled Playwright driver) to drive the UI and capture screenshot evidence.