Skip to content

Pull request unit test selector

TL;DR

Lint still discovers tests/ on pushes to main. A pull request can now run an explicit module list when every changed path is in the fail-closed map. The same pull-request file list can fast-success Komodo resources, Ansible lint, and generator drift when none of their inputs changed. This is source-only: nothing was deployed, and ci-gate still requires every needed job to finish as success.

What changed

python-unit-tests keeps the short dependency-contract step, then asks scripts/ci/pr_unit_test_selector.py for a decision. The script uses the paginated pull-request files API. Unmapped paths, force-full policy files, selector or map edits, removed or renamed tests, API errors, short pages, and crashes all print full and the job runs unittest discover -s tests. A crash is written to the step summary as full discovery, not as a narrow success.

Docs are one mapped case. A docs path with no other reader runs tests.test_docs and tests.test_documentation_remediation. Paths that other tests actually read add those modules. services/monitoring-canary/** and a handful of single-reader scripts are the other mapped prefixes. services/litellm/** and the rest of the service, Ansible, and inventory trees stay full. The rule list is Pull request CI scope.

komodo-resources-check, ansible-lint, and generators-check call scripts/ci/pr_job_scope.py on pull requests. Stdout must be fast or run. A fast result exits 0 with a step summary and does not run that job's checks. Pushes to main, an incomplete file list, a crash, and any unclassified path run the real checks. A crash is not described as a fast success. The jobs are not skipped, so ci-gate still sees success when the fast path is valid.

flowchart TD
  job[Komodo, Ansible lint, or generators] --> kind{pull_request?}
  kind -->|push to main| real[real checks]
  kind -->|yes| files[pulls files API]
  files -->|incomplete, crash, or relevant path| real
  files -->|every path outside the job| fast[exit 0 step summary]
flowchart TD
  contracts[contract tests always run] --> kind{pull_request?}
  kind -->|push to main| discover[full unittest discover]
  kind -->|yes| api[files API]
  api -->|fail closed| discover
  api -->|every path mapped| union[union of module lists]

Phase status

Area State
Push to main Full unittest discover -s tests
Mapped pull requests Explicit module list plus the contract step
Komodo, Ansible lint, generators Fast-success only when every changed path is outside that job
Other Lint jobs and ci-gate Still required; a skipped job still fails the gate
Runner labels and queue Untouched
Deploy workflows Untouched

Known gaps

No wall-clock saving was measured here. The unit-test selector only changes which modules python-unit-tests collects. Komodo resources, Ansible lint, and generator drift skip their real checks only for the diff shapes in the runbook. The OneUptime adapter, the Go collector, Paperless, Phoenix, and the runner observability jobs still do their existing work. Runner queue time is outside this change.

Most service diffs stay on full discovery because the tests that read them also walk backup manifests, Komodo resources, or several services at once. A wrong narrow list would be worse than a slow full run, so those prefixes were left unmapped.

What remains

  1. Add unit-test map entries only after a reader scan shows the module list is closed. Phoenix routing stays outside the gate.
  2. Do not fast-success the OneUptime adapter, Go collector, Paperless bridge, or security jobs until their inputs are enumerated the same way. Do not change runner routing to chase queue time.