Skip to content

CI hosted-runner audit (2026-08-09)

Trigger: Owner directive that no repo in the spadoople org should run GitHub Actions jobs on GitHub-hosted runners (ubuntu-latest, windows-latest, macos-latest). spadoople/homelab is the reference implementation for the correct pattern; this audit inventories every other repo in the org against it as a precursor to a migration effort.

Method: Enumerated all 21 repositories in the spadoople org and read every workflow file under .github/workflows/ in each (directory listing + raw content fetch, not code search — GitHub code search does not reliably index every file/repo in this org and undercounted matches when tried).

Reference pattern (from homelab)

homelab's workflows never hardcode a GitHub-hosted runner label. Instead:

  • Job-level runs-on is one of four repo/org variables, evaluated through fromJSON:
  • runs-on: ${{ fromJSON(vars.CI_RUNS_ON) }} — general-purpose jobs
  • vars.DOCKER_RUNS_ON — jobs needing Docker/service containers
  • vars.DEPLOY_RUNS_ON — deploy jobs
  • vars.RUNNER_FLEET_MAINTENANCE_RUNS_ON — fleet-maintenance workflows
  • Each variable holds a JSON array of on-prem runner labels, so the actual label set can change per-repo/per-environment without editing workflow YAML.
  • Shared logic lives in reusable workflows in spadoople/ci-toolkit (secret-scan.yml, semgrep.yml, workflow-lint.yml, runner-health.yml, stale-run-watchdog.yml, library-ci.yml), invoked with uses: spadoople/ci-toolkit/.github/workflows/X.yml@<sha> and a runs_on_json input threading the caller's runner labels through.
  • An explicit enforcement job (no-hosted-runners, running scripts/check-no-hosted-runners.sh) asserts zero occurrences of ubuntu-latest/windows-latest/macos-latest anywhere in the repo's workflows, so regressions fail CI instead of going unnoticed.
  • This isn't just a preference: infra hostnames the runners need to reach are LAN-only, so a GitHub-hosted runner is architecturally unable to do this work, not merely disallowed.

Summary

Repo Status Detail
homelab ✅ Compliant Reference implementation
dnd_session_parser ✅ Compliant Uses vars.CI_RUNS_ON/DOCKER_RUNS_ON; wires runs_on_json into ci-toolkit calls
Telegram ✅ Compliant —
dnd_transcriber ✅ Compliant —
tiktooker ✅ Compliant 6 workflows, all on vars.*
aiProject ✅ Compliant —
ci-toolkit ✅ Fixed 2026-08-09 stale-run-watchdog.yml now takes runs_on_json; see systemic finding below
dependency-steward ✅ Fixed 2026-08-09 Runner fix + missing runs_on_json bug in security.yml
nellis-auction-monitor ❌ Non-compliant All jobs on ubuntu-latest, no self-hosted pattern present
rexit_evolved ❌ Non-compliant Worst offender — no self-hosted pattern adopted anywhere, plus a macOS/Windows/Linux release matrix
github-actions (archived) ⚠️ Likely affected Duplicate copy of ci-toolkit workflows; low priority since archived
7 repos with no workflows N/A nfs-file-monitoring, reddit_explorer, kangapoo_databutts, home_assistant, homelabPatching, openstackPatching, AI-Troubleshooter---700-Batch
telegram_companion ✅ Onboarded 2026-08-12 PR #58 CI uses vars.CI_RUNS_ON / DOCKER_RUNS_ON; fleet instances telegram-companion-ci-01/-02 + telegram-companion-docker-01
pleskshared-BoxOfPlesk N/A Empty repo, no commits

Systemic finding: ci-toolkit's stale-run-watchdog.yml (highest leverage fix) — ✅ fixed 2026-08-09

Status: Fixed. ci-toolkit#5 added a required runs_on_json input and merged to main (a4eb873f75413fffe82724fbebf431aa84b21fb8). All five caller repos (dnd_session_parser, Telegram, dnd_transcriber, aiProject, tiktooker) were repointed at that commit and now pass runs_on_json: "${{ vars.CI_RUNS_ON }}". aiProject had been pinned to an unmerged ci-toolkit feature-branch commit rather than main; that was corrected in the same pass.

The reusable workflow spadoople/ci-toolkit/.github/workflows/stale-run-watchdog.yml hardcoded runs-on: ubuntu-latest on its watchdog job. Every other reusable workflow in ci-toolkit (secret-scan.yml, semgrep.yml, workflow-lint.yml, runner-health.yml, library-ci.yml) correctly takes a runs_on_json input — this one did not use it for its own job.

Because callers invoke this workflow via uses: rather than copying its YAML, every repo that calls it silently runs a job on a GitHub-hosted runner today, even though the calling repo itself looks fully compliant:

  • dnd_session_parser
  • Telegram
  • dnd_transcriber
  • aiProject
  • tiktooker

Fixing runs-on in this one file (add a runs_on_json input, apply fromJSON(inputs.runs_on_json), update the ~5 call sites to pass it) closes the gap for all five repos in a single PR — the single highest-leverage fix in this audit.

spadoople/github-actions (archived) holds a near-duplicate copy of these same reusable workflows and likely has the same bug, but since it's archived this is a one-line note rather than an action item.

dependency-steward: runner issue + separate correctness bug — ✅ fixed 2026-08-09

Status: Fixed. Three commits on main: reconcile.yml, test.yml, security.yml. Both workflow files now use runs-on: ${{ fromJSON(vars.CI_RUNS_ON) }}, and security.yml now passes the required runs_on_json input to all three ci-toolkit calls (also repointed at the latest ci-toolkit main commit). This repo already had two general-class runners provisioned (dependency-steward-ci-01/-02 in infra/github-runners/fleet.yml), so no new runner capacity was needed — this was config-only.

  • reconcile.yml: was runs-on: ubuntu-latest
  • test.yml: both jobs were on ubuntu-latest
  • security.yml: called the ci-toolkit reusable workflows but never passed runs_on_json at all. That input is required: true on the reusable workflows it calls, so this call was likely failing outright, not merely running on the wrong runner.

nellis-auction-monitor

ci.yml — all three jobs (test, pipx-test, docker) run on ubuntu-latest. No vars.*/self-hosted scaffolding exists in this repo at all yet; migration starts from zero here.

rexit_evolved (biggest lift)

No self-hosted pattern adopted anywhere in this repo:

  • audit.yml: runs-on: ubuntu-latest
  • ci.yml: all 4 jobs on ubuntu-latest
  • combine-prs.yml: runs-on: ubuntu-latest
  • cd.yml: a release matrix across macos-latest, ubuntu-latest, windows-latest, plus a separate publish-cargo job on ubuntu-latest

The cd.yml matrix is the one open design question in this audit: on-prem runners are presumably Linux, so the macOS/Windows legs of that matrix have no obvious on-prem equivalent. Migrating this repo will need an explicit decision on what happens to cross-platform release builds (e.g., keep a narrowly-scoped hosted-runner exception just for that matrix, use a cross-compilation toolchain from a single Linux runner, or drop non-Linux release artifacts) before the rest of the workflow can be ported mechanically.

Fully compliant repos (no action needed)

homelab, dnd_session_parser, Telegram, dnd_transcriber, tiktooker, aiProject — all use the vars.CI_RUNS_ON/DOCKER_RUNS_ON/DEPLOY_RUNS_ON + fromJSON(...) pattern and wire runs_on_json into ci-toolkit reusable workflow calls correctly. (Five of these six are still indirectly affected by the stale-run-watchdog.yml bug above.)

  1. ci-toolkit: stale-run-watchdog.yml — fixes 5 repos in one PR, zero new runner-label plumbing needed since the callers already pass compliant labels elsewhere.
  2. dependency-steward — fix runs-on in reconcile.yml/test.yml and add the missing runs_on_json input to the security.yml reusable-workflow calls in the same PR.
  3. rexit_evolved — largest single-repo lift: needs runner-label variables created from scratch, all 4 workflows rewritten, and an explicit decision on the cd.yml cross-platform matrix before that file can be finished.
  4. nellis-auction-monitor — same from-zero setup as rexit_evolved but only one workflow file, no cross-platform matrix complication.

Open question before remediation starts

Whether CI_RUNS_ON / DOCKER_RUNS_ON / DEPLOY_RUNS_ON are defined as org-level variables (auto-inherited by every repo including new ones) or per-repo. This wasn't checked as part of this audit — the GitHub MCP toolset used here didn't expose a direct "list repo/org variables" call. If they're per-repo only, each remediated repo needs these variables created before its workflow changes will actually resolve to a runner.

  • Komodo reliability audit — prior audit using the same "read everything, then triangulate a remediation order" approach.