Skip to content

GitHub organization workflow and runner audit — 2026-07-27

Scope and method

This audit covers every repository returned by the spadoople organization API on 2026-07-27. For each default branch it enumerated .github/workflows/*.{yml,yaml}, inspected all runs-on selectors and explicit repository-owner references, checked repository Actions variables, compared self-hosted consumers with infra/github-runners/fleet.yml, and inspected live repository-scoped runner registrations.

Totals:

  • 24 repositories audited
  • 15 repositories with workflows (68 workflow files)
  • 9 repositories without workflows
  • 6 active self-hosted workflow consumers
  • 22 superseded WSL/infra-services runners stopped, disabled, and unregistered

notarealemail remains a valid user identity for CODEOWNERS, Dependabot reviewers, Tailscale ACL membership, and Git provider naming. Those uses are not stale organization references. Repository/action URLs and uses: paths must use spadoople.

Executive findings

  1. P0 — aiProject had no matching runner. Its only workflow used bare self-hosted for seven jobs while the repository had zero permanent runners. Run 30221348082 remained queued for about fifteen hours. It was force-cancelled; the final zombie job was drained with a temporary runner and the temporary registration/files were removed.
  2. P0 — aiProject runner registration is owner-blocked. Managed general and Docker instances are declared and the Ansible rollout is otherwise green, but the SOPS runner-administration PAT is restricted to the original five repositories. GitHub returns 404 until that fine-grained PAT is granted aiProject access. The two new units are stopped to avoid restart loops.
  3. P1 — legacy fleet remained live. Twenty-one WSL services plus the old infra-services runner were still enabled/registered even though current repository variables select the managed pool. All 22 were idle, then stopped, disabled, and removed from GitHub. Only managed registrations remain.
  4. P1 — stale-run handling was alert-only. Prometheus already alerted at 10 and 30 minutes, but did not remediate. A GitHub-hosted reusable watchdog and six repo-local schedules now force-cancel queued runs after 30 minutes and in-progress runs after six hours. Manual dispatch defaults to dry-run.
  5. P1 — pre-transfer operational links remained. aiProject setup, tiktooker security/repository links, and comfyui-ollama clone/homelab links needed canonical spadoople URLs. Fix PRs are listed below.
  6. P2 — archived github-actions README remains historical. The archived predecessor still names notarealemail/github-actions and notarealemail/ci-toolkit; its workflow definitions do not execute on a schedule and active callers already use spadoople/ci-toolkit. No PR was opened because the repository is archived.

Repository checklist

Repositories with workflows

Repository Workflows Organization migration Runner strategy Result
aiProject 1 Workflow dynamic owner refs are safe; setup clone URL covered by existing PR #5 Failed audit: bare self-hosted, no runners; PR #6 + homelab #145 Owner PAT scope pending
ci-toolkit 5 Canonical; notarealemail in workflow-lint.yml is the intentional rejection pattern Reusable inputs are caller-scoped; watchdog PR #3 Pass
Civicomfy 1 No stale owner hardcodes GitHub-hosted only Pass
comfyui-ollama 1 Workflow clean; operational docs fixed by PR #1 GitHub-hosted only Pass with PR
dependency-steward 3 Uses owner: spadoople and spadoople/ci-toolkit GitHub-hosted only Pass
dnd_session_parser 8 Reusable workflow paths canonical; historical docs/user reviewer are valid Managed CI + Docker selectors correct; watchdog PR #123 Pass with PR
dnd_transcriber 3 Reusable workflow paths canonical Managed CI selector correct; watchdog PR #11 Pass with PR
github-actions 5 Workflows clean; archived README is historical/pre-transfer Archived reusable library; no active schedules Pass (archived note)
homelab 11 Workflow paths/URLs canonical Managed CI/LAN selectors correct; fleet/watchdog PR #145 Pass with PR
ins_loader 5 No stale owner hardcodes GitHub-hosted Linux/Windows Pass
nellis-auction-monitor 1 No stale owner hardcodes GitHub-hosted only Pass
OF-Scraper 8 No stale owner hardcodes GitHub-hosted matrix only Pass
Sandbox 5 No stale owner hardcodes GitHub-hosted matrix only Pass
Telegram 4 Reusable workflow paths canonical Managed CI + GitHub-hosted release; watchdog PR #82 Pass with PR
tiktooker 7 Workflow paths canonical; stale security/repo URLs fixed in PR #324 Managed CI + Docker selectors correct; obsolete SELF_HOSTED_RUNS_ON deleted Pass with PR

Repositories without workflows

  • [x] AI-Troubleshooter---700-Batch
  • [x] home_assistant
  • [x] homelabPatching
  • [x] kangapoo_databutts
  • [x] mediaanalyzer
  • [x] nfs-file-monitoring
  • [x] OF-Scraper-Docs
  • [x] openstackPatching
  • [x] pleskshared-BoxOfPlesk

Self-hosted runner checklist

  • [x] homelab: CI_RUNS_ON = ["self-hosted","Linux","X64","homelab","ci"]; DEPLOY_RUNS_ON = ["self-hosted","Linux","X64","homelab","lan","deploy"]
  • [ ] aiProject: variables set to managed aiProject ci / docker selectors; units declared, but registration PAT access remains pending
  • [x] dnd_session_parser: managed dnd_session_parser ci / docker selectors
  • [x] dnd_transcriber: managed dnd_transcriber ci selector
  • [x] Telegram: managed case-sensitive Telegram ci selector
  • [x] tiktooker: managed tiktooker ci / docker selectors; obsolete tiktok-ci variable deleted
  • [x] ci-toolkit and archived github-actions: reusable runner inputs only; no repository-local scheduled self-hosted jobs
  • [x] Legacy WSL runners: 21 services disabled and registrations deleted
  • [x] Legacy infra-services runner: service disabled and registration deleted

Stale/hung workflow remediation

  • [x] Existing 10m warning / 30m critical Prometheus alerts verified in monitoring/prometheus/alerts/github-actions.yml
  • [x] aiProject run 30221348082 force-cancelled and fully completed
  • [x] Reusable watchdog runs on ubuntu-latest, independent of self-hosted capacity
  • [x] Queue states covered: queued, waiting, pending, requested
  • [x] In-progress timeout covered at six hours
  • [x] Current watchdog run excluded from cancellation
  • [x] Manual dry-run mode provided
  • [x] Calls use repo-scoped GITHUB_TOKEN with actions: write; no org PAT
  • [ ] Merge the watchdog library and six caller PRs

Pull requests

Opened during this audit:

Already open and relevant:

Merge / rollout order

  1. Add spadoople/aiProject to the existing fine-grained runner-admin PAT's repository access. Re-encrypt registration.sops.yaml only if the token itself rotates, then restart the two stopped aiProject units.
  2. Verify aiproject-ci-01 and aiproject-docker-01 online with exact labels.
  3. Merge homelab #145 and aiProject #6; verify aiProject CI routes by class.
  4. Merge ci-toolkit #3, then the six watchdog callers (their full-SHA pin is already immutable and valid).
  5. Merge aiProject #5, tiktooker #324, and comfyui-ollama #1 for remaining operational URL cleanup.