Skip to content

GitHub organization workflow and runner audit — 2026-07-27

Scope and method

This audit covers every repository returned by the spadoople organization API on 2026-07-27. For each default branch it enumerated .github/workflows/*.{yml,yaml}, inspected all runs-on selectors and explicit repository-owner references, checked repository Actions variables, compared self-hosted consumers with infra/github-runners/fleet.yml, and inspected live repository-scoped runner registrations.

Totals:

  • 24 repositories audited
  • 15 repositories with workflows (68 workflow files)
  • 9 repositories without workflows
  • 6 active self-hosted workflow consumers
  • 22 superseded WSL/infra-services runners stopped, disabled, and unregistered

notarealemail remains a valid user identity for CODEOWNERS, Dependabot reviewers, Tailscale ACL membership, and Git provider naming. Those uses are not stale organization references. Repository/action URLs and uses: paths must use spadoople.

Executive findings

  1. P0 — aiProject had no matching runner. Its only workflow used bare self-hosted for seven jobs while the repository had zero permanent runners. Run 30221348082 remained queued for about fifteen hours. It was force-cancelled; the final zombie job was drained with a temporary runner and the temporary registration/files were removed.
  2. Resolved — aiProject runner registration. The owner added aiProject to the existing fine-grained runner-admin PAT on 2026-07-27. Both managed instances registered with exact class labels, and API/UI tests plus image publication completed on the intended general/Docker classes.
  3. P1 — legacy fleet remained live. Twenty-one WSL services plus the old infra-services runner were still enabled/registered even though current repository variables select the managed pool. All 22 were idle, then stopped, disabled, and removed from GitHub. Only managed registrations remain.
  4. P1 — stale-run handling was alert-only. Prometheus already alerted at 10 and 30 minutes, but did not remediate. A GitHub-hosted reusable watchdog and six repo-local schedules now force-cancel queued runs after 30 minutes and in-progress runs after six hours. Manual dispatch defaults to dry-run.
  5. P1 — pre-transfer operational links remained. aiProject setup, tiktooker security/repository links, and comfyui-ollama clone/homelab links needed canonical spadoople URLs. Fix PRs are listed below.
  6. P2 — archived github-actions README remains historical. The archived predecessor still names notarealemail/github-actions and notarealemail/ci-toolkit; its workflow definitions do not execute on a schedule and active callers already use spadoople/ci-toolkit. No PR was opened because the repository is archived.
  7. Resolved — aiProject production target drift. The documented production host 192.168.1.76 refused TCP/22 from both the workstation and the runner. The owner retired the stale SSH deploy stage in aiProject PR #7; the publish-only main run succeeded and obsolete deploy credentials were deleted.

Repository checklist

Repositories with workflows

Repository Workflows Organization migration Runner strategy Result
aiProject 1 Workflow dynamic owner refs are safe; setup clone URL fixed by PR #5 Managed general/Docker selectors and runners verified Pass
ci-toolkit 5 Canonical; notarealemail in workflow-lint.yml is the intentional rejection pattern Reusable inputs are caller-scoped; watchdog PR #3 Pass
Civicomfy 1 No stale owner hardcodes GitHub-hosted only Pass
comfyui-ollama 1 Workflow clean; operational docs fixed by PR #1 GitHub-hosted only Pass with PR
dependency-steward 3 Uses owner: spadoople and spadoople/ci-toolkit GitHub-hosted only Pass
dnd_session_parser 8 Reusable workflow paths canonical; historical docs/user reviewer are valid Managed CI + Docker selectors correct; watchdog PR #123 Pass with PR
dnd_transcriber 3 Reusable workflow paths canonical Managed CI selector correct; watchdog PR #11 Pass with PR
github-actions 5 Workflows clean; archived README is historical/pre-transfer Archived reusable library; no active schedules Pass (archived note)
homelab 11 Workflow paths/URLs canonical Managed CI/LAN selectors correct; fleet/watchdog PR #145 Pass with PR
ins_loader 5 No stale owner hardcodes GitHub-hosted Linux/Windows Pass
nellis-auction-monitor 1 No stale owner hardcodes GitHub-hosted only Pass
OF-Scraper 8 No stale owner hardcodes GitHub-hosted matrix only Pass
Sandbox 5 No stale owner hardcodes GitHub-hosted matrix only Pass
Telegram 4 Reusable workflow paths canonical Managed CI + GitHub-hosted release; watchdog PR #82 Pass with PR
tiktooker 7 Workflow paths canonical; stale security/repo URLs fixed in PR #324 Managed CI + Docker selectors correct; obsolete SELF_HOSTED_RUNS_ON deleted Pass with PR

Repositories without workflows

  • [x] AI-Troubleshooter---700-Batch
  • [x] home_assistant
  • [x] homelabPatching
  • [x] kangapoo_databutts
  • [x] mediaanalyzer
  • [x] nfs-file-monitoring
  • [x] OF-Scraper-Docs
  • [x] openstackPatching
  • [x] pleskshared-BoxOfPlesk

Self-hosted runner checklist

  • [x] homelab: CI_RUNS_ON = ["self-hosted","Linux","X64","homelab","ci"]; DEPLOY_RUNS_ON = ["self-hosted","Linux","X64","homelab","lan","deploy"]
  • [x] aiProject: managed aiProject ci / docker selectors; both exact runner registrations online and class routing live-verified
  • [x] dnd_session_parser: managed dnd_session_parser ci / docker selectors
  • [x] dnd_transcriber: managed dnd_transcriber ci selector
  • [x] Telegram: managed case-sensitive Telegram ci selector
  • [x] tiktooker: managed tiktooker ci / docker selectors; obsolete tiktok-ci variable deleted
  • [x] ci-toolkit and archived github-actions: reusable runner inputs only; no repository-local scheduled self-hosted jobs
  • [x] Legacy WSL runners: 21 services disabled and registrations deleted
  • [x] Legacy infra-services runner: service disabled and registration deleted

Stale/hung workflow remediation

  • [x] Existing 10m warning / 30m critical Prometheus alerts verified in monitoring/prometheus/alerts/github-actions.yml
  • [x] aiProject run 30221348082 force-cancelled and fully completed
  • [x] Reusable watchdog runs on ubuntu-latest, independent of self-hosted capacity
  • [x] Queue states covered: queued, waiting, pending, requested
  • [x] In-progress timeout covered at six hours
  • [x] Current watchdog run excluded from cancellation
  • [x] Manual dry-run mode provided
  • [x] Calls use repo-scoped GITHUB_TOKEN with actions: write; no org PAT
  • [x] Watchdog library and six caller PRs merged
  • [x] Manual dry-run succeeded in all six self-hosted repositories

Pull requests

Opened during this audit:

Relevant follow-up:

Closeout evidence

  1. aiproject-ci-01 and aiproject-docker-01 are online with exact labels.
  2. All six watchdog dry-runs completed successfully.
  3. aiProject run 30318955565 passed API tests, UI tests, Codecov validation, and both image builds.
  4. Homelab #146 merged; image 2.335.0-r4 with GPG 2.4.4 runs on all five managed hosts, and all 17 repository-scoped runners recovered online.
  5. aiProject #5 and #7 merged. Publish-only main run 30325421660 completed successfully.
  6. Retired secrets DEPLOY_HOST, DEPLOY_PATH, DEPLOY_USER, SSH_PRIVATE_KEY, GHCR_TOKEN, and GHCR_USERNAME were deleted.