Skip to content

Documentation validation

Repeatable procedure to verify repo documentation matches live homelab reality. Record results in a dated journal entry (docs/journal/YYYY-MM-DD-documentation-validation.md).

When to run

  • After a phase milestone (Phase 7 DNS, Tailscale, etc.)
  • Before handing the repo to another operator
  • Quarterly, or when README.md Owner TODO and reality diverge

1. Live smoke tests

Run from a machine on the LAN with SSH to managed hosts.

DNS (AdGuard)

ssh someone@192.168.6.17
dig @192.168.6.17 google.com +short
dig @192.168.6.17 infra-services.lab.local +short
docker ps --format '{{.Names}}' | grep -i adguard

Pass: public A record; rewrite returns 192.168.6.17; container listed.

Tailscale

ssh someone@192.168.6.17 'tailscale status | head -20'

Pass: infra-services, proxbox-cube, saltierpoop online with expected tags.

ACL GitOps

gh run list --repo spadoople/homelab --workflow=tailscale-acl.yml --limit 1

Pass: latest run success after any infra/tailscale/** change.

Off-LAN (optional)

From phone on cellular + Tailscale subnets enabled:

  • https://100.71.93.130:5001 — DSM
  • https://192.168.6.17 — subnet route to Servers VLAN

2. Repo checks

cd homelab
uv sync --locked
npm ci
uv run python scripts/lint-markdown.py --fix
uv run python scripts/lint-markdown.py
uv run python -m unittest tests.test_docs -v
uv run python scripts/check-docs.py
uv run mkdocs build --strict
uv run python scripts/check-docs.py --site-dir site --report artifacts/docs-audit/local.json

Pass: all commands exit 0. The checker covers entity documentation pointers, service/tool README presence and index representation, role/workflow/generator mappings, navigation, the bounded repository-link check below, and built HTML, sitemap and search inclusion. It does not inspect every private manual's contents. MkDocs checks internal site files and heading anchors; that is distinct from checking GitHub links. None of these establishes that prose is true or that a restore has been exercised.

The path-existence pass in scripts/check-docs.py has these explicit boundaries:

Boundary Implemented check
Documents scanned Links rendered from docs/**/*.md using Python-Markdown's fenced-code and table extensions; not private README bodies or all repository Markdown
URL forms recognized Exact https://github.com/spadoople/homelab/ prefix followed by blob/main/ or tree/main/ and a path
Revision used for existence The current local checked-out tree, not a Git lookup of main, another branch, a tag or a commit
Target test Local path existence; not file contents, blob/tree type correspondence, or GitHub heading/line fragments
Excluded forms Old owners such as notarealemail/homelab, non-main refs (including pinned commit URLs), other repositories, and raw-content URLs
Remote behavior No HTTP requests, authentication checks, redirect checks or remote-status assertions in this source pass

URL query strings and fragments are not part of the path-existence test. A path added on a local branch can pass before that path exists on remote main. Unrecognized forms are outside coverage, not verified links and not evidence of an HTTP failure. Service-index coverage means the README exists and is linked from the index; it does not mean links inside that private README were checked.

Preserve historical owner/ref links as dated evidence. Assess a pinned historical target against its own revision in a separately scoped review, not against the current checkout. Do not silently rewrite it to today's main merely to make a check pass. Any future alias/ref expansion must update this contract and its regression fixtures deliberately.

Generator drift is a separate check. Inventory changes follow the generator validation matrix, not an assumption that the documentation checker regenerates or validates outputs.

Accuracy and completeness review

Use the project map to visit every subsystem. Compare host facts to inventory and dated live reads, service details to Compose/README, backup instructions to runtime manifests and fetch scripts, and workflow prose to current YAML. Check purpose, access/auth, prerequisites, deployment, health, restore, and troubleshooting. Preserve historical records as dated evidence; flag unverified claims explicitly instead of treating old checkmarks as current acceptance. Report inventory drift separately from documentation corrections.

Stale language grep

rg -i 'to do|in progress|not deployed|REPLACE_ME' docs/ --glob '*.md' | rg -v 'documentation-validation|phase-7r-audit|lab-audit|network-observations'

Review hits manually — historical runbooks may legitimately contain past tense.

Owner TODO

Open README.md Owner TODO table. Every :white_check_mark: must be true; every :black_square_button: must still be open work.

Phase runbooks

3. mkdocs nav audit

Compare mkdocs.yml nav: to files under docs/:

# List all markdown under docs except hidden
find docs -name '*.md' | sort

Every Markdown page must appear exactly once in navigation and every nav entry must resolve. The source checker rejects duplicate routes and duplicate sibling labels. Keep historical and retired pages under History, and group operating procedures by task. Follow the navigation conventions when adding or retiring pages. The strict build fails on omissions, missing anchors, and unrecognized relative links. Repository manuals outside docs/ are private sources: link them from the public service directory instead of automatically copying their content.

4. Write evidence

Create or update:

  1. docs/journal/YYYY-MM-DD-documentation-validation.md — pass/fail table, command output summaries, list of doc files changed
  2. docs/journal/YYYY-MM-DD-checkin.md — narrative summary (optional if minor)
  3. Row in journal/index.md

5. Publish

Merge to main. Confirm Deploy Docs workflow succeeds and hldocs-c0acdec9.pages.dev shows the new journal entry. Build the exact merged revision, then run:

uv run python scripts/check-docs.py --site-dir site \
  --live-url https://hldocs-c0acdec9.pages.dev \
  --report artifacts/docs-audit/live.json

The live check compares every expected route to the sitemap and every searchable page/section with the local search index. Missing or stale content fails. It does not check visual layout, external service URLs, authentication, or execute runbooks. A passing local build is not a successful deployment.