Skip to content

Connect a Prox LXC as a Komodo Server

Register Docker LXCs (phoenix, sonarqube) so Core can pull /opt/homelab and deploy stacks via deploy-infra. These are source-defined targets, not evidence that a guest is healthy now. The recorded version checks below are dated evidence; the uplift's code rollout and application acceptance remain pending.

Preferred model: inbound periphery + GitOps [[server]]

Core dials each LXC on :8120. Servers are declared in services/komodo/resources.toml through the existing infra-services-01 Sync, not by creating Servers in the UI. Keep KOMODO_UI_WRITE_DISABLED=true. If the initial Sync/provider is absent, stop at the bootstrap gate before attempting onboarding.

[[server]]
name = "phoenix"
[server.config]
address = "https://192.168.6.124:8120"
enabled = true

Use insecure_tls = true while Periphery serves its self-signed cert (default ssl_enabled = true).

Prerequisites

  • LXC is Ubuntu with Docker Engine + Compose plugin
  • Full git clone at /opt/homelab (not a partial services/ tree)
  • GitHub deploy key at /root/.ssh/github_deploy (and/or Git Provider PAT for PullRepo)
  • Age key at /etc/homelab/age-key.txt for SOPS decrypt of stack .env
  • Port 8120/tcp reachable from infra-services (192.168.6.17)
  • Authorized maintenance window; confirm guest hostname/IP and intended Server name before running root commands in Bash on that guest, from /root
  • Owner reviews installed versions/digests and the pinned installer for compatibility before use; source pins alone do not prove what is running

Version baseline and read-only evidence

Keep Core, infra-services Periphery, LXC Periphery binaries, and the host-Python helper images on 2.3.4 together. Compose uses literal :2.3.4 image tags so Dependabot can see them; COMPOSE_KOMODO_IMAGE_TAG is not used. Review version bumps together with all three helper wrappers, this recipe, and tests/test_komodo_versions.py. Any KOMODO_PREDEPLOY_PYTHON_IMAGE override must identify a reviewed exact version or digest, not a moving tag.

Read-only checks on 2026-10-26 03:14 UTC (October 25 local time) found:

Component Version evidence
Core Existing startup log: Komodo Core version: v2.3.4; image digest sha256:91b2afd7b3bb2abea0ebd3046d65ec457eb1f3d505868d1d60cb0f8562490ef9
infra-services Periphery periphery --version: periphery 2.3.4; image digest sha256:fb4b52961fcce7a31e3776930e422967b6640f653de876f59f26dea243e29beb
Phoenix (192.168.6.124, CT 124) /usr/local/bin/periphery --version: periphery 2.3.4; systemd active/running
SonarQube (192.168.6.101, CT 101) /usr/local/bin/periphery --version: periphery 2.3.4; systemd active/running

Both LXC installed binaries and their running /proc/<pid>/exe files had SHA-256 46d50313970aea59d293bbe237648f4becea74e8abc2f30f1bb62b73456fc72e. This is observed production evidence, not an upstream signature. The image IDs/digests agree with the capability audit. Those checks performed no live installation, pull, restart, or deployment.

Guided preparation update: the exact Periphery :2.3.4 helper tag is now cached on infra-services at the running digest shown above; no container was recreated. Running containers still reference :2. Compose pin application remains pending, as does cache verification on any other Docker host needing the fallback. Before using the helpers, explicitly verify the reviewed ghcr.io/moghtech/komodo-periphery:2.3.4 tag on the host that will run them. All helpers use --pull never; a missing exact tag must fail rather than download a runtime during deployment. Roll out the pinned Core/Periphery Compose pair through the controlled self-managed Komodo maintenance path (not deploy-infra). The LXCs already match; do not reinstall them just to apply this documentation change. Recheck versions at rollout time; an unreachable or unverified host remains an open rollout requirement, not presumed compliant.

To repeat the guest checks from the workstation, use the documented alias and patch-controller key (the private key stays on infra-services):

ssh -o BatchMode=yes -o ConnectTimeout=10 infra-services \
  'sudo -n ssh -i /etc/homelab/patch-controller/id_ed25519 -o IdentitiesOnly=yes -o BatchMode=yes -o ConnectTimeout=10 -o StrictHostKeyChecking=yes root@192.168.6.124 "/usr/local/bin/periphery --version; systemctl is-active periphery; sha256sum /usr/local/bin/periphery"'
ssh -o BatchMode=yes -o ConnectTimeout=10 infra-services \
  'sudo -n ssh -i /etc/homelab/patch-controller/id_ed25519 -o IdentitiesOnly=yes -o BatchMode=yes -o ConnectTimeout=10 -o StrictHostKeyChecking=yes root@192.168.6.101 "/usr/local/bin/periphery --version; systemctl is-active periphery; sha256sum /usr/local/bin/periphery"'

For Core use container/image inspection and existing startup logs. Never run core --version: this build starts a second server instead of just printing a version. See SSH access if a guest needs the documented prox route rather than direct SSH; do not weaken host-key checks.

Pinned installer and preparation

These are operator maintenance instructions, not automatic rollout steps. The v2.3.4 installer source was checked: --version (alias -v) takes a release tag including v. It fetches https://github.com/moghtech/komodo/releases/download/v2.3.4/periphery-x86_64 (or periphery-aarch64 on ARM64). Pin the installer and --config-url, whose upstream default otherwise points at the development branch. Never use an installer from a moving branch or release alias.

Upstream eagerly requests latest-release metadata while constructing its CLI, even with --version v2.3.4; GitHub API access/rate limits can therefore block it. That lookup does not override the explicit binary version. Do not drop the flag as a workaround. The installer stops/replaces the binary and starts the service itself; it preserves existing config and unit files unless explicitly forced.

As root on the guest, in one Bash session, download for review before executing:

set -euo pipefail
umask 077
WORK_DIR="$(mktemp -d /root/komodo-periphery-2.3.4.XXXXXX)"
printf 'Keep this private rollback directory: %s\n' "$WORK_DIR"

# Existing installations: preserve the binary, config, and unit before any change.
# Keep existing authentication keys/certificates and unit drop-ins in place.
if test -e /usr/local/bin/periphery; then
  /usr/local/bin/periphery --version > "$WORK_DIR/version.before"
  systemctl is-enabled periphery > "$WORK_DIR/enabled.before" || true
  systemctl is-active periphery > "$WORK_DIR/active.before" || true
  tar -C / -czf "$WORK_DIR/previous-install.tar.gz" \
    usr/local/bin/periphery etc/komodo/periphery.config.toml \
    etc/systemd/system/periphery.service
fi

curl --fail --show-error --location \
  https://raw.githubusercontent.com/moghtech/komodo/v2.3.4/scripts/setup-periphery.py \
  --output "$WORK_DIR/setup-periphery.py"
curl --fail --show-error --location \
  https://raw.githubusercontent.com/moghtech/komodo/v2.3.4/config/periphery.config.toml \
  --output "$WORK_DIR/periphery.config.toml"
sha256sum "$WORK_DIR/setup-periphery.py" "$WORK_DIR/periphery.config.toml"

Retain the downloaded artifacts/checksums with the maintenance record. Inspect the script before running it. For nonstandard binary/config/unit paths, first inspect systemctl cat periphery and adjust the backup/restore paths; do not overwrite an unrecorded installation. The archive above deliberately excludes checkout/data directories; separately back up keys/certs and any drop-ins before editing them.

Install systemd periphery (inbound)

For a fresh guest, prepare its config before the installer auto-starts. This example is Phoenix; use sonarqube instead for that guest. Do not run this block over an existing config. The pinned template already has port = 8120, TLS on, and inbound mode by default; edit existing keys, never append duplicate TOML keys.

test ! -e /etc/komodo/periphery.config.toml
install -d -m 0700 /etc/komodo
sed -e 's/^allowed_ips = \[\]$/allowed_ips = ["192.168.6.17"]/' \
    -e 's/^# connect_as = "server-name"$/connect_as = "phoenix"/' \
    "$WORK_DIR/periphery.config.toml" > /etc/komodo/periphery.config.toml

For an upgrade, retain/review the existing inbound config, IP restriction, and Core authentication instead. Existing config is not rewritten by installer flags. connect_as must match the [[server]] / [[stack]].server name. Omit --core-address for inbound mode; ensure the existing config does too.

python3 "$WORK_DIR/setup-periphery.py" \
  --version v2.3.4 \
  --config-url https://raw.githubusercontent.com/moghtech/komodo/v2.3.4/config/periphery.config.toml \
  --connect-as=phoenix \
  --root-directory=/etc/komodo

/usr/local/bin/periphery --version | grep -Fx 'periphery 2.3.4'
systemctl enable --now periphery
systemctl is-active periphery
ss -lntp | grep ':8120'

After the external resources.toml Sync applies the [[server]] blocks, confirm Servers show OK and the expected version in Komodo. Verify the installed binary checksum and the running process executable agree (find the PID with pgrep -ax periphery, then hash /proc/<pid>/exe); a version probe alone does not prove systemd is running that binary. Record version, checksum, connectivity, and service state before declaring the rollout complete.

Verification failure / rollback

Do not retry with a floating tag. For an existing standard installation, restore from the same session's private directory (or set WORK_DIR to the recorded path):

test -s "$WORK_DIR/previous-install.tar.gz"
systemctl stop periphery
tar -C / -xzf "$WORK_DIR/previous-install.tar.gz"
systemctl daemon-reload
systemctl start periphery
/usr/local/bin/periphery --version
systemctl is-active periphery

Compare with version.before, repeat the running-binary checksum and Core connectivity checks, and restore the original enabled/active state recorded in enabled.before / active.before (do not leave a previously disabled agent enabled). Restore any separately backed-up config drop-ins/keys only if they were changed. For a failed new install with no previous archive, stop/disable Periphery and retain the artifacts/config for diagnosis; there is no previous binary to restore. For container rollback, retain the pre-change Core/Periphery digests and Komodo backup and restore the pair together through the self-managed maintenance path.

Alternate: outbound + onboarding key

If Core cannot reach :8120, stop and ask the owner to approve the outbound onboarding model. Key issuance and initial registration permissions depend on the installed version and authorized identity; that onboarding path remains unverified by the relay Service User's read-only authentication. Keep the UI write lock enabled; do not temporarily unlock it to mint a key. Verify a supported bootstrap/key-management path and private secret delivery with the owner before constructing an outbound install command. Executing an existing Sync is separate from creating a Server or onboarding key.

An approved outbound recipe must use the same pinned installer/config/binary and backup/verification gates above, not the fresh inbound config block. Existing config is not replaced by installer flags. Do not put onboarding secrets in shell history, process arguments or logs; verify the supported private delivery mechanism first.

/opt/homelab checkout

# If replacing a snowflake tree, move it aside first (Docker volumes are separate)
mv /opt/homelab /opt/homelab.snowflake.bak
git clone git@github.com:spadoople/homelab.git /opt/homelab
# Restore any host-only .env files from the bak tree into services/<name>/

Stacks and deploy-infra

Each LXC needs:

  • [[repo]] with server = "<hostname>" and path = "/opt/homelab"
  • [[stack]] with server = "<hostname>" and run_directory = "/opt/homelab/services/<name>"

New resource additions require reviewed external bootstrap Sync and Read grants before normal relay use. After merging the reviewed change to main, use the canonical manual trigger, not a direct host-side bypass. Native GitHub reviewer/branch protection is currently absent; explicit operator review and maintenance coordination are required.

The relay runs external RunSync for infra-services-01 first and waits for its exact successful terminal Update before invoking deploy-infra and waiting for that Update too. Sync stays outside the Procedure whose definition it manages. Inspect nested pull/deploy results and application health as well.

The top-level deploy-infra has one parallel stage of RunProcedure calls:

  • deploy-infra-services: pull homelab, then its conditional stack deployments.
  • deploy-phoenix: pull homelab-phoenix, then its conditional stack deployment.
  • deploy-sonarqube: pull homelab-sonarqube, then its conditional stack deployment.

Each child owns its ordered pull → conditional deployment stages. A failed pull must stop that child's deployment without blocking the other hosts' independent children. There is no internal RunSync or shared all-host pull barrier; Komodo (self-managed) and OneUptime (manual-upgrade) remain outside the automatic targets.

Stop further dispatches on failed, missing or still-running Updates, a wrong guest identity or a Server that is not OK. Resolve connectivity/configuration first; review any revert in git and its pending Sync actions before retrying the same canonical recipe. Do not delete guest data, replace a live checkout or relax the write lock to make an onboarding status appear healthy.