Skip to content

Wazuh SIEM (managed Linux)

Role: Security event correlation, FIM, vuln detection on managed Linux hosts. Deploy: Docker stack services/wazuh/ on infra-services (GitOps). Central syslog: Graylog for UDM/UniFi — separate path.

Scope

Host Agent Notes
prox Wazuh agent Hypervisor OS logs, SSH, sudo
infra-services Wazuh agent Docker host; compose logs also flow through Loki separately
saltierpoop Wazuh agent OS-layer only (Saltbox containers unchanged)
graylog Wazuh agent Central syslog LXC OS logs; UDM/UniFi syslog still lands in Graylog
harbor-registry Wazuh agent Container registry LXC OS logs
octoprint Wazuh agent 3D printer controller LXC OS logs
phoenix Wazuh agent Phoenix trace workbench LXC OS logs
pulse Wazuh agent Managed LXC OS logs
recordurbate Wazuh agent Whrrr VMM customer app guest OS logs
ubuncap Wazuh agent Whrrr VMM customer app guest OS logs

The table mirrors the generated wazuh_agents Ansible group. Do not duplicate full managed-Linux syslog into Graylog; Graylog remains the central syslog path for network gear and unmanaged senders.

Deploy stack

On infra-services (first time includes cert bootstrap):

cd /opt/homelab/services/wazuh
cp secrets.sops.yaml.example secrets.sops.yaml   # first bootstrap only
sops secrets.sops.yaml                           # set WAZUH_* passwords, save encrypted
bash /opt/homelab/scripts/render-wazuh-compose-env.sh
bash scripts/bootstrap-certs.sh
bash scripts/sync-wazuh-yml.sh
docker compose --env-file compose.env up -d
bash scripts/komodo-post-deploy.sh
docker compose --env-file compose.env ps

Commit encrypted secrets.sops.yaml — never commit plaintext compose.env.

bootstrap-certs.sh is normally one-time. If it is rerun while Wazuh is live, it recreates indexer, manager, and dashboard so the regenerated TLS chain is not mixed with old in-memory certificates.

Traefik routes dashboard at wazuh.infra.realemail.app with Authentik forward-auth. OpenSearch proxy auth: wazuh-edge-sso.md.

Dashboard breaks after a reboot (EACCES / config errors)

Symptom: after a restart/reboot the dashboard logs repeat Error getting configuration: EACCES ... wazuh.yml or config/TLS errors, even though docker compose ps shows it "healthy" (the healthcheck only probes the local process, not the indexer backend).

Cause: config/wazuh_dashboard/wazuh.runtime.yml must be owned 1000:1000 and read-only. Two failure modes ping-pong if this is wrong:

  • Owned by someone (uid 1001) / mode 0600 → the uid-1000 dashboard cannot read it → EACCES. A running container caches the file, so the break only appears on the next restart.
  • Writable by the dashboard → on startup it appends a second hosts: block → duplicated mapping key (8:1) → config load fails.

Fix (now automatic): the file is rendered 1000:1000 mode 0440 and mounted :ro; sync-wazuh-yml.sh enforces this, and ansible-pull re-asserts cert + runtime-config ownership on boot and every ~30 min, so the dashboard self-heals on its retry loop. To force it immediately:

cd /opt/homelab/services/wazuh
sudo bash scripts/fix-cert-permissions.sh
bash scripts/sync-wazuh-yml.sh

Readiness gates before declaring the stack done:

  1. wazuh-indexer, wazuh-manager, and wazuh-dashboard are healthy.
  2. Owner has set passwords in secrets.sops.yaml, rendered compose.env, and run bash scripts/sync-internal-users-from-env.sh (hashes flow from rendered compose.env into internal_users.yml).
  3. bash scripts/apply-security-config.sh has applied proxy auth and role mappings after any security config change.
  4. A restic snapshot exists for services/wazuh/backup.yml.
  5. The Wazuh restore drill has been completed.

Version upgrade

Bump all three image tags together (see services/wazuh/compose.yml). After git pull:

cd /opt/homelab/services/wazuh
docker compose --env-file compose.env pull
docker compose --env-file compose.env up -d
# wait for indexer GREEN (~60s)
bash scripts/apply-security-config.sh
bash scripts/sync-wazuh-yml.sh
docker compose --env-file compose.env up -d --force-recreate wazuh-dashboard

Diff upstream config from matching wazuh-docker tag. Preserve Authentik/Traefik files: opensearch_dashboards.yml, config.yml, roles_mapping.yml.

Agent install (Ansible)

Agents are installed by the wazuh-agent Ansible role on hosts in the wazuh_agents inventory group. Manager/indexer/dashboard and the agent role pin are on 4.14.7 (infra/ansible/roles/wazuh-agent/defaults/main.yml). The role holds the installed agent package after convergence so unattended upgrades cannot move an agent ahead of the manager. A future role pin change clears that hold with dpkg_selections before the controlled upgrade so older ansible-pull hosts (for example saltierpoop on ansible-core 2.10) stay compatible.

Each host converges via ansible-pull (same as baseline roles). To apply immediately from infra-services:

cd /var/lib/ansible-pull/homelab/infra/ansible
sudo SOPS_AGE_KEY_FILE=/etc/homelab/age-key.txt \
  ansible-playbook -i inventory/generated.yml playbooks/site.yml \
  --limit wazuh_agents --tags wazuh-agent

Manager listens on 192.168.6.17:1514/1515 (and 127.0.0.1 for the infra-services host agent). UFW on infra-services allows Servers VLAN (192.168.6.0/24). The manager compose file binds both addresses explicitly because Docker's 0.0.0.0 publish rejects LAN connections on this host.

Verify in the Wazuh dashboard (Agents) that the expected managed hosts are Active. Optional smoke test: deliberate failed SSH on a host should produce an alert.

Role: infra/ansible/roles/wazuh-agent/.

Active response

Disabled by default in homelab. Enable only with owner approval.

Retention and disk safety

OpenSearch disk watermarks are enabled in services/wazuh/config/wazuh_indexer/wazuh.indexer.yml:

  • low: 85%
  • high: 90%
  • flood stage: 95%

Review Wazuh index growth monthly. Delete or archive old Wazuh indices before the low watermark persists; do not disable disk thresholds to silence the alert. If the flood-stage watermark is reached, OpenSearch can mark indices read-only until disk pressure is cleared.