Skip to content

LLM observability — LiteLLM + Phoenix

Split stack for homelab LLM operations:

Tool Host URL Purpose
LiteLLM infra-services https://litellm.infra.realemail.app Usage metrics, spend, virtual keys, OpenAI-compatible gateway
Phoenix prox LXC 124 https://phoenix.infra.realemail.app Trace interrogation, RAG debugging, evals

LiteLLM — usage and gateway

Route traffic through the proxy

Point any OpenAI-compatible client at:

https://litellm.infra.realemail.app/v1

Authenticate with a virtual key (Authorization: Bearer sk-…), not your Authentik session. Create keys in the Admin UI: https://litellm.infra.realemail.app/ui (Authentik-gated).

Add models

Edit services/litellm/config.yaml on infra-services and redeploy:

model_list:
  - model_name: gpt-4o
    litellm_params:
      model: openai/gpt-4o
      api_key: os.environ/OPENAI_API_KEY

Provider API keys live in .env (from SOPS). After Phase 9 Ollama returns, add a local model entry pointing at the inference guest.

Grafana

Open Grafana → Homelab → LiteLLM for request rate, token throughput, and cumulative spend (Prometheus job litellm scrapes litellm:4000/metrics).

Smoke test

curl -s https://litellm.infra.realemail.app/v1/models \
  -H "Authorization: Bearer $LITELLM_VIRTUAL_KEY"

Phoenix — traces and research

OTLP endpoint

From any host on 192.168.6.0/24:

http://192.168.6.124:4317

(gRPC OTLP — not exposed through Traefik.)

Python example (OpenInference)

pip install openinference-instrumentation-openai arize-phoenix-otel openai

from phoenix.otel import register
from openinference.instrumentation.openai import OpenAIInstrumentor
import openai

register(endpoint="http://192.168.6.124:4317/v1/traces", project_name="homelab")
OpenAIInstrumentor().instrument()
client = openai.OpenAI()  # or point base_url at LiteLLM
client.chat.completions.create(model="gpt-4o-mini", messages=[{"role": "user", "content": "ping"}])

Open https://phoenix.infra.realemail.app and inspect the trace tree.

Edge auth (Traefik + Authentik)

Path Authentik Auth mechanism
litellm…/ui Yes Authentik forward-auth
litellm…/v1 No LiteLLM virtual key
phoenix.infra… Yes Authentik forward-auth
OTLP :4317 N/A LAN reachability only

Configure Authentik proxy providers for litellm.infra.realemail.app and phoenix.infra.realemail.app per authentik-cross-host-sso.md (owner UI — required before browser access works through Traefik).

Cloudflare DNS: both hostnames are covered by *.infra.realemail.app → 192.168.6.17 (infra-dns-rewrites.yaml).

After merge to main, on infra-services encrypt secrets with SOPS (.env.sops.yaml.example → encrypt → decrypt to gitignored .env). Phoenix secrets stay on LXC 124 only.

Wazuh agent on phoenix: runs on next ansible-pull / patch wave once the branch is merged and someone SSH is fully bootstrapped (patch-controller key on root@192.168.6.124 is in place).

Operations

LiteLLM (infra-services)

cd /opt/homelab/services/litellm
docker compose up -d
docker compose logs -f litellm

Phoenix (prox LXC 124)

ssh someone@192.168.6.124
cd /opt/homelab/services/phoenix
docker compose up -d

Backups

Both stacks are tier 2 — Postgres volumes documented in each backup.yml.