Alethurgy — Homelab Observability Stack

alethurgy (ἀληθουργία; uh-LEE-thur-jee, /əˈliːθɚdʒi/): the procedures by which what is hidden is brought to light. Not Authelia.

A self-hosted monitoring, logging, alerting, and visualization stack, deployed onto Proxmox VMs by one guided script. Built from widely-adopted open-source projects, pinned to current stable releases.

Architecture

flowchart TB
  subgraph metrics["aleth-metrics VM"]
    caddy[Caddy<br/>Let's Encrypt via<br/>Cloudflare DNS-01] --> grafana[Grafana<br/>+ dashboard library<br/>+ llm-app]
    prom[Prometheus<br/>90d retention] --> grafana
    prom --> am[Alertmanager]
    loki[Loki] --> grafana
    alloy[Alloy] --> loki
    bb[Blackbox<br/>ICMP/HTTP/DNS/cert] --> prom
    snmpx[snmp_exporter] --> prom
    kuma[Uptime Kuma]
    lnms[LibreNMS<br/>SNMP discovery]
    ntop[ntopng + netflow2ng]
    nr[Node-RED<br/>webhook bus]
    netdata[Netdata<br/>edge ML + MCP]
    pvex[pve_exporter] --> prom
  end
  subgraph logs["aleth-logs VM"]
    graylog[Graylog 7.1] --> os[OpenSearch]
    graylog --> mongo[MongoDB 7]
  end
  subgraph siem["aleth-siem VM"]
    wazuh[Wazuh 4.14<br/>manager+indexer+dashboard]
  end
  am -->|webhook| nr
  kuma -->|webhook| nr
  wazuh -->|syslog bridge| graylog
  devices[Firewall / switches / routers] -->|syslog 514| graylog
  devices -->|NetFlow 2055| ntop
  devices -->|SNMP v3| lnms
  nodes[Linux / Windows / Docker nodes] -->|node_exporter, cAdvisor| prom
  nodes -->|Wazuh agents| wazuh

Deployment phases

The deploy script follows the build order below; each phase is health-checked before the next starts.

Foundation — Docker, shared network, Caddy reverse proxy (wildcard Let’s Encrypt cert via Cloudflare DNS-01, or internal CA), secrets in .env
Metrics core — Prometheus, Grafana (+ preloaded dashboards), Alertmanager, exporters, baseline alert rules
Availability — Uptime Kuma
Logs — Loki + Alloy (app logs) and Graylog + OpenSearch + MongoDB (network/security syslog)
Network — LibreNMS (SNMP), ntopng + netflow2ng (flows), blackbox ICMP latency history
Security — Wazuh single-node
Automation — Node-RED webhook bus, dedup flow
LLM/MCP — grafana-llm-app, Grafana MCP, Netdata MCP
Hardening/retention — backups, alert-noise review

Key design decisions

Decision	Choice	Why
Log stack	Loki and Graylog	Loki is cheap and Grafana-native for app logs; Graylog handles syslog inputs, streams, and correlation for network/security events
Latency history	blackbox_exporter ICMP only	One ping load, unified alerting; SmokePing would duplicate it
Log shipper	Grafana Alloy	Promtail hit EOL 2026-03-02
Topology	Split 3 VMs (fallback: single)	OpenSearch + Wazuh indexer each want 8–16 GB
NetFlow	netflow2ng → ntopng	nProbe requires a paid license; netflow2ng is the established free path
VM CPU type	`host`	MongoDB ≥5 requires AVX; Proxmox’s default kvm64 doesn’t expose it
TLS	Let’s Encrypt wildcard via Cloudflare DNS-01	Real certs with no inbound exposure and no per-client CA trust; `TLS_MODE=internal` (Caddy private CA) remains available

Continue with Prerequisites.