Zum Inhalt

Enterprise Infrastructure Telemetry: Zabbix 7 LTS, TimescaleDB Hypertables, Zabbix Agent 2 & PSK Encryption

Executive Summary: While synthetic ping monitors tell you whether an endpoint responds, true infrastructure observability requires deep kernel, storage, SMART, systemd, and SNMP hardware telemetry correlated with predictive anomaly triggers. This architectural reference details the production deployment of Zabbix 7.0 LTS backed by PostgreSQL 17 + TimescaleDB hypertable compression, Go-based Zabbix Agent 2 with TLS Pre-Shared Key (PSK) encryption, SNMPv3 network switch polling, and hysteresis trigger engineering.

  • TimescaleDB Hypertable Compression

    Partition history and trends tables into time-series chunks with native columnar compression (~90% disk reduction) and instant chunk drops instead of slow SQL DELETE loops.

  • Zabbix Agent 2 (Go Architecture)

    Replace legacy C agents with the concurrent Go-based zabbix-agent2 supporting native Docker, systemd, SMART (smartctl), PostgreSQL, and Redis plugins.

  • TLS-PSK Encrypted Active Checks

    Authenticate and encrypt all agent-to-server telemetry over port 10050/10051 using 256-bit Pre-Shared Keys (TLSConnect=psk).

  • Predictive & Hysteresis Triggers

    Eliminate flapping alerts using recovery expressions, timeleft() disk-exhaustion forecasting, and Low-Level Discovery (LLD) prototypes.

1. Architecture: Zabbix Server Pollers, Active/Passive Agents & TimescaleDB

Zabbix 7 LTS operates a multi-process C server engine (zabbix_server) with dedicated internal worker pools (Poller, Trapper, HistorySyncer, Preprocessor, LLDManager) writing high-velocity metric streams into PostgreSQL + TimescaleDB:

%%{init: {'flowchart': {'wrappingWidth': 600, 'subGraphTitleMargin': {'top': 15, 'bottom': 15}}}}%%
flowchart TB
    subgraph MonitoredFleet["Monitored Infrastructure Fleet"]
        LinuxHost["Linux Nodes (Zabbix Agent 2)<br/>Systemd + SMART + Docker<br/>TLS-PSK :10050/:10051"]
        Switch["Managed Core Switches<br/>SNMPv3 authPriv (SHA/AES)"]
    end

    subgraph ZabbixCore["Zabbix 7.0 LTS Core Engine"]
        Pollers["Poller & Trapper Pool<br/>Preprocessors + LLD Engine"]
        VCache["ValueCache & HistoryCache<br/>Shared Memory Buffers"]
        WebUI["Zabbix Web Frontend<br/>Nginx + PHP-FPM"]
    end

    subgraph StorageTier["Time-Series Storage Backend"]
        TSDB[("PostgreSQL 17 + TimescaleDB<br/>Compressed Hypertable Chunks")]
    end

    LinuxHost <-->|"Encrypted TLS-PSK"| Pollers
    Switch -->|"SNMPv3 BulkWalk"| Pollers
    Pollers --> VCache
    VCache -->|"Batch Insert"| TSDB
    WebUI <--> TSDB

Why TimescaleDB Eliminates the Zabbix "Housekeeper" Bottleneck

In a standard relational database, Zabbix's internal Housekeeper process periodically executes massive DELETE FROM history WHERE clock < ... queries to purge expired metrics. On large installations, this causes severe PostgreSQL table bloat, high I/O wait, and autovacuum thrashing. With TimescaleDB, the history, history_uint, and trends tables are partitioned into time chunks (e.g., 1-day intervals). Dropping 90-day-old data becomes an instantaneous drop_chunks() metadata operation requiring zero row-by-row deletion.

2. Production Stack: Zabbix 7.0 LTS + PostgreSQL 17 TimescaleDB

/opt/zabbix/compose.yaml
services:
  postgres-tsdb:
    image: timescale/timescaledb-ha:pg17
    container_name: zabbix-timescaledb
    restart: unless-stopped
    shm_size: "1g"
    environment:
      POSTGRES_USER: zabbix
      POSTGRES_PASSWORD: "${ZBX_DB_PASSWORD}"
      POSTGRES_DB: zabbix
    command:
      - "postgres"
      - "-c"
      - "shared_buffers=1GB"
      - "-c"
      - "work_mem=32MB"
      - "-c"
      - "maintenance_work_mem=256MB"
      - "-c"
      - "max_connections=200"
    volumes:
      - ./pgdata:/home/postgres/pgdata/data:Z
    networks:
      - zbx_net

  zabbix-server:
    image: zabbix/zabbix-server-pgsql:alpine-7.0-latest
    container_name: zabbix-server
    restart: unless-stopped
    depends_on:
      - postgres-tsdb
    ports:
      - "10051:10051/tcp"
    environment:
      DB_SERVER_HOST: postgres-tsdb
      POSTGRES_USER: zabbix
      POSTGRES_PASSWORD: "${ZBX_DB_PASSWORD}"
      POSTGRES_DB: zabbix
      ZBX_ENABLE_SNMP_TRAPS: "false"
      # Tune Internal Pollers and Shared Memory Caches
      ZBX_STARTPOLLERS: 10
      ZBX_STARTPREPROCESSORS: 8
      ZBX_STARTLLDPROCESSORS: 4
      ZBX_CACHESIZE: 64M
      ZBX_HISTORYCACHESIZE: 64M
      ZBX_HISTORYINDEXCACHESIZE: 32M
      ZBX_TRENDCACHESIZE: 32M
      ZBX_VALUECACHESIZE: 128M
    networks:
      - zbx_net

  zabbix-web:
    image: zabbix/zabbix-web-nginx-pgsql:alpine-7.0-latest
    container_name: zabbix-web
    restart: unless-stopped
    depends_on:
      - postgres-tsdb
      - zabbix-server
    ports:
      - "127.0.0.1:8085:8080/tcp"
    environment:
      ZBX_SERVER_HOST: zabbix-server
      DB_SERVER_HOST: postgres-tsdb
      POSTGRES_USER: zabbix
      POSTGRES_PASSWORD: "${ZBX_DB_PASSWORD}"
      POSTGRES_DB: zabbix
      PHP_TZ: "Europe/Berlin"
    networks:
      - zbx_net

networks:
  zbx_net:
    driver: bridge

Enable the TimescaleDB extension and initialize Zabbix's hypertable schema after initial startup:

Enabling TimescaleDB Hypertables & Columnar Compression
# 1. Create the TimescaleDB extension inside the zabbix database
docker exec -i zabbix-timescaledb psql -U zabbix -d zabbix \
  -c "CREATE EXTENSION IF NOT EXISTS timescaledb CASCADE;"

# 2. Apply the official Zabbix TimescaleDB schema migration script
docker exec -i zabbix-server cat /usr/share/doc/zabbix-sql-scripts/postgresql/timescaledb/schema.sql \
  | docker exec -i zabbix-timescaledb psql -U zabbix -d zabbix

3. Deploying Zabbix Agent 2 with TLS-PSK Encryption & SMART/Docker Plugins

On every monitored Linux host, install zabbix-agent2 (written in Go) rather than the legacy zabbix-agent package. Generate a 256-bit hexadecimal Pre-Shared Key (PSK):

Provisioning Zabbix Agent 2 with 256-bit TLS-PSK on Debian/Ubuntu
# 1. Generate a 256-bit (32-byte hex) Pre-Shared Key
sudo mkdir -p /etc/zabbix/psk
openssl rand -hex 32 | sudo tee /etc/zabbix/psk/agent.psk
sudo chown -R zabbix:zabbix /etc/zabbix/psk
sudo chmod 0400 /etc/zabbix/psk/agent.psk

# 2. Grant zabbix user access to Docker socket and NVMe/SATA SMART telemetry
sudo usermod -aG docker zabbix
cat << 'EOF' | sudo tee /etc/sudoers.d/zabbix-smartctl
zabbix ALL=(ALL) NOPASSWD: /usr/sbin/smartctl
EOF
sudo chmod 0440 /etc/sudoers.d/zabbix-smartctl

Configure /etc/zabbix/zabbix_agent2.conf to enforce psk encryption for both passive and active checks:

/etc/zabbix/zabbix_agent2.conf
Server=192.168.10.25
ServerActive=192.168.10.25
Hostname=proxmox-node-01.lan.nejat-hakan.de

# Enforce TLS-PSK for both incoming (Passive) and outgoing (Active) connections
TLSConnect=psk
TLSAccept=psk
TLSPSKIdentity=PSK_PROXMOX_NODE_01
TLSPSKFile=/etc/zabbix/psk/agent.psk

# Enable built-in Go plugins for Systemd, Docker, and SMART disk health
Plugins.Smart.Path=/usr/sbin/smartctl

4. Engineering Flapping-Resistant Triggers (Hysteresis & Forecasting)

Naive threshold triggers (last(/host/system.cpu.util) > 85) fire and resolve repeatedly whenever a metric hovers between 84.9% and 85.1%. In Zabbix 7 expression syntax, combine time-window averages, Hysteresis recovery expressions, and linear regression forecasting (timeleft):

Production Zabbix 7 Trigger Expressions
# 1. Hysteresis Trigger: Alert when 5m CPU load > 85%, resolve only when 10m CPU load drops < 70%
Problem Expression:  avg(/Linux by Zabbix agent/system.cpu.util, 5m) > 85
Recovery Expression: avg(/Linux by Zabbix agent/system.cpu.util, 10m) < 70

# 2. Predictive Disk Exhaustion: Alert if linear regression predicts filesystem will hit 100% within 24h
Problem Expression:  last(/Linux by Zabbix agent/vfs.fs.dependent.size[/,pused]) > 80
                     and timeleft(/Linux by Zabbix agent/vfs.fs.dependent.size[/,pused], 6h, 100) < 24h

# 3. NVMe / SATA SMART Critical Warning (Immediate Hardware Replacement Required)
Problem Expression:  last(/SMART by Zabbix agent 2/smart.disk.critical_warning[{#NAME}]) > 0

5. Verification & Remote Agent Diagnostics

Use zabbix_get from the Zabbix server to verify TLS-PSK negotiation and test live zabbix-agent2 plugin keys:

Testing TLS-PSK Handshake & SMART Plugin Telemetry via zabbix_get
# 1. Test local Agent 2 plugin keys directly on the monitored node
sudo zabbix_agent2 -t smart.disk.get
sudo zabbix_agent2 -t docker.containers

# 2. Query remote Agent 2 over TLS-PSK from the Zabbix server
docker exec zabbix-server zabbix_get \
  -s 192.168.10.20 -p 10050 \
  --tls-connect psk \
  --tls-psk-identity "PSK_PROXMOX_NODE_01" \
  --tls-psk-file /var/lib/zabbix/enc/agent.psk \
  -k "system.uname"