Enterprise Zabbix Extension Pattern: Collecting K8S Pod Network Traffic (via cAdvisor)

By default, Zabbix Kubernetes templates provide only basic metrics such as CPU, memory, disk, node status, and container status. Pod-level network traffic is missing.
In many environments, this gap is covered by Prometheus. Here, however, is a case study where Zabbix alone was extended to collect Pod-level RX/TX traffic metrics.

This design was built from scratch without external references and has already been validated in a production environment.

Design Overview

• Data source: Kubelet cAdvisor metrics
• Collection flow: HTTP agent → Master item → Discovery (LLD) → Item prototypes → Preprocessing chain
• Goal: Per-Pod / Namespace / Interface RX and TX traffic, expressed in bps

1. Master Item

• Use Kubernetes Kubelet by HTTP: Get cadvisor metrics as the master item.
• All Discovery rules and item prototypes depend on this data.

2. Discovery Rule (Prod K8S NODE)

• Name: cadvisor network interface discovery
• Type: Dependent item
• Key: kubelet.cadvisor.net.discovery
• Master item: Prod-K8S NODE: Get cadvisor metrics

Preprocessing (JavaScript)

Extract Pod, Namespace, and Interface. Deduplicate results.

var lines = value.split("\n");
var seen = {};
var result = [];

for (var i = 0; i < lines.length; i++) {
    // Filter only RX metrics
    var match = lines[i].match(/^container_network_receive_bytes_total\{([^}]+)\}/);
    if (!match) continue;

    var labelBlock = match[1];
    var iface = (labelBlock.match(/interface="([^"]+)"/) || [])[1];
    var pod   = (labelBlock.match(/pod="([^"]+)"/) || [])[1];
    var ns    = (labelBlock.match(/namespace="([^"]+)"/) || [])[1];

    // Skip if any label is missing
    if (!iface || !pod || !ns) continue;

    var key = iface + pod + ns;
    if (seen[key]) continue;
    seen[key] = true;

    result.push({
        "{#IFACE}": iface,
        "{#POD}": pod,
        "{#NAMESPACE}": ns
    });
}

return JSON.stringify({ data: result });

3. Item Prototypes

Collect per-Pod RX/TX bytes and convert to bps.

A. RX Traffic

• Name: RX: [{#NAMESPACE}] {#POD} ({#IFACE})
• Key: cadvisor.net.rx[{#IFACE},{#POD},{#NAMESPACE}]
• Unit: bps
• Type: Dependent item (float)
• Master item: Prod K8S NODE: Get cadvisor metrics

Preprocessing Chain

1. JavaScript – extract latest RX value

var lines = value.split("\n");
var iface = "{#IFACE}";
var pod   = "{#POD}";
var ns    = "{#NAMESPACE}";
var latestTs = 0;
var latestVal = 0;

for (var i = 0; i < lines.length; i++) {
    var m = lines[i].match(/^container_network_receive_bytes_total\{([^}]+)\}\s+([0-9.eE+-]+)\s+(\d+)$/);
    if (!m) continue;

    var labels = m[1];
    var val = parseFloat(m[2]);
    var ts  = parseInt(m[3]);

    if (labels.includes('interface="' + iface + '"') &&
        labels.includes('pod="' + pod + '"') &&
        labels.includes('namespace="' + ns + '"')) {
        if (ts > latestTs) {
            latestTs = ts;
            latestVal = val;
        }
    }
}

if (latestTs > 0) return latestVal;
throw "No matching metric found";

2. Change per second – calculate delta per second (Byte/s)
3. JavaScript – convert Byte → bit

return value * 8;  // Byte → bit

B. TX Traffic

• Name: TX: [{#NAMESPACE}] {#POD} ({#IFACE})
• Key: cadvisor.net.tx[{#IFACE},{#POD},{#NAMESPACE}]
• Unit: bps
• Type: Dependent item (float)
• Master item: Prod K8S NODE: Get cadvisor metrics

Preprocessing Chain

1. JavaScript – extract latest TX value

var lines = value.split("\n");
var iface = "{#IFACE}";
var pod   = "{#POD}";
var ns    = "{#NAMESPACE}";
var latestTs = 0;
var latestVal = 0;

for (var i = 0; i < lines.length; i++) {
    var m = lines[i].match(/^container_network_transmit_bytes_total\{([^}]+)\}\s+([0-9.eE+-]+)\s+(\d+)$/);
    if (!m) continue;

    var labels = m[1];
    var val = parseFloat(m[2]);
    var ts  = parseInt(m[3]);

    if (labels.includes('interface="' + iface + '"') &&
        labels.includes('pod="' + pod + '"') &&
        labels.includes('namespace="' + ns + '"')) {
        if (ts > latestTs) {
            latestTs = ts;
            latestVal = val;
        }
    }
}

if (latestTs > 0) return latestVal;
throw "No matching metric found";

2. Change per second – calculate delta per second (Byte/s)
3. JavaScript – convert Byte → bit

return value * 8;  // Byte → bit

4. Trigger Strategy

Physical network interfaces are already monitored separately.
Pod-level metrics here are used for traffic flow analysis only.
No triggers were defined, though it would be straightforward to extend prototypes into alert conditions if needed.

5. Practical Value

• Overcomes the limitation of default Zabbix templates.
• Provides Pod-level network visibility without Prometheus.
• Discovery + prototype structure ensures automatic scaling with new Pods.
• Validated in a production environment.

This design turns Zabbix from a simple node/resource monitoring tool into a solution capable of covering Pod-level network traffic as well.

🛠 마지막 수정일: 2025.09.18