What is Prometheus?

Prometheus is an open-source, time-series monitoring system. Its defining trait is the pull model: the Prometheus server periodically scrapes each target's /metrics endpoint.

Pull model: the server connects to targets and collects metrics
Multi-dimensional data model: metric name plus label combinations
PromQL: a powerful time-series query language
Service discovery: static config plus dynamic discovery (K8s, Consul, etc.)

Node Exporter is the official exporter that exposes OS-level metrics from a Linux host—CPU, memory, disk, network, and more.

Architecture

CODE

[Node Exporter:9100] <--scrape-- [Prometheus:9090] --query--> [Grafana]
         (host metrics)               (TSDB storage)         (visualization)

Component	Port	Role
Prometheus	9090	Collect, store, query, alert
Node Exporter	9100	Expose host OS metrics
Alertmanager	9093	Alert routing (optional)

1. Install Node Exporter

Bash

VER=1.8.2
cd /tmp
curl -LO https://github.com/prometheus/node_exporter/releases/download/v${VER}/node_exporter-${VER}.linux-amd64.tar.gz
tar xzf node_exporter-${VER}.linux-amd64.tar.gz
sudo cp node_exporter-${VER}.linux-amd64/node_exporter /usr/local/bin/

sudo useradd --no-create-home --shell /bin/false node_exporter

systemd unit /etc/systemd/system/node_exporter.service:

INI

[Unit]
Description=Prometheus Node Exporter
After=network-online.target

[Service]
User=node_exporter
Group=node_exporter
Type=simple
ExecStart=/usr/local/bin/node_exporter \
  --collector.systemd \
  --collector.processes

[Install]
WantedBy=multi-user.target

Bash

sudo systemctl daemon-reload
sudo systemctl enable --now node_exporter
curl -s localhost:9100/metrics | head

If you see metrics such as node_cpu_seconds_total and node_memory_MemAvailable_bytes, the install succeeded.

2. Install Prometheus

Bash

VER=2.53.0
cd /tmp
curl -LO https://github.com/prometheus/prometheus/releases/download/v${VER}/prometheus-${VER}.linux-amd64.tar.gz
tar xzf prometheus-${VER}.linux-amd64.tar.gz
cd prometheus-${VER}.linux-amd64
sudo cp prometheus promtool /usr/local/bin/
sudo mkdir -p /etc/prometheus /var/lib/prometheus
sudo cp -r consoles console_libraries /etc/prometheus/

3. Configure prometheus.yml

/etc/prometheus/prometheus.yml:

YAML

global:
  scrape_interval: 15s
  evaluation_interval: 15s
  external_labels:
    cluster: 'prod'

rule_files:
  - "/etc/prometheus/rules/*.yml"

alerting:
  alertmanagers:
    - static_configs:
        - targets: ['localhost:9093']

scrape_configs:
  - job_name: 'prometheus'
    static_configs:
      - targets: ['localhost:9090']

  - job_name: 'node'
    static_configs:
      - targets:
          - '192.168.1.31:9100'
          - '192.168.1.32:9100'
        labels:
          env: 'prod'
    relabel_configs:
      # tidy the instance label so it looks like a hostname
      - source_labels: [__address__]
        regex: '([^:]+):.*'
        target_label: host
        replacement: '$1'

Validate the config, then start:

Bash

promtool check config /etc/prometheus/prometheus.yml

/usr/local/bin/prometheus \
  --config.file=/etc/prometheus/prometheus.yml \
  --storage.tsdb.path=/var/lib/prometheus \
  --storage.tsdb.retention.time=30d \
  --web.enable-lifecycle

Enabling --web.enable-lifecycle allows a zero-downtime reload.

Bash

curl -X POST http://localhost:9090/-/reload

Check target health in the web UI at http://<server>:9090/targets, or:

Bash

curl -s 'http://localhost:9090/api/v1/targets' | jq '.data.activeTargets[].health'

4. Essential PromQL queries

CPU utilization (%)

PROMQL

100 - (avg by (instance) (rate(node_cpu_seconds_total{mode="idle"}[5m])) * 100)

Memory utilization (%)

PROMQL

(1 - (node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes)) * 100

Disk utilization (%) — root partition

PROMQL

100 - (node_filesystem_avail_bytes{mountpoint="/"} / node_filesystem_size_bytes{mountpoint="/"} * 100)

Network receive traffic (bytes/s)

PROMQL

rate(node_network_receive_bytes_total{device!~"lo|veth.*"}[5m])

Disk-full prediction (risk if below 0 in 4 hours)

PROMQL

predict_linear(node_filesystem_avail_bytes{mountpoint="/"}[6h], 4 * 3600) < 0

Use rate() only on Counter metrics (monotonically increasing). Do not use rate on Gauges (values that go up and down, e.g. memory).

5. Cut query cost with recording rules

Precompute frequently used, expensive queries and store them as new time series. /etc/prometheus/rules/node.yml:

YAML

groups:
  - name: node-recording
    interval: 30s
    rules:
      - record: instance:node_cpu_utilization:ratio
        expr: 100 - (avg by (instance) (rate(node_cpu_seconds_total{mode="idle"}[5m])) * 100)

      - record: instance:node_memory_utilization:ratio
        expr: (1 - (node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes)) * 100

In Grafana or alerting, reference the precomputed series, e.g. instance:node_cpu_utilization:ratio.

6. Retention policy and storage

Option	Meaning
--storage.tsdb.retention.time=30d	Keep 30 days
--storage.tsdb.retention.size=50GB	Size-based retention

For long-term retention and multi-cluster aggregation, connect remote storage such as Thanos or VictoriaMetrics. A single Prometheus instance's local disk is suited to short-term retention.

Summary

Item	Key point
Collection	Pull — Prometheus scrapes /metrics
Host metrics	Node Exporter (:9100)
Config check	promtool check config
Zero-downtime reload	--web.enable-lifecycle + POST /-/reload
Counter / Gauge	rate() for Counters; use Gauges as-is
Performance	Precompute with recording rules
Retention	retention.time / .size; Thanos for long term

The core of Prometheus is distinguishing metric types—"rate for Counters, leave Gauges as-is"—and the PromQL patterns that follow. Once OS metrics are collected reliably with Node Exporter, you can naturally extend into Grafana dashboards and alerting.

#prometheus#monitoring#node-exporter#metrics

Editorial note

AI tools may assist drafting. Check the evidence in each guide for whether commands were run and which environment was used. Confirm the current docs for your version before applying instructions, and email us about errors. email us

Topic-related official docs (auto-linked · not per-claim sources)Linux man pages ↗Prometheus documentation ↗

How we write and review →Editorial policy →

Questions & answers

Ask a question about this guide. We'll review and reply.

Prometheus + Node Exporter — Building Server Metrics Monitoring

What is Prometheus?

Architecture

1. Install Node Exporter

2. Install Prometheus

3. Configure prometheus.yml

4. Essential PromQL queries

CPU utilization (%)

Memory utilization (%)

Disk utilization (%) — root partition

Network receive traffic (bytes/s)

Disk-full prediction (risk if below 0 in 4 hours)

5. Cut query cost with recording rules

6. Retention policy and storage

Summary

Questions & answers