Push-Gateway

Prometheus Pushgateway is a component of the Prometheus monitoring system, which is used to collect and store time-series data. The Pushgateway allows for the collection of metrics from short-lived jobs or batch processes that cannot be instrumented directly with Prometheus.

In Prometheus, data is typically collected from services that expose metrics in a pull model, where Prometheus periodically scrapes metrics endpoints exposed by the services. However, there are scenarios where it may not be feasible or practical to instrument every job or process, especially those that are short-lived or run on demand.

This is where the Pushgateway comes in. The Pushgateway allows for these short-lived jobs or batch processes to push metrics data to the gateway, which then stores the metrics until they can be scraped by Prometheus. This enables monitoring of ephemeral jobs and allows for monitoring of batch jobs and ad hoc scripts that may not be running continuously.

Install and Run Pushgateway

Get the latest version of pushgateway from prometheus.io, then download and extract

$ wget https://github.com/prometheus/pushgateway/releases/download/v0.8.0/pushgateway-0.8.0.linux-amd64.tar.gz

$ tar -xvf pushgateway-0.8.0.linux-amd64.tar.gz

Create the pushgateway user

$ useradd --no-create-home --shell /bin/false pushgateway

Move the binary in place and update the permissions to the user that we created:

$ cp pushgateway-0.8.0.linux-amd64/pushgateway /usr/local/bin/pushgateway

chown pushgateway:pushgateway /usr/local/bin/pushgateway

Create the systemd unit file

$ cat > /etc/systemd/system/pushgateway.service << EOF
[Unit]
Description=Prometheus Pushgateway
Wants=network-online.target
After=network-online.target
[Service]
User=pushgateway
Group=pushgateway
Type=simple
ExecStart=/usr/local/bin/pushgateway
[Install]
WantedBy=multi-user.target
EOF

Reload systemd and restart the pushgateway service:

$ systemctl daemon-reload

$ systemctl restart pushgateway

Ensure that pushgateway has been started

$ systemctl status pushgateway

Configure Prometheus

global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'prometheus'
    scrape_interval: 5s
    static_configs:
      - targets: ['localhost:9090']

  - job_name: 'node_exporter'
    scrape_interval: 5s
    static_configs:
      - targets: ['localhost:9100']

  - job_name: 'pushgateway'
    honor_labels: true
    static_configs:
      - targets: ['localhost:9091']

$ systemctl restart prometheus