Introduction Pushing Metrics

https://github.com/prometheus/pushgateway
Diagram
Sometimes metrics cannot be scraped
- Example:
  - batch jobs, servers are not reachable due to NAT, firewall
Pushgateway is used as an intermediary service which allows you to push metrics.
Pitfalls
- Most of the times this is a single instance so this results in a SPOF
- Prometheus's automatic instance health monitoring is not possible
- The Pushgateway never forgets the metrics unless they are deleted via the api
  - Example:
Only 1 valid use case for the Pushgateway
- Service-level batch jobs and not related to a specific machine
If NAT or/both firewall is blocking you from using the pull mechanism
- Move the Prometheus server on the same network

Pushing Metrics - Python Example

Python example:
Pushgateway functions take a grouping key.
- push_to_gateway replaces metrics with the same grouping key
- pushadd_to_gateway only replaces metrics with the same name and grouping key
- delete_from_gateway deletes metrics with the given job and grouping key.