In my previous post I wrote about how to load test GKE Workload Identity. In this post I’ll describe how to get metrics from gke-metadata-server, the part of Workload Identity that runs on your GKE clusters’ nodes. This solution is a temporary workaround until GKE provides a better way to get metrics on gke-metadata-server.
Gke-metadata-server runs as a K8s DaemonSet. It exposes metrics about itself in Prometheus
text-based format. I want to have an external scraper make HTTP requests to periodically collect
these metrics. Unfortunately, the Prometheus HTTP server only listens on the Container’s
interface. So how can we expose these metrics, i.e. make the HTTP endpoint available externally?
tl;dr lessons learned
- If something you need is running on a computer you control, you can always find a way extract info from it if you’re resourceful enough.