Skip to main content

6 · Mimir deployment detail — Kubernetes

namespace: mimir · helm: mimir-distributed · ring: memberlist gossip :7946

Write pathingest · flushRead pathquery · fetchBackground · storagecompaction · rules · bucketsdashed = traffic · animated namespace: mimir · helm: mimir-distributedgateway (nginx)Deployment ×2 · one Service for clients/api/v1/push/prometheus/*WRITEdistributorDeployment ×3+ · HPA on CPUingesterStatefulSet ×3 per zonePVC WALREADquery-frontendDeployment ×2 · statelessquery-schedulerDeployment ×2 · per-tenant fair queuequerierDeployment ×2+ · HPA on queue depthstore-gatewayStatefulSet ×1 per zone · PVC index cacheBACKENDcompactorStatefulSet ×1 · PVC scratch diskrulerDeployment ×2alertmanagerStatefulSet ×3 · gossip HA · PVC silencesmemcached ×4chunks · index · results · metadatahash ring · memberlist gossip :7946 — no etcd/consul · zone-aware RF=3

Components​

gateway (nginx)​

Single entry Service for all Mimir traffic. Routes write and read paths to the right internal Services so clients only need one endpoint.

  • Deployment ×2 behind a ClusterIP/Ingress
  • /api/v1/push → distributor Service
  • /prometheus/* → query-frontend Service
  • Can enforce auth + inject X-Scope-OrgID at the edge
protocolin :80/:443 · out :8080 internal Services

distributor (K8s)​

Stateless Deployment — the easiest component to autoscale. Sized by ingest throughput.

  • Deployment, 3+ replicas · HPA on CPU
  • No volume, no identity — safe to kill anytime
  • PodDisruptionBudget: keep majority up during node drains
protocolService: mimir-distributor:8080

ingester (K8s)​

The stateful heart. Zone-aware StatefulSets — one per availability zone — so RF=3 lands one replica per zone and a zone outage loses nothing.

  • 3 StatefulSets: ingester-zone-a/b/c · pod anti-affinity
  • PVC per pod for WAL + head blocks (fast SSD class)
  • Scale up: fine. Scale down: needs careful ring 'leaving' + flush
  • Rolling updates one zone at a time (rollout-operator)
protocolService: mimir-ingester:9095 (gRPC)
tenancymemory limited per tenant (max series)

query-frontend (K8s)​

Stateless Deployment fronting the read path; pairs with query-scheduler so queriers pull work instead of being pushed.

  • Deployment ×2 · behind mimir-query-frontend Service
  • Results cache → memcached-results
protocolService: mimir-query-frontend:8080

query-scheduler (K8s)​

Optional but recommended: decouples the queue from query-frontends so both scale independently.

  • Deployment ×2
  • Per-tenant fair scheduling; queriers connect and pull jobs
protocolgRPC :9095

querier (K8s)​

Stateless PromQL workers. The component to autoscale for read-heavy clusters.

  • Deployment · HPA on scheduler queue depth or CPU
  • No volumes; scale 2 → 20 in seconds under dashboard load
protocolpulls from query-scheduler

store-gateway (K8s)​

Zone-aware StatefulSets like ingesters — blocks are sharded + replicated across zones for query availability.

  • StatefulSet per zone · PVC caches lazy-loaded index headers
  • Restart is cheap-ish but re-syncs block index → PVC keeps it warm
protocolgRPC :9095 · reads mimir-blocks bucket

compactor (K8s)​

Singleton-style StatefulSet with a scratch PVC for downloading + merging blocks. Not on the query or write path — safe to restart.

  • StatefulSet ×1 (shardable per tenant at scale)
  • PVC scratch space ≈ largest tenant's block set
  • Falling behind → slow queries (too many small blocks)
protocolreads/writes mimir-blocks bucket

ruler (K8s)​

Deployment evaluating rule groups sharded across replicas via the ring.

  • Deployment ×2 · rules in object storage per tenant
  • Remote-evaluation mode: sends PromQL to query-frontend
protocolnotify → alertmanager :8080

alertmanager (K8s)​

Mimir's multi-tenant Alertmanager: StatefulSet ×3 with gossip replication so silences and notification state survive pod loss.

  • StatefulSet ×3 · PVC for silences + nflog
  • Per-tenant Alertmanager configs stored in the bucket
protocol:8080 UI/API · gossip :9094
tenancyisolated config + routing per tenant

memcached caches​

Four separate cache pools so one hot cache can't evict another's entries. The single biggest read-latency win.

  • chunks-cache: raw chunk data (biggest, GBs)
  • index-cache: block index lookups
  • results-cache: whole PromQL responses (query-frontend)
  • metadata-cache: bucket listings + block metas
protocolmemcached :11211 · one StatefulSet each

hash ring · memberlist​

How components find each other and shard work: every pod gossips ring membership over memberlist — no etcd or Consul to operate.

  • Gossip on :7946 between all ring members
  • Distributors use it to pick 3 ingesters per series (zone-aware)
  • Store-gateways/compactors/rulers shard blocks + rules via the same mechanism
  • Pod dies → ring notices via gossip → traffic reroutes
protocolmemberlist gossip :7946