Most teams treat ConfigMaps as static config: bake it in at startup, mount it into a pod, move on. But the Kubernetes API has a built-in watch stream. Every ConfigMap creation, update, or deletion is an event pushed by the server — not fetched on a timer.
This article explores a small Python demo that turns that stream into a real-time pub/sub system. Multiple clients, each doing something different, all get notified the instant a ConfigMap changes — using nothing but Python's standard asyncio primitives.
Before You Start
You'll need:
- Python 3.11+ with
uvfor dependency management - A running Kubernetes cluster — orbstack, k3s, or Docker Desktop all work
kubectlconfigured and pointed at your cluster
The only non-stdlib dependency that matters for this article is kr8s — an async-first Kubernetes client for Python. It exposes the watch API as an async generator, which fits naturally into asyncio-based code. No threading, no blocking calls.
The full source is available at github.
The Problem with "Static Config"
The standard ConfigMap story goes like this:
Pod starts → reads ConfigMap → uses values → ConfigMap changes → nothing happensYour service is holding stale values. To pick up the change you either restart the pod or poll the API on a timer and hope you catch it quickly enough. Neither is good. Polling wastes resources and adds latency proportional to your poll interval. Pod restarts are disruptive and slow.
The Kubernetes API already has the answer built in. You just need to use it.
Watch, Don't Poll
The Kubernetes API server doesn't just serve requests — it can stream events. When you open a watch connection, it stays open and the server pushes you changes as they happen. This is exactly how kubectl get pods -w works under the hood.
Our demo sits between that stream and the rest of the application:
Kubernetes API
│
│ ADDED / MODIFIED / DELETED ← pushed by the server
▼
Watcher (one long-lived connection)
│
▼
Notifier (fan-out broker)
│
├──► Logger Client — logs every event
├──► Counter Client — counts total events seen
└──► Filter Client — reacts only to ADDEDOpening the Stream
kr8s makes the stream feel like a normal Python loop. You call .watch() with a label selector to scope which ConfigMaps you care about:
async for event, configmap in api.watch(ConfigMap.kind, label_selector="app=demo"):
# event → "ADDED", "MODIFIED", or "DELETED"
# configmap → full object with .name, .data, .labels,
...The label selector is important. Without it you'd receive events for every ConfigMap in the namespace — including ones created by system components you have no interest in. Scoping to app=demo means you only see ConfigMaps that explicitly opt in.
Keeping the Stream Alive
Long-lived HTTP connections drop. The watch stream will time out, get reset by the API server, or close on a network blip. The fix is to wrap the loop in a while True and reconnect on failure:
while True:
try:
async with ConfigMapManager() as manager:
async for event, configmap in manager.watch(selector="app=demo"):
await notifier.notify(
{
"event": event,
"name": configmap.name,
"data": configmap.data,
}
)
except httpx.ReadError:
logger.warning("Watch stream dropped, reconnecting...")
await asyncio.sleep(0.5)When the stream drops, the inner async for raises httpx.ReadError. The outer loop catches it, waits half a second, and opens a fresh connection. The clients waiting on their queues never notice — they just keep waiting.
Retry the entire generator on failure is a pattern worth keeping in your toolkit for any long-running watch loop.
The Fan-Out Problem
A single Kubernetes event needs to reach all clients at once. The mechanism for this is asyncio.Queue — each client subscribes to the Notifier and gets its own private queue. When an event arrives, notify() puts a copy onto every queue in one pass:
class Notifier:
def __init__(self):
self._subscribers: list[asyncio.Queue] = []
self._lock = asyncio.Lock()
async def subscribe(self) -> asyncio.Queue:
q = asyncio.Queue()
async with self._lock:
self._subscribers.append(q)
return q
async def unsubscribe(self, q) -> None:
async with self._lock:
try:
self._subscribers.remove(q)
except ValueError:
pass
async def notify(self, event: dict) -> None:
async with self._lock:
for q in self._subscribers:
await q.put(event)The asyncio.Lock is load-bearing. A client can subscribe or disconnect at any moment — including while notify() is in the middle of iterating the list. The lock ensures the subscriber list is never modified mid-iteration.
What this looks like on each event:
Kubernetes pushes MODIFIED
│
▼
notifier.notify(event) called
│
├──► Queue A ← event [Logger: was blocking → now unblocks]
├──► Queue B ← event [Counter: was blocking → now unblocks]
└──► Queue C ← event [Filter: was blocking → checks event type]All three run independently. A slow client processing its event doesn't delay the others.
Three Clients, Three Reactions
This is where the "state store" idea gets concrete. The same ConfigMap change can mean different things to different parts of your system. The demo makes this explicit with three clients running as concurrent asyncio tasks.
The Logger receives every event unconditionally:
while True:
event = await q.get()
log(f"[logger] {event['event']} → {event['name']}")The Counter maintains state across events — it doesn't care what changed, only how many times something changed:
count = 0
while True:
await q.get()
count += 1
log(f"[counter] total events: {count}")The Filter is selective — it only acts on ADDED and silently discards everything else:
while True:
event = await q.get()
if event["event"] == "ADDED":
log(f"[filter] new configmap: {event['name']}")Three behaviours. Three reactions. All triggered by the same single event delivery. In a real system these could be a cache invalidator, a metrics recorder, and a feature flag reloader — each maintaining its own view of cluster state, without knowing the others exist.
All three share one watch connection to Kubernetes. The fan-out is cheap. The connection is not.
Running It
Install and start:
uv sync
uv run python -m k8smapThe process opens the watch connection. All three clients block on their queues. Now open a second terminal and create a ConfigMap with the app=demo label:
kubectl apply -f - <<EOF
apiVersion: v1
kind: ConfigMap
metadata:
name: my-config
labels:
app: demo
data:
environment: staging
log_level: info
EOFAll three clients fire immediately:
[logger] ADDED -> my-config
[counter] total events: 1
[filter] new configmap detected: my-configUpdate it:
kubectl apply -f - <<EOF
apiVersion: v1
kind: ConfigMap
metadata:
name: my-config
labels:
app: demo
data:
environment: production
log_level: warning
EOFLogger and counter react. Filter stays silent.
[logger] MODIFIED -> my-config
[counter] total events: 2Delete it:
kubectl delete configmap my-config
[logger] DELETED -> my-config
[counter] total events: 3Three clients. One stream. Zero polling.
This Is a State Store
It's worth naming what's actually happening here.
A traditional state store — Redis, etcd, a relational database — lets you write a value and read it later. What ConfigMaps add, via the watch API, is the ability to subscribe to changes. You don't just read the current value; you get notified every time it moves.
That's a reactive state store. And Kubernetes ships one with every cluster.
The data field on a ConfigMap is the state. The watch stream is the change feed. The Notifier is the fan-out layer that connects the change feed to however many observers need it.
Real-world client behaviours that follow this exact shape:
- Reload in-memory config on
MODIFIED— live configuration without pod restart - Invalidate local cache on any event — cache stays consistent with cluster state
- Increment a counter on each event — soft metrics with no external metrics store
- Alert on unexpected
DELETED— configuration drift detection - Recompute routing table on
MODIFIED— dynamic service discovery
Each of these is a different while True: event = await q.get() loop. The machinery underneath is identical.
How This Compares to Redis and Kafka
If you're already thinking "I'd just use Redis pub/sub for this" — fair. Here's how they compare for this specific use case.
ConfigMap watch requires zero extra infrastructure. Access control comes from existing Kubernetes RBAC. State persists across restarts and is always inspectable with a single kubectl get configmap -l app=demo. There is nothing new to deploy, secure, or maintain.
Redis Pub/Sub is the right answer when you need sub-millisecond latency, high event frequency, or consumers outside the cluster. It doesn't persist state across restarts by default and requires you to manage your own access control.
Kafka is the right answer when you need durable event history, replay, or cross-datacenter fan-out. It's also Kafka — with everything that implies operationally.
For configuration propagation inside a cluster — which is low-frequency and high-importance — ConfigMaps are already the right tool. The watch API just makes them reactive.
Use this for lightweight, non-critical, low-frequency use cases only.
Why asyncio Is the Right Tool Here
The entire demo is single-threaded. No thread pool, no subprocess, no external concurrency primitives. Asyncio cooperative multitasking fits naturally because every actor in the system spends most of its time waiting:
- The watcher waits for the next event from the Kubernetes API
- Each client waits for the next item on its queue
While the watcher is blocked on the network, all three clients can run. While a client is processing, the watcher can receive the next event. They never interfere with each other.
kr8s is designed around this model. Its async generators don't spawn threads — they await the underlying HTTP connection and hand control back to the event loop between events. That's what makes the whole thing compose so cleanly.
The Pattern Under the Hood
Strip away Kubernetes and what you have is this:
long-lived push stream (K8s watch, WebSocket, gRPC stream, DB change feed...)
│
▼
reconnecting loop (wraps the generator, retries on drop)
│
▼
fan-out notifier (one asyncio.Queue per observer, asyncio.Lock for safety)
│
├──► observer A (does one thing)
├──► observer B (does something else)
└──► observer C (filters, may ignore)This works wherever you have a push stream and want to fan it out to multiple concurrent consumers — without a message broker. The specific tools — kr8s, asyncio.Queue, asyncio.Lock — are Python idioms. The pattern is not.
~150 lines of Python. Zero extra infrastructure. If you're already on Kubernetes and need lightweight real-time config propagation, the stream is already there — you just have to listen.
The full source is available at github.
Thanks for taking time to read my article :)