Strimzi — Architecture, Reconciliation, CRD Landscape

1. Architecture

A Strimzi deployment is three operator processes plus a set of CRDs they watch. The Cluster Operator is the central one: it watches every Kafka and KafkaNodePool in its scoped namespaces and, on each change, reconciles the cluster into a set of StrimziPodSet resources (one per node role — controllers, brokers, mixed) plus Services, ConfigMaps, and TLS Secrets. The Topic Operator watches KafkaTopic CRs and creates / updates / deletes topics inside the target cluster via Kafka's Admin API. The User Operator does the same for KafkaUser — credentials and ACLs. CruiseControl is optional; when enabled by the Cluster Operator, it is reconciled as its own Deployment + Service (not a sidecar on the broker pods) that the operator calls when a KafkaRebalance CR is created.

Blue arrows: Kubernetes watches by the operators on their respective CRs. Green arrows: resources reconciled into the target namespace. Each operator runs as its own Deployment; the Cluster Operator co-locates the Topic and User Operator inside an EntityOperator pod by default, but they can also run stand-alone.

2. Reconciliation

A typical create a new cluster reconciliation, with KRaft mode and the EntityOperator co-located. The Cluster Operator's reconciliation loop runs every STRIMZI_FULL_RECONCILIATION_INTERVAL_MS (default 2 min) and is also kicked on every CR change.

The reconciliation loop is idempotent and level-triggered: re-running it always converges to the desired state, so a missed event or a transient API failure is recovered on the next tick. The same loop also drives rolling updates — for example, certificate renewal generates new secrets in step 3–4 and then rolls pods one-by-one through step 5–8, gated on broker readiness probes.

3. CRD Landscape

• Kafka — top-level cluster CR. Declares the listeners (plain, TLS, external load balancer / nodeport / ingress / route), authentication / authorization, EntityOperator, CruiseControl, JMX exporter, and (legacy) ZooKeeper or KRaft topology. In modern Strimzi, the node topology lives in KafkaNodePools referenced from this CR.
• KafkaNodePool — one per role group. Holds replicas, storage class / size, JVM options, resources, tolerations / affinity, and the role flag (controller, broker, or mixed). The Cluster Operator turns each pool into one StrimziPodSet and one StatefulSet-equivalent set of pods.
• KafkaTopic — managed by the Topic Operator. Declares partitions, replicas, and topic configs. Modern Strimzi uses the Unidirectional Topic Operator: the CR is the source of truth (CR change → CreateTopics / AlterConfigs) and the CR's status reflects the actual topic state; broker-side topic mutations are not propagated back into spec.
• KafkaUser — managed by the User Operator. Declares authentication type (TLS, SCRAM-SHA-512) and ACLs / quotas. The operator creates a Secret with the user's credentials and registers ACLs via AdminClient.
• KafkaConnect / KafkaConnector — a Kafka Connect cluster Deployment with a REST API. Connector instances can be created either declaratively via KafkaConnector CRs or through the Connect REST API directly.
• KafkaMirrorMaker2 — a managed MirrorMaker 2.0 Deployment for cross-cluster replication. Source / target cluster aliases, replication policies, and connector configs go here.
• KafkaBridge — HTTP REST front-end to a Kafka cluster (producer / consumer over HTTP). Useful when a client cannot speak the Kafka wire protocol directly.
• KafkaRebalance — kicks Cruise Control into computing and (optionally) executing a rebalance proposal. The CR's status reports proposal load metrics; the user transitions it to approve to execute.

4. Strimzi vs Apache Kafka

Strimzi does not replace Apache Kafka — it runs Apache Kafka. The two sit on different layers: Apache Kafka is the data plane (broker / controller processes that own the log and serve produce / fetch), and Strimzi is the Kubernetes-native control plane that creates those processes, manages their TLS material, advances their version, and reacts to declarative CRs. A Strimzi-managed cluster is the exact same Kafka binary you would run by hand — wrapped in operator-driven lifecycle.

	Apache Kafka (bare)	Strimzi on Kubernetes
Layer / role	Data plane — the broker / KRaft controller processes themselves.	Control plane — Kubernetes operator that reconciles those processes from CRs. Runs the same broker binary inside the pods it creates.
Deployment unit	JVM process on bare metal / VM / container. Started with kafka-server-start.sh.	StrimziPodSet → Pods with PVCs + Services + ConfigMaps + Secrets, all materialised by the Cluster Operator.
Configuration model	server.properties / shell scripts. Imperative; usually wrapped in Ansible / Chef / Helm.	Declarative CRs (Kafka, KafkaNodePool, KafkaTopic, KafkaUser, ...). CR change → reconcile.
Scaling brokers	Edit configs, start a new broker process, run kafka-reassign-partitions.sh by hand.	Change KafkaNodePool.spec.replicas. Operator creates the pod / PVC / Service; KafkaRebalance + Cruise Control handles partition movement.
TLS / certificates	Operator team generates keystores / truststores out-of-band and distributes them.	ClusterCa + ClientsCa objects auto-generate, rotate, and mount per-broker secrets. Renewal is rolling and automatic.
Users / ACLs	kafka-acls.sh or AdminClient calls, managed manually.	KafkaUser CR. User Operator creates the credential Secret and registers ACLs via AdminClient.
Topics	kafka-topics.sh or AdminClient, managed manually.	KafkaTopic CR. The Unidirectional Topic Operator drives the topic from the CR; status reflects the actual topic state.
Metrics / observability	JMX + (optional) JMX Prometheus exporter sidecar. ServiceMonitor wired by hand.	JmxPrometheusExporter / StrimziMetricsReporter / KafkaExporter wired by the operator. PodMonitor / Grafana dashboards shipped.
Rolling upgrades	Manual broker-by-broker restart. Operator team tracks inter-broker protocol bump steps.	KafkaRoller orchestrates the rolling restart with quorum awareness; Kafka.spec.kafka.version bump steps through inter-broker protocol upgrade automatically.
Rebalancing	Cruise Control is a separate process you start and call yourself.	Cruise Control is one boolean in the Kafka CR; rebalance is a KafkaRebalance CR whose status surfaces the proposal.
Mirroring / bridges / Connect	Each is its own daemon / Connect cluster you bring up and configure.	KafkaMirrorMaker2, KafkaBridge, KafkaConnect CRs each create their own managed Deployment.
Failure recovery	Whoever is on-call restarts brokers, restores PVs, re-issues certs.	Operator reconciliation loop is level-triggered: lost pod → recreated, lost PVC → reattached, expired cert → rolled. The on-call read of Kafka.status usually replaces the playbook.

The short version: if you already run Apache Kafka well by hand on Kubernetes, Strimzi is what the operator pattern would have you build anyway. If you do not run Kubernetes at all, Strimzi is the wrong tool — the value is the controller loop, not the broker binary.

5. CPU / Memory / Network at Runtime

The Strimzi operator pods themselves are cheap — a typical install is three small JVMs that spend almost all their time blocked on Kubernetes Watch events. The heavy runtime work lives in the Kafka cluster Strimzi manages (covered in Apache Kafka — Memory / CPU / I/O Pipeline). What follows is the operator-side picture only: where the operator pods spend CPU, what they hold in heap, and which network connections they keep open.

CPU / threading model (operator pods)

• Vert.x event loop + worker executor — each operator (cluster, topic, user) is a Vert.x application. The event loop handles watch events and scheduling; the actual reconcile work — cert generation, Kafka AdminClient calls, the bulk of the cluster reconcile — runs on the Vert.x worker executor via executeBlocking. The event loop itself is rarely the bottleneck.
• Reconcile is mostly waiting — a reconcile loop wakes on a Kube Watch event (CR change, pod status change, Secret change), diffs desired vs actual, and emits Kube API calls. Most of the wall-clock time is spent waiting on those Kube API responses, not on CPU. A namespace with 5 Kafka clusters and no churn typically shows the cluster-operator CPU well under 5%.
• Burst: certificate generation — every Cluster CA rotation, every Clients CA rotation, every new broker join is an RSA/ECDSA signing operation. certificate-manager/ uses Bouncy Castle on the worker pool; this is the most CPU-heavy operator activity and shows up as short spikes around CA renewal windows.
• Burst: rolling restart orchestration — KafkaRoller serialises pod restarts in a KRaft-aware order (unready controllers, then ready controllers with the active controller last, then unready brokers, then ready brokers), guarded by quorum and ISR safety checks (canRoll), and only moves on after each pod reaches Ready and rejoins the ISR. Wall-clock for a full roll is dominated by Kafka startup time per node, not by operator CPU; but the operator does hold the orchestration thread for the duration.
• Concurrency knob — STRIMZI_FULL_RECONCILIATION_INTERVAL_MS sets the periodic full-resync floor (default 120 s). Event-driven reconciles run as soon as a Watch event arrives; the periodic full resync exists only as a safety net for missed events.

Memory layout (operator pods)

• JVM heap (several hundred MB typical) — the default Cluster Operator deployment sets -Xmx in the hundreds of MB; installations range from ~256 MB on a single-cluster deploy up to roughly 1 GB on operators managing many Kafka CRs.
• Informer cache (fabric8 client) — the operator opens one Watch per CR kind (Kafka, KafkaNodePool, KafkaTopic, KafkaUser, KafkaConnect, KafkaMirrorMaker2, KafkaBridge, KafkaRebalance, plus the secondary resources it owns: StrimziPodSet, Service, ConfigMap, Secret, Pod). Each Watch backs a local cache of every object the operator cares about. This is the single largest heap consumer and scales linearly with the number of managed clusters.
• Per-reconcile working set — during a reconcile, the operator materialises the full desired state for that one cluster (the generated StrimziPodSet, ConfigMaps, Secrets, RBAC). Allocated, diffed, applied, dropped. Visible as periodic young-gen activity, not as long-lived heap growth.
• Topic / User Operator — even smaller. Topic Operator's main state is a local (KafkaTopic CR ↔ topic-in-cluster) map sized by topic count. User Operator's main state is a (KafkaUser CR ↔ secret + ACLs) map sized by user count.
• Kafka pods are separate — every memory-sized warning in apache-kafka.html#runtime (page cache dominance, JVM heap kept small relative to RAM) applies to the broker pods Strimzi creates, not to the operator pods themselves.

Network traffic

• Watch streams to kube-apiserver — one long-lived HTTPS connection (server-sent stream) per CR kind being watched. Quiet most of the time; emits an event whenever a watched object changes. This is how the operator hears about new CRs, pod status flips, Secret rotations.
• Kube API writes — every reconcile result is a burst of PATCH / UPDATE / CREATE / DELETE calls back to kube-apiserver over the same TLS connection. Counts and sizes are dominated by Secret writes (CA rotation) and StrimziPodSet diffs (rolling restart).
• Kafka AdminClient — Topic Operator and User Operator each keep one AdminClient per managed cluster, talking mTLS to the brokers (certs come from the cluster's Clients CA). Used to CreateTopics / DeleteTopics / AlterConfigs on topic-operator side and to DescribeAcls / CreateAcls on user-operator side. Idle connection pool; reuses TCP.
• Cluster Operator → broker readiness (kafka-agent) — each broker pod runs kafka-agent (small Java agent baked into the image) that serves broker-state / KRaft quorum state on an HTTPS endpoint (port 8443, /v1/broker-state, mTLS with the Cluster CA). The operator calls it directly via java.net.http.HttpClient; this is one of the signals — alongside Kube Pod readiness and probes — that KafkaRoller uses between pod restarts.
• Cluster Operator → Cruise Control — when Cruise Control is enabled, the operator opens HTTPS calls to the Cruise Control Service (a separate Deployment) for every KafkaRebalance CR (proposal request, status polling, approval, abort). Trust is established from the Cluster CA cert Secret (see PR #12885).
• What the operator does not touch — it never proxies client traffic. Producers, consumers, MirrorMaker2 instances, Connect workers all talk to the Kafka cluster directly; the operator sits beside them, not in front of them.

The operator's runtime cost is dominated by the informer cache (heap, scales with number of managed clusters) and by transient CPU bursts during CA rotation / rolling restart. The interesting performance picture — page cache, sendfile, ISR replication — lives in the broker pods, not here. The operator never sits on a hot data path.

6. Source Tree (for contributors)

Official source: github.com/strimzi/strimzi-kafka-operator. Strimzi is a CNCF graduated project. Build is Maven (pom.xml at the root, plus a top-level Makefile that wraps Maven + Docker image builds + system tests); JDK 17+; contribution flow is fork → branch off main → PR with the shared Strimzi PR template. Governance documents (GOVERNANCE.md, MAINTAINERS, AI_POLICY.md) live in the sibling strimzi/governance repo.

Three flows, top to bottom. Codegen (green) regenerates CRD YAML and the broker-config JSON schema at mvn package time — touching api/ means the resulting CRD changes too. Runtime (blue) is what runs in the operator pods: the three operators consume the same api/ + operator-common/ + certificate-manager/ trio and emit Kube + Kafka side-effects. Image build (amber) bundles the operators with the per-pod Java agents into the published quay.io/strimzi/* images. Tests live alongside (mockkube/ for unit, systemtest/ for E2E).

Top-level modules

• cluster-operator/ — the main reconciler. Watches Kafka, KafkaNodePool, KafkaConnect, KafkaMirrorMaker2, KafkaBridge, KafkaRebalance CRs. Builds StrimziPodSet + Services + ConfigMaps + Secrets. Owns the cert lifecycle (ClusterCa, ClientsCa), the rolling-restart orchestration (KafkaRoller), and the Cruise Control client. The largest module by far.
• topic-operator/ — reconciles KafkaTopic CRs against the actual Kafka cluster via AdminClient. Bidirectional sync (CR → cluster on user changes, cluster → CR on out-of-band creates).
• user-operator/ — reconciles KafkaUser CRs. Creates the credential Secret (per authentication type) and registers ACLs via AdminClient.
• api/ — the CRD model classes (Java POJOs) consumed by every operator. Kafka.java, KafkaSpec.java, KafkaTopicSpec.java, etc. Adding a new field to a CR starts here.
• certificate-manager/ — Cluster CA / Clients CA generation, signing, renewal, key rotation. Decoupled from cluster-operator/ so it can be unit-tested without a live Kube.
• crd-annotations/ + crd-generator/ — an annotation processor and code generator. Annotated POJOs in api/ are turned into the published CRD YAMLs under install/ at build time.
• config-model/ + config-model-generator/ — parse the Apache Kafka config documentation and generate a JSON schema for valid broker / Connect / Streams config keys. Lets Strimzi validate Kafka.spec.kafka.config at admission time.
• operator-common/ — shared reconciliation abstractions: the AbstractOperator base, fabric8 client wrappers, result types. Imported by all three operators.
• kafka-agent/, kafka-init/, tracing-agent/ — small Java agents and init containers baked into the operator's container images. kafka-agent exposes broker readiness / KRaft quorum state to the operator; kafka-init wires per-pod identity at start-up; tracing-agent injects OpenTelemetry into Kafka components.
• install/ — the canonical install YAML (CRDs + RBAC + Deployment). Generated; bumping the bundle here is part of every release.
• helm-charts/ — Helm chart for the operator. Mirrors install/ but parameterised.
• docker-images/ — Dockerfile per component. Built via the top-level Makefile + Makefile.docker.
• packaging/ — release-bundle assembly. Produces the .tar.gz and the published artifacts.
• systemtest/ — end-to-end tests that stand up a real Kube cluster (kind / minikube / OCP) and exercise the operators. Heaviest part of CI.
• mockkube/ — an in-memory fake Kubernetes API used by unit tests so reconciliation logic can be tested without spinning up a cluster.
• documentation/ — AsciiDoc source for the user-facing docs at strimzi.io/documentation.
• development-docs/ — contributor-facing docs (how to set up an env, how to run system tests, release process).
• examples/ — sample CRs (single-node KRaft cluster, dual-listener TLS, Connect with KafkaConnector, etc.). Doubles as smoke fixtures.
• test/, tools/ — shared test utilities and dev-only tools.
• kafka-versions.yaml — single source of truth for which Apache Kafka versions a given Strimzi release supports. Every Strimzi PR that bumps Kafka touches this file.

Version history

• 0.1 – 0.30 (2018 – 2022) — pre-1.0 era. ZooKeeper-only Kafka clusters. CRs and reconciler structure evolved heavily; API versions on the CRDs (v1alpha1, v1beta1, v1beta2) reflect those rewrites.
• 0.32 – 0.40 (2023) — KRaft support added (preview, then mostly-supported). KafkaNodePool CR introduced — first-class node roles instead of one monolithic StatefulSet.
• 0.41 – 0.46 (2024) — KRaft becomes the default for new clusters; ZooKeeper migration tooling matures. StrimziPodSet replaces StatefulSet as the underlying primitive for finer rolling-update control.
• 0.47 – 0.51 (late 2024 – early 2025) — ZooKeeper removed in lockstep with Apache Kafka 4.0. Cluster Operator strictly KRaft-only.
• 1.0.0 (May 2025) — API stability promise. CRDs promoted to v1. Strimzi commits to N-1 minor compatibility for CRs from this point on.
• 1.1.0 (latest at the time of writing) — continued polish on the 1.x API: tighter Cruise Control integration, faster rolling restarts on large clusters, improved observability defaults.

Branching & release model

main tracks the next minor. Each released minor gets a release-X.Y.x branch; bugfixes are committed to main and cherry-picked back. Release cadence is roughly one minor every 6 – 8 weeks during the pre-1.0 era, slower now post-1.0. Releases are tagged X.Y.Z-rcN for release candidates, then X.Y.Z after the maintainer vote. Container images are published to quay.io/strimzi/*; Helm chart artefacts to strimzi.io/charts/. The supported Apache Kafka matrix for a given Strimzi release is the source of truth in kafka-versions.yaml.

Related

• Sourcestrimzi/strimzi-kafka-operator
• Docsstrimzi.io/documentation
• My PRsStrimzi contributions
• SiblingApache Kafka — Architecture, Message Flow, CLI
• SiblingSpring Kafka — Architecture, Internals