Admin Tabla de contenido
¿Qué son las particiones de Kafka?
Las particiones son distribuidos entre los servidores del cluster Kafka, donde cada servidor maneja los datos y las peticiones de una parte de los particiones. Las particiones se replican en un número configurable de servidores para asegurar la tolerancia a fallos.
¿Qué es Mirror Maker?
Mirror Maker es una herramienta incluida en Apache Kafka que nos permite mantener una réplica de los datos de Kafka en otro clúster.
¿Quién administra o sincroniza los clusters de Kafka?
Zookeeper: gestiona el cluster de Kafka, detecta errores, almacena secretos, mantiene la configuración, etc.
What is the replacement for mirrormaker?
I propose to replace MirrorMaker with a new multi-cluster, cross-data-center replication engine based on the Connect framework, MirrorMaker 2.0 (MM2). The new engine will be fundamentally different from the legacy MirrorMaker in many ways, but will provide a drop-in replacement for existing deployments.
How does mirrormaker2 work with Kafka?
Internally, MirrorMaker2 uses the Kafka Connect framework which in turn use the Kafka high level consumer to read data from Kafka. Kafka high level consumer coordinates such that the partitions being consumed in a consumer group are balanced across the group and any change in metadata triggers a consumer rebalance.
Why does mirrormaker only have one producer?
However with Kafka 0.9, MirrorMaker changed to having a single producer while the number of consumers was configurable. This design works better if MirrorMaker is situated in the same data center as the aggregate Kafka cluster. This way it consumes using many threads over the high latency connection and produces over the low latency connection.
How do I know if mirrormaker is working?
You use groupsBlacklistPattern if you want to use blacklists. The old way of checking that MirrorMaker was working, on Kubernetes at least, was by using standard Kubernetes healthcheck probes to know when MirrorMaker can accept traffic and when it needs a restart.