Quantcast
Channel: Kafka Timeline
Viewing all articles
Browse latest Browse all 1519

Kafka Mirroring Issue

$
0
0
Hello All,

When i start the mirroring process, events are written only to partition
"1" on the target cluster and no data is written to partition 0.

I am stuck on this and wondering what am i doing wrong .

My production setup looks something like this
1. Datacenter1 : 3 kafka nodes/3 zookeeper nodes
2. Datacenter 2 : 3 kafka nodes/3 zookeeper nodes

All the kafka nodes are configured with auto.create.topics.enable=true and
the default partition is 2 .

I start the kafka mirroring tool with the following command

bin/kafka-run-class.sh kafka.tools.MirrorMaker --consumer.config
mirror-maker/consumer.properties --producer.config
mirror-maker/producer.properties --whitelist=".*" *--num.streams 2
--num.producers 2*

the consumer.properties for mirror-config looks like this
group.id=test-group
zookeeper.connect=<ZK_IP_LIST>

and producer.properties for mirror-config looks like this

metadata.broker.list=<KAFKA_IP_LIST>
producer.type=async
topic.metadata.refresh.interval.ms=-1

Regards

Viewing all articles
Browse latest Browse all 1519

Trending Articles