Quantcast
Channel: Kafka Timeline
Viewing all articles
Browse latest Browse all 1519

MirrorMaker consumer does not use broker.list property

$
0
0
Hi,

I'd like to bump this issue:
https://mail-archives.apache.org/mod_mbox/kafka-users/201212.mbox/%3CFA0E8A0482D176408729D604142248F319D22CA7%40EXCHANGE14.actuate.com%3E
as I'm encountering the same problem.

It seems that the MirrorMaker does the following:
1) if zk.connect is defined for the consumer, MirrorMaker script ignores
broker.list value
2) if zk.connect is not definied for the consumer, MirrorMaker fails with
an error message:

[2013-05-28 11:24:57,457] INFO group1_ip-<hostname> Connecting to zookeeper
instance at null (kafka.consumer.ZookeeperConsumerConnector)
[2013-05-28 11:24:57,457] INFO Initiating client connection,
connectString=null sessionTimeout=6000
watcher=org.I0Itec.zkclient.ZkClient [ at ] c820344(org.apache.zookeeper.ZooKeeper)
[2013-05-28 11:24:57,461] INFO Starting ZkClient event thread.
(org.I0Itec.zkclient.ZkEventThread)
Exception in thread "main" java.lang.NullPointerException
at org.apache.zookeeper.ClientCnxn.<init>(ClientCnxn.java:361)
at org.apache.zookeeper.ClientCnxn.<init>(ClientCnxn.java:332)
at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:383)
at org.I0Itec.zkclient.ZkConnection.connect(ZkConnection.java:64)
at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:872)
at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:98)
at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:84)
at
kafka.consumer.ZookeeperConsumerConnector.connectZk(ZookeeperConsumerConnector.scala:152)
at
kafka.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:122)
at
kafka.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:129)
at kafka.tools.MirrorMaker$$anonfun$3.apply(MirrorMaker.scala:102)
at kafka.tools.MirrorMaker$$anonfun$3.apply(MirrorMaker.scala:102)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206)
at
scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:61)
at scala.collection.immutable.List.foreach(List.scala:45)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:206)
at scala.collection.immutable.List.map(List.scala:45)
at kafka.tools.MirrorMaker$.main(MirrorMaker.scala:102)
at kafka.tools.MirrorMaker.main(MirrorMaker.scala)

The command I run is

$KAFKA_ROOT/bin/kafka-run-class.sh kafka.tools.MirrorMaker
--consumer.config ./mirror_consumer.properties --producer.config
./mirror_producer.properties --whitelist=".*" --num.streams 1

The ./mirror_consumer.properties file contains:

broker.list=1:localhost:59092,2:localhost:59093
groupid=group1
shallowiterator.enable=true

When broker.list is commented out and zk.connect is defined instead (or
when both are defined), mirroring happens successfully.

Any leads?

Thank you,
Aman

Viewing all articles
Browse latest Browse all 1519

Trending Articles