ConsumerOffsetChecker not able to parse broker info
Hi all, I'm running Kafka 0.7 and I'm having problems running the ConsumerOffsetChecker. I have messages loaded in a topic called eventdata and I can consume the messages fine but when I try to run...
View ArticleUse cases where NOT to use Kafka
Do we know particular use cases or scenarios where Kafka shouldn't be use and why? Please share. Thanks, Menka "Don't let what you cannot do interfere with what you can do" - by John Wooden
View ArticleKafka authentication with Zookeeper question
Hi, Is there any support in Kafka for authentication with Zookeeper regarding the publication of brokers endpoints etc. to ZK? i.e. If we wanted to secure writes/reads of the /brokers node using e.g....
View ArticleKafka 0.8 High-Level Consumer: fetcher threads
Hi folks! I'm trying High-Level Consumer and wondering why it uses only 1 fetcher per broker? Why can't it be scaled on partition basis? I even see the parameter '/numFetchers/' in...
View Articlea consumer question
Hi folks, For this parameters, if consumer.timeout.ms is set to 0, whenever I call ConsumerIterator's hasNext(), if there is a message available, a timeout exception will be thrown. Is my understanding...
View ArticleLogs and distributed systems
For anyone that's interested, I did a blog post on logs, stream processing, and distributed systems:...
View ArticleConnection Timeouts
Hi All, I have a situation where one producer/consumer is causing timeout errors on the Kafka broker. The exception in the logs looks like this: [2013-12-16 17:32:25,992] ERROR Closing socket for...
View Articleredis versus zookeeper to track consumer offsets
I am leaning towards using redis to track consumer offsets etc., but I see how using zookeeper makes sense since it already part of the kafka infra. One thing which bothers me is, how are you guys...
View ArticleKafka 0.8.0 server-stop.sh does not stop broker
Hi, We have been trying out the kafka 0.8.0 beta1 for awhile and recently attempted to upgrade to 0.8.0 but noticed that the stop server script doesn't seem to stop the broker anymore. I noticed...
View ArticleKilling broker leader
Hi, I installed zookeeper and kafka 8.0 following the quick start ( https://kafka.apache.org/documentation.html#quickstart) and when i try to kill my leader, i got a lot of exception in my producer and...
View Articleregarding run-simulator.sh
i am not able to find run-simulator.sh in 0.8 even after building perf.if this tool has been deprecated what are other alternatives available now for perf testing? Regards, Pushkar
View ArticleCan we archive our kafka-requestlog in kafka0.8??
Hello All, I am using kafka 0.8. I want to archive my kafka-request.log, server.log, controller.log, staet-change.log. Is there any property in log4j by which i can archive my logs in zip folders or...
View Articlekafka build error scala 2.10
While doing dev setup as described in https://cwiki.apache.org/confluence/display/KAFKA/Developer+Setup im getting following build errors. immutable is already defined as class immutable...
View ArticleData loss in case of request.required.acks set to -1
Hi All, We are having kafka cluster of 2 nodes. (using 0.8.0 final release) Replication Factor: 2 Number of partitions: 2 I have configured request.required.acks in producer configuration to -1. As...
View ArticleWriting unit tests for Kafka code
I'm writing some Kafka client code and I'd like to unit test it. I've found some resources that say this is possible. For example: http://ransilberman.wordpress.com/2013/07/19/how-to-unit-test-kafka/...
View Articledoubt regarding the metadata.brokers.list parameter in producer properties
Hi, I am running kafka 0.8 and zoo keeper along with it. The problem i have is with the metadata.broker.list which is present in the producer properties. As i am using the zookeeper can i just ignore...
View Articleupgrade from beta1 to 0.81
Hi folks, As the tools in 0.8 are not stable and we don't want to take the risk. we want to skip 0.8 and upgrade from beta1 to 0.81 directly. So my question is whether we can do an in place upgrade and...
View Articleunderstanding OffsetOutOfRangeException's....
In our broker logs, we occasionally see errors like this: 2013-12-23 05:02:08,456 ERROR [kafka-request-handler-2] server.KafkaApis - [KafkaApi-45] Error when processing fetch request for partition...
View ArticleUnderstanding the min fetch rate metric
I'm realizing I'm not quite sure what the 'min fetch rate' metrics is indicating, for consumers. Can someone offer an explanation? Is it related to the 'max lag' metric? Jason
View Articlenode.js client library?
Hi, I wanted to reach out if folks are using https://github.com/cainus/Prozess for a node.js client library? Are there other node.js implementations folks are using or is that primarily it? Are there...
View Article