Clean up Consumer Group
Hi, It seems that my consumers cannot be shut down properly. I can still see many unused consumers on the portal. Is there a way to get rid of all these consumers? I tried to call shutdown explicitly,...
View Articlepython kafka and lots of errors.
I am looking on how to get kafka to work with python and sending messages. I am using python kafka to produce and consume messages. https://github.com/mumrah/kafka-python the messages that I am sending...
View ArticleConsumer group ID for high level consumer
Hi all, I am referring to this e.g: https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Group+Example. What is the consumer group ID being referred here ? Thanks Binita
View ArticleDelay in fetching messages - High level consumer -Kafka 0.8
Hi, I am using kafka 0.8. I have 3 brokers on three systems and 3 zookeepers running. I am using the high level consumer which is in examples folder of kafka. I am able to push the messages into the...
View Articlecalling ad-hoc topic creation from code
Hi there, I'm trying to create a topic from java code, by calling CreateTopicCommand: *ZkClient zkClient = new ZkClient(zookeeperCluster, 30000, 30000);* *CreateTopicCommand.createTopic(zkClient,...
View ArticleConsumer group ID for high level consumer
Hi all, I am referring to this e.g: https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Group+Example. What is the consumer group ID being referred here ? Thanks Binita
View ArticleConsumerRebalanceFailedException
Hi all, I tried to reproduce this exception. In case one, when no broker was running, I launched all consumers and got this exception. In case two, while the consumers and brokers were running, I...
View ArticleHow ot had scala libraries to path
Hi, This is how I start kafka. command = /var/lib/kafka-<%=@version%>-src/bin/kafka-server-start.sh /var/lib/kafka-<%=@version%>-src/config/server.properties In another application I get...
View Articlememory requirement for kafka - how to estimate
Hi, Can i know what all should i consider while calculating the memory requirements for the kafka broker. I had tried to search on net about this, but doulc not find nay. If anyone can please suggest...
View ArticlePuppet module for deploying Kafka released
Hi everyone, I have released a Puppet module to deploy Kafka 0.8 in case anyone is interested. The module uses Puppet parameterized classes and as such decouples code (Puppet manifests) from...
View ArticleKafkaStream hasNext() blocking and I'm not sure why
Hi all, I have created 6 Kafka topics each with 2 partitions, and now I want to consume from all partitions in a Java app. I have created a Executors.newScheduledThreadPool(12) for this, and then...
View ArticleProducer fails when old brokers are replaced by new
Hi all, I ran into a problem with the Kafka producer when attempting to replace all the nodes in a 0.8.0 Beta1 Release Kafka cluster, with 0.8.0 Release nodes. I started a producer/consumer test...
View ArticleStrange issue with Camel Kafka integration
Hi I am trying to integrate Camel with Kafka. I have implemented following route. *<bean id="publisherPoint" class="org.vaida.esb.camel.orderdispatcher.OrderDispatcher.PublisherPoint">...
View ArticleUnable to consume Snappy compressed messages with Simple Consumer
Publisher (using librdkafka C api) has sent both gzip and snappy compressed messages. I find that the java Simple Consumer (...
View ArticleProblems consuming snappy compressed messages via SimpleConsumer
Publisher (using librdkafka C api) has sent both gzip and snappy compressed messages. I find that the java Simple Consumer (...
View ArticleReg Partition and Replica?
Hi, I have a doubt regarding to Partition and replica. What is the difference between them? I created a topic "test1" with 5 partition and 3 replicas. I am sending a message to the topic "test1". I see...
View ArticleKafka 0.8: delete topic with data
Hi, I'm using Kafka 0.8 which does not have a command to delete topic. However, I need the functionality and I'm trying to adopt this approach:...
View Articlecan't connect to kafka from a java client.
Goal: I am trying to send messages to kafka from a java cleint. And it has been a pain.. Let me describe in brief. 1. I have installed kafka on a centos VM. 2. I ran the zookeeper that comes with it,...
View ArticleDoes high level consumer support setting offset
Hi， In kafka 0.8, does high level consumer support setting offset? Our service reads kafka data but won't flush the data immediately, so if restarted the data in memory will be lost. We want to reset...
View ArticleHow does one measure performance of an existing Kafka cluster?
Let's say I have a running cluster and users/apps are pounding away at it. Is there a quick and easy way to measure its current throughput? I know there are utilities for generating volume to get...
View Article
