Consuming "backwards"?
Hi, Does Kafka offer a way to consume messages in batches, but "from the end"? This would be valuable to have in all systems where the most recent data is a lot more important than older data, such as...
View ArticlePartition reassignment stucked
Hi, We are trying to add a broker to a 10 node cluster. We have 7 different topics, each of them is divided in 10 partitions, and their replication factor is 3. To send traffic to this new node, we...
View ArticleUsing Vagrant with Apache Kafka 0.8.0
Hi all, I have been using Vagrant <http://www.vagrantup.com/> a lot recently and had the need/desire to build an Apache Kafka cluster with Vagrant for some recent client work. Vagrant allows you...
View ArticleKafka Consumer does not receive any message after a while
Hi, I have a problem in fetching messages from Kafka. I am using simple consumer API in Java to fetch messages from kafka ( the same one which is stated in Kafka introduction example). The problem is...
View ArticleKafka Consumer does not receive any message after a while
Hi, I have a problem in fetching messages from Kafka. I am using simple consumer API in Java to fetch messages from kafka ( the same one which is stated in Kafka introduction example). The problem is...
View ArticleAbout speed contorl in Kafka
Hi, In my application, the produce speed could be very high at some specific time in a day while return to a low speed at the rest of time. Frequently, my data logs are flushed away before they are...
View ArticlezkClient version with Kafka 0.8 & scala-library 2.8.0
Hi which dependency version should i use with kafka 0.8 , 0.1 0.2 or 0.3 ? <groupId>com.101tec</groupId> <artifactId>zkclient</artifactId> Also, is this groupID correct or...
View ArticlezkClient dependency version with kafka 0.8
Hi Which dependency version of zkClient , is recommended to be used with kafka 0.8 , scala-library 2.8.0 ? I see 2 dependencies for zkClient, 1.
View Articlewhy log files are never deleted?
Hi I just wonder why the log files, in {kafka_path}/log , are not deleted automatically? Is there any way to purge those files? Also is there anyway to purge the Kafka queue (make it empty) without...
View ArticleEmbedding a Kafka 0.8 broker for testing
Hi, I would like to embed a Kafka broker in my Java program, for testing purposes. The idea is being capable of running a Junit test that starts a local kafka broker, and then perform tests on some...
View Articlekafka_2.8.0-0.8.0 maven dependency
Hi All, I am upgrading my system from kafka-0.8.0 beta release to kafka 0.8.0 final release. I am using below maven dependency for kafka 0.8 final release: <dependency>...
View ArticleTopic not created
Hi All, We are having kafka cluster of 2 nodes. (Using Kafka 0.8.0 version) Replication Factor: 2 Number of partitions: 2 If any one node goes down then topic is not created in kafka. Since we are...
View ArticleTopic not created
Hi All, We are having kafka cluster of 2 nodes. (Using Kafka 0.8.0 version) Replication Factor: 2 Number of partitions: 2 If any one node goes down then topic is not created in kafka. Since we are...
View Articleproblem with high-level consumer stream filter regex....
All, I've filed: https://issues.apache.org/jira/browse/KAFKA-1180 We are needing to create a stream selector that essentially combines the logic of the BlackList and WhiteList classes. That is, we want...
View ArticleScala 2.10.1 - Issues with starting zookeeper
Hi, I have downloaded kafka-0.8.0-src.tgz<https://www.apache.org/dyn/closer.cgi?path=/kafka/0.8.0/kafka-0.8.0-src.tgz>, and executed the sbt update, sbt package "++ 2.10.1 package", and the...
View Articlepartition reassignment stuck on 0.8
I was running a 2-node kafka cluster off github trunnk at: eedbea6526986783257ad0e025c451a8ee3d9095 ...for a few weeks with no issues. I recently downloaded the 0.8 stable version, configured/started...
View ArticleHow to monitor consumer connection?
Hi, In kafka 0.7.2, we use 2 brokers and 12 consumers. I have 2 questions: 1. seems if one broker is restarted then all consumers need restart to reconnect to kafka. right? 2. How to monitor whether...
View Articlecluster expansion
Hi folks, There are three brokers running 0.8-beta1 in our cluster currently. Assume all the topics have six partitions. I am going to add another three brokers to the cluster and upgrade all of them...
View ArticleKafka 8.0 performance
Hi All, we are testing performance of our kafka 0.8 release cluster: - 5 kafka nodes with one zookeeper instance on each machine - FreeBSD 9.2 - Nagle off (sysctl net.inet.tcp.delayed_ack=0) - all...
View ArticleUnable to start consumers in Tomcat
Hi , Can you guys lets us know why are we getting this error when try to spawn a consumer ?. ZookeeperConsumerConnector can create message streams at most once Balaji
View Article