Command to list my brokers
Hi all, I just want a way to query all of my brokers to see if they're all connected and online, without creating a topic. Or is creating a topic the best way to verify all my brokers are up and...
View ArticleQuery regarding serialization
Hi, I am new to kafka and still learning .I have a query .. As per my understanding the serialization is happening before the partitioning and grouping of messages per broker . Is my understanding...
View ArticleIn Apache Kafka, how can one achieve delay queue support (similar to what...
Hi Kafka Users Community, In Apache Kafka, how can one achieve delay queue support (similar to what ActiveMQ has)? Has anyone solved similar problem before? Thanks in advance? Regards,Vish
View Articlekafka shutdown automatically
Hi， our kafka cluster is shut down automatically today, here is the log. I don't find any error log. Anything wrong? [2015-01-18 05:01:01,788] INFO [BrokerChangeListener on Controller 0]: Broker change...
View Articlecan't iterate consumed messages when checking errorCode first
Hi all, I'm using Kafka low level consumer api and find in the below codes "iterator.hasNext" always return false. Through debugging, I'm sure the messageSet has the size of "fetchSize" * val consumer...
View ArticleIssue size message
Hi All, I am having an issue when using kafka with librdkafka. I've changed the message.max.bytes to 2MB in my server.properties config file, that is the size of my message, when I run the command line...
View ArticleJar files needed to run in Java environment (without Maven)
Hello All, I posted a question last week ( http://mail-archives.apache.org/mod_mbox/kafka-users/201501.mbox/browser) but I'm not able to respond to the thread for some reason? The suggestion on the...
View ArticleKafka Out of Memory error
Hi All, I have a kafka cluster setup which has 2 topics topic1 with 10 partitions topic2 with 1000 partitions. While, I am able to consume messages from topic1 just fine, I get following error from the...
View Article0.8.2.0 Candidate 1
Thanks for reporting the issues in RC1. I will prepare RC2 and start a new vote. Jun
View Articlejava.nio.channels.ClosedChannelException...Firewall Issue?
Hello Everyone, Thank you for the help! Preface: I've created producers/consumers before and they have worked. I have also made consumers/producers using java programs, but they have all been locally....
View Article0.8.2.0 Candidate 2
This is the second candidate for release of Apache Kafka 0.8.2.0. There has been some changes since the 0.8.2 beta release, especially in the new java producer api and jmx mbean names. It would be...
View ArticleHow to setup inter DC replication in Kafka 0.8.1.1
Hi, I want to setup inter DC replication b/w Kafka clusters. Is there any inbuilt tool to do this? I already have tried MirrorMaker tool but the problem is, if MM killed then some messages get...
View ArticleHow to handle broker disk failure
I'm trying to figure out the best way to handle a disk failure in a live environment. The obvious (and naive) solution is to decommission the broker and let other brokers taker over and create new...
View Articletypo in wiki
In the wiki - there is a statement that a partition must fit on a single machine, while technically true, isn't it so that a partition must fit on a single disk on that machine....
View ArticleKafka-Storm Integration
Hi, I am runnning a Kafka-storm topology . The topology works exactly as expected in LocalCluster mode. However in Distributed mode the Workers are not starting , supervisor has not started. Only...
View ArticleUpdated Logstash-Kafka plugin
Hello Everyone, As we wait for the 1.5 release of Logstash which will include the Kafka plugin by default I've put some work into the logstash-kafka plugin which makes it much easier to install and...
View ArticleHow to run Kafka Producer in Java environment? How to set mainClass in pom...
Hello Everyone, Sorry for the duplicate post, I hadn't subscribed to the list yet, so I couldn't reply to suggestions. That has been fixed now. 1) Using Maven, I wrote a Kafka Producer similar to the...
View ArticlePoll: Producer/Consumer impl/language you use?
Hi, I was wondering which implementations/languages people use for their Kafka Producer/Consumers.... not everyone is using the Java APIs. So here's a 1-question poll:...
View ArticleCan I run Kafka Producer in BG and then run a program that outputs to the...
Hello Everyone, Sorry for asking multiple questions, but I am currently trying another approach to run a kafka producer. 1) I started the kafka console producer as mentioned here in the background...
View Article