Quantcast
Channel: Kafka Timeline
↧

Command to list my brokers

Hi all, I just want a way to query all of my brokers to see if they're all connected and online, without creating a topic. Or is creating a topic the best way to verify all my brokers are up and...

View Article


Query regarding serialization

Hi, I am new to kafka and still learning .I have a query .. As per my understanding the serialization is happening before the partitioning and grouping of messages per broker . Is my understanding...

View Article


In Apache Kafka, how can one achieve delay queue support (similar to what...

Hi Kafka Users Community, In Apache Kafka, how can one achieve delay queue support (similar to what ActiveMQ has)? Has anyone solved similar problem before?  Thanks in advance? Regards,Vish

View Article

kafka shutdown automatically

Hi, our kafka cluster is shut down automatically today, here is the log. I don't find any error log. Anything wrong? [2015-01-18 05:01:01,788] INFO [BrokerChangeListener on Controller 0]: Broker change...

View Article

can't iterate consumed messages when checking errorCode first

Hi all, I'm using Kafka low level consumer api and find in the below codes "iterator.hasNext" always return false. Through debugging, I'm sure the messageSet has the size of "fetchSize" * val consumer...

View Article


Issue size message

Hi All, I am having an issue when using kafka with librdkafka. I've changed the message.max.bytes to 2MB in my server.properties config file, that is the size of my message, when I run the command line...

View Article

Jar files needed to run in Java environment (without Maven)

Hello All, I posted a question last week ( http://mail-archives.apache.org/mod_mbox/kafka-users/201501.mbox/browser) but I'm not able to respond to the thread for some reason? The suggestion on the...

View Article

Join

Please subscribe myself.

View Article


Kafka Out of Memory error

Hi All, I have a kafka cluster setup which has 2 topics topic1 with 10 partitions topic2 with 1000 partitions. While, I am able to consume messages from topic1 just fine, I get following error from the...

View Article


0.8.2.0 Candidate 1

Thanks for reporting the issues in RC1. I will prepare RC2 and start a new vote. Jun

View Article

java.nio.channels.ClosedChannelException...Firewall Issue?

Hello Everyone, Thank you for the help! Preface: I've created producers/consumers before and they have worked. I have also made consumers/producers using java programs, but they have all been locally....

View Article

0.8.2.0 Candidate 2

This is the second candidate for release of Apache Kafka 0.8.2.0. There has been some changes since the 0.8.2 beta release, especially in the new java producer api and jmx mbean names. It would be...

View Article

How to setup inter DC replication in Kafka 0.8.1.1

Hi, I want to setup inter DC replication b/w Kafka clusters. Is there any inbuilt tool to do this? I already have tried MirrorMaker tool but the problem is, if MM killed then some messages get...

View Article


How to handle broker disk failure

I'm trying to figure out the best way to handle a disk failure in a live environment. The obvious (and naive) solution is to decommission the broker and let other brokers taker over and create new...

View Article

typo in wiki

In the wiki - there is a statement that a partition must fit on a single machine, while technically true, isn't it so that a partition must fit on a single disk on that machine....

View Article


Kafka-Storm Integration

Hi, I am runnning a Kafka-storm topology . The topology works exactly as expected in LocalCluster mode. However in Distributed mode the Workers are not starting , supervisor has not started. Only...

View Article

Updated Logstash-Kafka plugin

Hello Everyone, As we wait for the 1.5 release of Logstash which will include the Kafka plugin by default I've put some work into the logstash-kafka plugin which makes it much easier to install and...

View Article


How to run Kafka Producer in Java environment? How to set mainClass in pom...

Hello Everyone, Sorry for the duplicate post, I hadn't subscribed to the list yet, so I couldn't reply to suggestions. That has been fixed now. 1) Using Maven, I wrote a Kafka Producer similar to the...

View Article

Poll: Producer/Consumer impl/language you use?

Hi, I was wondering which implementations/languages people use for their Kafka Producer/Consumers.... not everyone is using the Java APIs. So here's a 1-question poll:...

View Article

Can I run Kafka Producer in BG and then run a program that outputs to the...

Hello Everyone, Sorry for asking multiple questions, but I am currently trying another approach to run a kafka producer. 1) I started the kafka console producer as mentioned here in the background...

View Article