Leader doesn't get assigned for new topics
We are seeing a problem that we we try to send messages to new topic it fails kafka.common.LeaderNotAvailableException. But usually this problem will be transient and if we re-send messages to same...
View ArticleKafka 0.8 - can I separate the replication traffic to be sent on a separate...
Hi, We're considering the following network architecture: Each kafka server will have 2 network cards: one will be used to send/receive messages to/from consumer/producer (external cluster traffic) the...
View Articlepreparing for the 0.8 final release
Hi, Everyone, We have been stabilizing the 0.8 branch since the beta1 release. I think we are getting close to an 0.8 final release. I made an initial list of the remaining jiras that should be fixed...
View ArticlePartition Balancing
Is it possible to specify data center information to Kafka such that all replicas for a given partition are not in the same data center? We have a cluster that spans 2 data centers and I'd like to...
View ArticleUsing Kafka Metrics
Hi, I am using Kafka-0.8 and have a module to write metrics to Graphite. I have a multi node cluster and was wondering if I need to enable metrics reporting on all of them. Any advice will be helpful....
View ArticleProducer thread safety
Hi, I'm testing Kafka 0.8 and I encounter an error when accessing a Producer from mutliple threads. I've always thought Producer was thread safe but it seems this is no longer the case. This is the...
View ArticleTopic messages with partitions=1 stored on multiple brokers
My current setup has 2 brokers, multiple topics, each topic with 1 partition, kafka 0.7.1 Some topics are logged onto multiple brokers inspite of partitions set to 1. Kafka 0.7.1 does not have...
View ArticleUnderstanding java options used by Kafka
Hi, I am fairly new to java and I am trying to figure out the reason behind each option listed on http://kafka.apache.org/documentation.html#java. Sorry if this a noob question :) In this list, one of...
View Articlefull disk
Hi - We ran into a situation on our dev cluster (3 nodes, v0.8) where we ran out of disk on one of the nodes . As expected, the broker shut itself down and all of the clients switched over to the other...
View Articlewill this cause message loss?
Hi team, We are using beta1. I am going to delete all topics and create them with more partitions. But I don't want to lose any messages. Assume the consumers are online all the time for the following...
View ArticleLeader not local
Hi team, I saw this line within a long time span in our logs for the same topic and partition. [2013-11-14 11:13:41,647] WARN [KafkaApi-1] Produce request with correlation id 529240 from client on...
View ArticleHigh-level consumer load-balancing problem
Hi, I have questions about the load balancing of kafka high-level consumer Suppose I have 4 partition And the producer throughput to these 4 partitions are like this 0 1 2 3 10MB/s 10MB/s 1MB/s 1MB/s...
View ArticleMultiple consumer connectors in process - lag metrics?
Hi, We are experimenting with having multiple consumer connectors running in the same process, under the same groupId (but with different topic filters). I'm wondering what the expected effect of this...
View ArticleList of topics with JMX?
Hello, Is it possible to see the list of topics in Kafka using JMX like it is possible using the kafka-list-topic.sh command? In addition is it possible to see the number of messages stored for a topic...
View ArticleProducer SSL?
Hi, While searching the user group messages I found the following thread -...
View ArticleNodeExists Keeper exception
Hi team, Still this is from beta1. I notice this exception occurred frequently in our broker logs. [2013-11-14 21:09:58,714] INFO Got user-level KeeperException when proce ssing...
View ArticleConsumer example not working for 0.8 beta
I am a newbie for Kafka and trying to execute the samples for 0.8 beta release from...
View ArticleMirror Maker
Hi there, I'm sorry about this silly question, just trying to make sure I got this right: Having a separate cluster for aggregate, means that I would need a second zookeeper for that cluster right? I...
View Article./sbt assembly-package-dependency needed for Kafka 0.7.2?
Do we need to run ./sbt assembly-package-dependency for Kafka 0.7.2? I am running into this. [idhuser [ at ] vmidh kafka-0.7.2-incubating-src]$ ./sbt assembly-package-dependency [info] Building project...
View ArticleHow to link kafka servers
Hi, My endpoint for kafka is in Europe where the a consumers gets the final messages. But I want to write to kafka servers in SG, and the US east and west and will then forward to kafka servers in the...
View Article
