addition of a partition
Hi team, I am reading this link: https://cwiki.apache.org/confluence/display/KAFKA/Replication+tools#Replicationtools-5.AddPartitionTool and this JIRA...
View Articleabout compiling broker in 0.8 branch
Hi team, For the current 0.8 branch, is it recommended to compile it with Scala 2.10? I remember someone said previously it is best to compile the broker with Scala 2.80. Thanks. Regards, Libo
View Articleupgrade from 0.8-beta1 to 0.8
Hi team, Currently we are using 0.8-beta1. We plan to upgrade to 0.8. My concern is whether we need to purge all existing kafka and zookeeper data on the hard drive for this upgrade. In other words,...
View Articlebug in documentation?
*'Not that partitioning means Kafka only provides a total order over messages within a partition. This combined with the ability to partition data by key is sufficient for the vast majority of...
View ArticleLoggly's use of Kafka on AWS
A couple of us here at Loggly recently spoke at AWS reinvent, on how we use Kafka 0.72 in our ingestion pipeline. The slides are at the link below, and may be of interest to people on this list....
View ArticleChanging the Replication Factor
Hi, Using 0.8 beta1, how can I go about changing the replication factor for a topic that already exists? Thanks, Ryan
View ArticleAdd a new node in an existing kafka 0.8 cluster
Hi, Last week we set-up a new kafka 0.8 cluster, using the beta1 released available here : http://kafka.apache.org/downloads.html It worked fine until we tried to replace a node in this cluster. We...
View ArticleKafka trouble with Avro encoding
Hello - I am trying to understand my trouble passing an Avro message through Kafka (0.8) From what I see, the class tries to create an instance of the encoder but fails as it can not find the...
View ArticleControlled shutdown and "UnderReplicatedPartitions" state?
The default value for "controlled.shutdown.enable" is false. Does that mean that stopping a broker without a controlled shutdown and using a "kill ?9" might lead to an under "UnderReplicatedPartitions"...
View Articleset new retention size on the fly
Hi folks, For 0.8, it is possible to add a partition dynamically. Is it possible to increase the retention size on the fly? This feature will be very useful for operation. I know rolling start can pick...
View Articlesnappy 1.0.4.1, java7, macos issue...
So, It looks like there is an issue with using snappy version 1.0.4.1, with java 7, on MacOSX, which is fixed in version 1.0.5. We've verified this. https://github.com/ptaoussanis/carmine/issues/5...
View ArticleApache Kafka 0.8.0 Released
The Apache Kafka community is pleased to announce the general availability release of Apache Kafka 0.8.0. Apache Kafka is publish-subscribe messaging system rethought as a distributed commit log. **...
View ArticleConsumer not able to create message streams
Hi All I am using Kafka 0.8 , with inbuilt zookeeper. I am runnings consumers in a jar. Configuration properties for creating consumerConfig : - zookeeper.connect=IP group.id=consumerGroup...
View Articlekafka_2.8.0/0.8.0 pom seems invalid
Hi All, I was trying to upgrade the kafka to 0.8 but I get an empty jar file for <dependency> <groupId>org.apache.kafka</groupId> <artifactId>kafka_2.8.0</artifactId>...
View ArticleClientUtils.fetchTopicMetadata reports smaller ISR than...
I'm working on some monitoring tools for Kafka, and I've seen a couple of clusters get into a state where ClientUtils.fetchTopicMetadata will show that not all replicas are in the ISR. At the same...
View ArticleHow to set kafka path in zk
Hi, If I don't want to register kafka in zk root and I want to make it under a namespace, for example kafka1. If I set only one host in zk property something like 10.237.0.1:2181/kafka, it works. But...
View ArticleIs there a way to delete partition at runtime?
Hi guys, I found there is a tool to add partition on the fly. My question is, is there a way to delete a partition at runtime? Thanks! Best, Siyuan
View ArticleUpdated kafka client (producer and consumer)
In our environment we use currently use Kafka 0.7.1. The core features I am looking for in a client are this: 1. Provide confirmation of produce requests (or notification of disconnection during...
View ArticlePreventing topics from appearing on specific brokers
Hello, Say we are using Zookeeper-based Producers, and we specify a topic to be written to. Since we don't specify the actual brokers, is there a way to prevent a topic from appearing on a specific...
View ArticleTrade-off between topics and partitions?
Hello, According to the Kafka FAQ "How do I choose the number of partitions for a topic", clusters with more than 10K partitions are not tested. I am looking for advice on how to scale the number of...
View Article