Quantcast
Channel: Kafka Timeline
Viewing all articles
Browse latest Browse all 1519

how is the partitioner working when compression is enabled?

$
0
0
I am using Kafka 0.8 and message compression can only be enabled on producer side.

However one batch of messages (decided by the producer configuration batch.num.messages) is compressed as a single message and stored at the broker side. I think one batch compressed message will be stored at one broker.

I am wondering in this case how the producer side partitioner class will work? The batch will be partitioned to the partition of the first or last message in the batch?

Lex

Viewing all articles
Browse latest Browse all 1519

Trending Articles