Quantcast
Channel: Kafka Timeline
Viewing all articles
Browse latest Browse all 1519

Trouble with snappy and SimpleConsumer

$
0
0
Hi,
I am using
kafka_2.9.2-0.8.1 and
snappy-java-1.1.0.1.jar

I have been able to successfully use gzip with the same library. however “snappy” doesn’t work in consumer side.
Producer is able to send snappy messages to broker though.
I have made sure that snappy java lib is the same on both consumer and producer side.

Is there a problem with 1.1.0.1 snappy for usage with kafka ??? Should I switch to an older version??

Anyone else having this problem?? Thanks in advance for any pointers.

Thanks
-Vinay

——————————Exception ——————————————

topic=events partition=2leaderId=3 beginOffset=2225451 offset=2225451 checksum=0 time=1402008438921
java.lang.Exception: org.xerial.snappy.SnappyNative.uncompressedLength(Ljava/lang/Object;II)I
at org.xerial.snappy.SnappyNative.uncompressedLength(Native Method)
at org.xerial.snappy.Snappy.uncompressedLength(Snappy.java:541)
at org.xerial.snappy.SnappyInputStream.hasNextChunk(SnappyInputStream.java:350)
at org.xerial.snappy.SnappyInputStream.rawRead(SnappyInputStream.java:158)
at org.xerial.snappy.SnappyInputStream.read(SnappyInputStream.java:142)
at java.io.InputStream.read(InputStream.java:82)
at kafka.message.ByteBufferMessageSet$$anonfun$decompress$1.apply$mcI$sp(Unknown Source)
at kafka.message.ByteBufferMessageSet$$anonfun$decompress$1.apply(Unknown Source)
at kafka.message.ByteBufferMessageSet$$anonfun$decompress$1.apply(Unknown Source)
at scala.collection.immutable.Stream$.continually(Stream.scala:1129)
at kafka.message.ByteBufferMessageSet$.decompress(Unknown Source)
at kafka.message.ByteBufferMessageSet$$anon$1.makeNextOuter(Unknown Source)
at kafka.message.ByteBufferMessageSet$$anon$1.makeNext(Unknown Source)
at kafka.message.ByteBufferMessageSet$$anon$1.makeNext(Unknown Source)
at kafka.utils.IteratorTemplate.maybeComputeNext(Unknown Source)
at kafka.utils.IteratorTemplate.hasNext(Unknown Source)
at kafka.javaapi.message.ByteBufferMessageSet$$anon$1.hasNext(Unknown Source)

Caused by: java.lang.UnsatisfiedLinkError: org.xerial.snappy.SnappyNative.uncompressedLength(Ljava/lang/Object;II)I
... 31 more

Viewing all articles
Browse latest Browse all 1519

Trending Articles