I wrote my client code as
val pipeGzipGet: HttpRequest => Future[String] = (
addHeader("Accept-Encoding", "gzip ")
~> sendReceive
~> logByteArray
~> decode(Gzip)
~> unmarashal[String]
)
It works fine most times , but will sometimes throw a
java.util.zip.ZipException: Corrupt data (CRC32 checksum error)
at spray.httpx.encoding.GzipDecompressor.fail$1(Gzip.scala:91) ~[io.spray.spray-httpx-1.2.0.jar:na]
at spray.httpx.encoding.GzipDecompressor.readTrailer$1(Gzip.scala:117) ~[io.spray.spray-httpx-1.2.0.jar:na]
at spray.httpx.encoding.GzipDecompressor.decomp(Gzip.scala:131) ~[io.spray.spray-httpx-1.2.0.jar:na]
at spray.httpx.encoding.GzipDecompressor.decompress(Gzip.scala:85) ~[io.spray.spray-httpx-1.2.0.jar:na]
at spray.httpx.encoding.Decompressor.decompress(Decoder.scala:41) ~[io.spray.spray-httpx-1.2.0.jar:na]
at spray.httpx.encoding.Decoder$class.decode(Decoder.scala:28) ~[io.spray.spray-httpx-1.2.0.jar:na]
at spray.httpx.encoding.Gzip.decode(Gzip.scala:23) ~[io.spray.spray-httpx-1.2.0.jar:na]
at spray.httpx.ResponseTransformation$$anonfun$decode$1.apply(ResponseTransformation.scala:28) ~[io.spray.spray-httpx-1.2.0.jar:na]
at spray.httpx.ResponseTransformation$$anonfun$decode$1.apply(ResponseTransformation.scala:28) ~[io.spray.spray-httpx-1.2.0.jar:na]
at scala.util.Success$$anonfun$map$1.apply(Try.scala:206) ~[org.scala-lang.scala-library-2.10.3.jar:na]
at scala.util.Try$.apply(Try.scala:161) ~[org.scala-lang.scala-library-2.10.3.jar:na]
at scala.util.Success.map(Try.scala:206) ~[org.scala-lang.scala-library-2.10.3.jar:na]
at scala.concurrent.Future$$anonfun$map$1.apply(Future.scala:235) ~[org.scala-lang.scala-library-2.10.3.jar:na]
at scala.concurrent.Future$$anonfun$map$1.apply(Future.scala:235) ~[org.scala-lang.scala-library-2.10.3.jar:na]
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32) [org.scala-lang.scala-library-2.10.3.jar:na]
at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.processBatch$1(BatchingExecutor.scala:67) [com.typesafe.akka.akka-actor_2.10-2.2.3.jar:2.2.3]
at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:82) [com.typesafe.akka.akka-actor_2.10-2.2.3.jar:2.2.3]
at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:59) [com.typesafe.akka.akka-actor_2.10-2.2.3.jar:2.2.3]
at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:59) [com.typesafe.akka.akka-actor_2.10-2.2.3.jar:2.2.3]
at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72) [org.scala-lang.scala-library-2.10.3.jar:na]
at akka.dispatch.BatchingExecutor$Batch.run(BatchingExecutor.scala:58) [com.typesafe.akka.akka-actor_2.10-2.2.3.jar:2.2.3]
at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:42) [com.typesafe.akka.akka-actor_2.10-2.2.3.jar:2.2.3]
at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386) [com.typesafe.akka.akka-actor_2.10-2.2.3.jar:2.2.3]
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [org.scala-lang.scala-library-2.10.3.jar:na]
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [org.scala-lang.scala-library-2.10.3.jar:na]
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [org.scala-lang.scala-library-2.10.3.jar:na]
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [org.scala-lang.scala-library-2.10.3.jar:na]
I also found some message may related to this from akka log
[akka://application/system/IO-TCP/selectors/$a/14] Message [akka.io.Tcp$Write] from Actor[akka://application/user/IO-HTTP/group-0/14#592690058] to Actor[akka://application/system/IO-TCP/selectors/$a/14#-1139525337] was not delivered. [10] dead letters encountered, no more dead letters will be logged. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'
Then I log the response with
val logByteArray = (r: HttpResponse) => {
val bytes = r.entity.as[Array[Byte]]
import java.util.Date
import java.io._
import java.text.SimpleDateFormat
val sdf = new SimpleDateFormat("HH_mm_ss_SSS" + idSeq.incrementAndGet)
val fout = new FileOutputStream(s"${sys.env("HOME")}/dump-${sdf.format(new Date)}")
fout.write(bytes.right.get)
fout.flush()
fout.close()
r
}
I use gzip -t
to test that file. It says
invalid compressed data--crc error
Seems response is corrupt.
But I tried with the request with an chrome plugin, it never fail.
Is it possible to avoid the exception ?