spark interpreter read hive metastore time out

780 views Asked by At

everyone,i'm use the spark interpreter of zeppelin to read hive data. my version info list:

  • zeppelin:Version 0.8.2
  • CDH: Cloudera Express 5.13.1
  • hive:1.1.0

when i run this sql in a paragraph of spark interpreter notebook %sql select * from the_table_not_exists evenif the table is not exists,it also run 300s,i think 300s is the configuration hive.metastore.client.socket.timeout=300

And the log is bellow,Info and WARN are spaced exactly 300s apart

INFO [2020-10-16 11:32:42,887] ({pool-2-thread-7} SchedulerFactory.java[jobStarted]:114) - Job 20200723-090353_389722279 started by scheduler org.apache.zeppelin.spark.SparkSqlInterpreter113305554
 WARN [2020-10-16 11:37:42,930] ({pool-2-thread-7} RetryingMetaStoreClient.java[invoke]:184) - MetaStoreClient lost connection. Attempting to reconnect.
org.apache.thrift.transport.TTransportException: java.net.SocketTimeoutException: Read timed out
...
Caused by: java.net.SocketTimeoutException: Read timed out
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
    at java.net.SocketInputStream.read(SocketInputStream.java:171)
    at java.net.SocketInputStream.read(SocketInputStream.java:141)
    at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
    at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
    at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
    at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
    ... 93 more
 INFO [2020-10-16 11:37:43,933] ({pool-2-thread-7} HiveMetaStoreClient.java[open]:376) - Trying to connect to metastore with URI thrift://host:9083
 INFO [2020-10-16 11:37:43,948] ({pool-2-thread-7} HiveMetaStoreClient.java[open]:472) - Connected to metastore.
 INFO [2020-10-16 11:37:43,956] ({pool-2-thread-7} SchedulerFactory.java[jobFinished]:120) - Job 20200723-090353_389722279 finished by scheduler org.apache.zeppelin.spark.SparkSqlInterpreter113305554

then reconnect hive metastore now I have no idea how to fix it ,is anyone can help me ? thank you

0

There are 0 answers