hiveserver2 hangs while starting Remote PostgreSQL Database for the Hive Metastore

67 views Asked by At

I am working in setting Remote PostgreSQL Database for Hive Metastore using

Hive 3.1.3
Hadoop 3.3.4
Spark 3.2.4
PostgresSQL 14.9

I could use Hive Schema Tool to create metastore tables in PostgreSQL using /home/ivan/hive/bin/schematool -dbType postgres -initSchema

with output

Metastore connection URL:    jdbc:postgresql://localhost:5432/metastore
Metastore Connection Driver :    org.postgresql.Driver
Metastore connection User:   hive
Starting metastore schema initialization to 2.3.0
Initialization script hive-schema-2.3.0.postgres.sql
Initialization script completed
schemaTool completed

I checked the schema in pgAdmin and everything works as expected so far. However, when I run hive --service metastore and hive --service hiveserver2 --hiveconf hive.server2.thrift.port=10000 --hiveconf hive.root.logger=INFO,console to start Hive Metastore Server and HiveServer2 services respectively, I get

hive --service metastore
2023-11-20 17:15:26: Starting Hive Metastore Server
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/ivan/hive/lib/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/ivan/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]

and

hive --service hiveserver2 --hiveconf hive.server2.thrift.port=10000 --hiveconf hive.root.logger=INFO,console
2023-11-20 17:17:25: Starting HiveServer2
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/ivan/hive/lib/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/ivan/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Hive Session ID = 003418b8-aae6-4061-ac77-6c36451ce5a2
2023-11-20T17:17:26,960  INFO [main] SessionState: Hive Session ID = 003418b8-aae6-4061-ac77-6c36451ce5a2
Hive Session ID = f9e4647e-d61c-4960-8fc4-32198b868c9f
2023-11-20T17:18:27,410  INFO [main] SessionState: Hive Session ID = f9e4647e-d61c-4960-8fc4-32198b868c9f
Hive Session ID = 5de05aeb-df6e-4ce8-8502-48d24e38699a
2023-11-20T17:19:27,449  INFO [main] SessionState: Hive Session ID = 5de05aeb-df6e-4ce8-8502-48d24e38699a
Hive Session ID = 140b29ae-a899-4bce-a873-3bba14ac896a
2023-11-20T17:20:27,489  INFO [main] SessionState: Hive Session ID = 140b29ae-a899-4bce-a873-3bba14ac896a

What proves that hiveserver2 is hanging. I run beeline (the HiveServer2 CLI), followed by !connect jdbc:hive2://localhost:10000 and I get the following error

beeline> !connect jdbc:hive2://localhost:10000
Connecting to jdbc:hive2://localhost:10000
Enter username for jdbc:hive2://localhost:10000: ivan
Enter password for jdbc:hive2://localhost:10000: ****
Could not open connection to the HS2 server. Please check the server URI and if the URI is correct, then ask the administrator to check the server status.
Error: Could not open client transport with JDBC Uri: jdbc:hive2://localhost:10000: java.net.ConnectException: Connection refused (Connection refused) (state=08S01,code=0)

I run the command netstat -ntpl | egrep "10000|10001|10002" to check if there is a hiveserver2 port running, but I get

netstat -ntpl | egrep "10000|10001|10002"
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)

what proves that hiveserver2 is still hanging. I would really appreciate your help

The most important settings in the hive-site.xml are the following ones

<configuration>
  <property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:postgresql://localhost:5432/metastore</value>
  </property>

  <property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>org.postgresql.Driver</value>
  </property>

  <property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>hive</value>
  </property>

  <property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>1234</value>
  </property>

  <property>
    <name>hive.metastore.warehouse.dir</name>
    <value>hdfs://192.168.0.5:9000/user/hive/warehouse</value>
  </property>
  <property>
    <name>hive.server2.thrift.http.port</name>
    <value>10001</value>
    <description>Port number of HiveServer2 Thrift interface when hive.server2.transport.mode is 'http'.</description>
  </property>
  <property>
    <name>hive.server2.thrift.port</name>
    <value>10000</value>
    <description>Port number of HiveServer2 Thrift interface when hive.server2.transport.mode is 'binary'.</description>
  </property>
</configuration>
0

There are 0 answers