Connecting Kylin to Postgres in Docker: Unable to Retrieve Tables

80 views Asked by At

I have a project where I need to establish communication between Kylin and Postgres, and I've already containerized everything. It appears that Kylin can detect the Postgres container (as verified by the 'ping' command from within the Kylin container). However, I'm facing an issue when attempting to retrieve the database tables from the Kylin interface.

The error is: cannot get HiveTableMeta

I have placed the Postgres JAR file in the lib/ext directory and the XML file in the conf/datasource directory with the following content:

<DATASOURCE_DEF NAME="kylin" ID="mypostgres" DIALECT="postgresql"/>

I have attached the configuration from my docker-compose.yml and kylin.properties

docker-compose:

version: '3'
services:   
  postgres-database:
    image: postgis/postgis:12-3.1-alpine
    container_name: ms-postgres-database
    environment: 
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
    volumes:
      - ms-pgsqldb:/var/lib/postgresql/data
      - ./database/postgres/init-database.sh:/docker-entrypoint-initdb.d/init-database.sh
      - ./database/postgres/init_sql:/init_sql
    ports:
      - "13000:5432"
    networks:
      - ms-network      
  ms-pgadmin:
    image: dpage/pgadmin4:6.8
    container_name: ms-pgadmin
    ports:
      - "13001:80"
    environment:
      PGADMIN_DEFAULT_EMAIL: //MAIL
      PGADMIN_DEFAULT_PASSWORD: //PASS
    volumes:
      - ./pgadmin/servers.json:/pgadmin4/servers.json
    networks:
      - ms-network
    depends_on:
      - postgres-database
  ms-kylin:
    container_name: kylin
    image: apachekylin/apache-kylin-standalone:4.0.0
    ports:
      - "7070:7070"
      - "8088:8088"
      - "50070:50070"
      - "8032:8032"
      - "8042:8042"
      - "2181:2181"
    environment:
      - JDBC_URL='jdbc:postgresql://postgres-database:5432/kylin
      - JDBC_USER=${DB_POSTGRES_USERNAME}
      - JDBC_PASSWORD=${DB_POSTGRES_PWD}
    volumes:
      - ms-kylin:/app/kylin_home
    networks:
      - ms-network
networks:
  ms-network:
    driver: bridge
volumes:
  ms-pgsqldb:
  ms-kylin:

kylin.properties:

kylin.env=prod
kylin.query.pushdown.runner-class-name=org.apache.kylin.query.pushdown.PushdownRunnerSDKImpl
kylin.metadata.url=jdbc:postgresql://postgres-database:5432/kylin
kylin.metadata.driver=org.postgresql.Driver
kylin.metadata.username=user
kylin.metadata.password=pass

kylin.source.jdbc.url=jdbc:postgresql://postgres-database:5432/kylin
kylin.source.jdbc.user=user
kylin.source.jdbc.pass=pass
kylin.source.jdbc.dialect=postgresql
kylin.source.jdbc.driver=org.postgresql.Driver

POSTGRESSQL.XML:

<DATASOURCE_DEF NAME="kylin" ID="mypostgres" DIALECT="postgresql"/>
0

There are 0 answers