How to setup development environment for kylin with Hadoop in Windows

121 views Asked by At

I want to use a Kylin instance on my Windows machine.

I saw from the docs this link (https://kylin.apache.org/development40/dev_env.html) that explains how to setup development environment for Kylin on Windows. my kylin.properties is as follows:

kylin.metadata.sync-retries=3
kylin.env.hdfs-working-dir=C:\Users\path\Desktop\kylin\examples\test_case_data\sample_local
kylin.env=LOCAL
kylin.env.zookeeper-base-path=/kylin
kylin.env.zookeeper-is-local=true
kylin.env.zookeeper-connect-string=sandbox.hortonworks.com
kylin.server.mode=all
kylin.server.cluster-servers=localhost:7070
kylin.web.timezone=GMT+8
kylin.web.query-timeout=300000
kylin.web.cross-domain-enabled=true
kylin.web.export-allow-admin=true
kylin.web.export-allow-other=true
kylin.web.hide-measures=RAW
kylin.restclient.connection.default-max-per-route=20
kylin.restclient.connection.max-total=200
kylin.engine.default=6
kylin.storage.default=4
kylin.web.hive-limit=20
kylin.web.help.length=4
kylin.web.help.0=start|Getting Started|http://kylin.apache.org/docs/tutorial/kylin_sample.html
kylin.web.help.1=odbc|ODBC Driver|http://kylin.apache.org/docs/tutorial/odbc.html
kylin.web.help.2=tableau|Tableau Guide|http://kylin.apache.org/docs/tutorial/tableau_91.html
kylin.web.help.3=onboard|Cube Design Tutorial|http://kylin.apache.org/docs/howto/howto_optimize_cubes.html
kylin.web.link-streaming-guide=http://kylin.apache.org/
kylin.htrace.show-gui-trace-toggle=false
kylin.web.link-hadoop=
kylin.web.link-diagnostic=
kylin.web.contact-mail=
kylin.server.external-acl-provider=
kylin.source.hive.database-for-flat-table=default
kylin.web.default-time-filter=0
kylin.storage.clean-after-delete-operation=false
kylin.job.retry=0
kylin.job.max-concurrent-jobs=10
kylin.job.sampling-percentage=100
kylin.job.scheduler.provider.100=org.apache.kylin.job.impl.curator.CuratorScheduler
kylin.job.scheduler.default=100
kylin.cube.cuboid-scheduler=org.apache.kylin.cube.cuboid.DefaultCuboidScheduler
kylin.cube.segment-advisor=org.apache.kylin.cube.CubeSegmentAdvisor
kylin.cube.aggrgroup.max-combination=32768
kylin.cube.cubeplanner.enabled=false
kylin.cube.cubeplanner.enabled-for-existing-cube=false
kylin.cube.cubeplanner.expansion-threshold=2.5
kylin.cube.cubeplanner.recommend-cache-max-size=200
kylin.cube.cubeplanner.mandatory-rollup-threshold=1000
kylin.cube.cubeplanner.algorithm-threshold-greedy=8
kylin.cube.cubeplanner.algorithm-threshold-genetic=23
kylin.query.cache-enabled=true
kylin.query.cache-threshold-scan-count=10240
kylin.query.cache-threshold-duration=2000
kylin.query.cache-threshold-scan-bytes=1048576
kylin.query.large-query-threshold=1000000
kylin.query.calcite.extras-props.caseSensitive=true
kylin.query.calcite.extras-props.unquotedCasing=TO_UPPER
kylin.query.calcite.extras-props.quoting=DOUBLE_QUOTE
kylin.query.calcite.extras-props.conformance=LENIENT
kylin.query.security.table-acl-enabled=true
kylin.query.interceptors=org.apache.kylin.rest.security.TableInterceptor
kylin.query.escape-default-keyword=false
kylin.query.transformers=org.apache.kylin.query.util.DefaultQueryTransformer
kylin.security.profile=testing
kylin.security.acl.admin-role=admin
kylin.security.ldap.connection-server=ldap://ldap_server:389
kylin.security.ldap.connection-username=
kylin.security.ldap.connection-password=
kylin.security.ldap.connection-truststore=
kylin.security.ldap.user-search-base=
kylin.security.ldap.user-search-pattern=
kylin.security.ldap.user-group-search-base=
kylin.security.ldap.user-group-search-filter=(|(member={0})(memberUid={1}))
kylin.security.ldap.service-search-base=
kylin.security.ldap.service-search-pattern=
kylin.security.ldap.service-group-search-base=
kylin.security.saml.metadata-file=classpath:sso_metadata.xml
kylin.security.saml.metadata-entity-base-url=https://hostname/kylin
kylin.security.saml.keystore-file=classpath:samlKeystore.jks
kylin.security.saml.context-scheme=https
kylin.security.saml.context-server-name=hostname
kylin.security.saml.context-server-port=443
kylin.security.saml.context-path=/kylin
kylin.spark-conf.auto.prior=true
kylin.spark.driver.log4j.properties=spark-driver-log4j-default.properties
kylin.spark.executor.log4j.properties=spark-executor-log4j-default.properties
kylin.engine.spark-conf.spark.master=local
kylin.engine.spark-conf.spark.submit.deployMode=client
kylin.engine.spark-conf.spark.yarn.queue=default
kylin.engine.spark-conf.spark.eventLog.enabled=true
kylin.engine.spark-conf.spark.eventLog.dir=C:UserspathDesktopkylinexampleseventLog
kylin.engine.spark-conf.spark.history.fs.logDirectory=hdfs:///spark-history
kylin.engine.spark-conf.spark.hadoop.yarn.timeline-service.enabled=false
kylin.engine.spark-conf.spark.executor.extraJavaOptions=-Dhdp.version=current
kylin.engine.spark-conf.spark.driver.extraJavaOptions=-Dhdp.version=current
kylin.query.auto-sparder-context-enabled=false
kylin.query.spark-conf.spark.master=yarn
kylin.query.spark-conf.spark.driver.cores=1
kylin.query.spark-conf.spark.driver.memory=4G
kylin.query.spark-conf.spark.driver.memoryOverhead=1G
kylin.query.spark-conf.spark.executor.cores=1
kylin.query.spark-conf.spark.executor.instances=1
kylin.query.spark-conf.spark.executor.memory=4G
kylin.query.spark-conf.spark.executor.memoryOverhead=1G
kylin.query.spark-conf.spark.serializer=org.apache.spark.serializer.JavaSerializer
kylin.query.spark-conf.spark.hadoop.yarn.timeline-service.enabled=false
kylin.query.spark-conf.spark.executor.extraJavaOptions=-Dhdp.version=current -Dlog4j.configuration=spark-executor-log4j-default.properties -Dlog4j.debug -Dkylin.hdfs.working.dir=C:\Users\path\Desktop\kylin\examples\test_case_data\sample_local -Dkylin.metadata.identifier=C:\Users\path\Desktop\kylin\examples\test_case_data\sample_local -Dkylin.spark.category=sparder -Dkylin.spark.identifier={{APP_ID}}
[email protected]
kylin.source.hive.client=cli
kylin.storage.url=C:\Users\path\Desktop\kylin\examples\test_case_data\sample_local
kylin.engine.mr.reduce-input-mb=500
kylin.engine.mr.uhc-config-override.mapreduce.reduce.memory.mb=500
kylin.engine.mr.uhc-config-override.mapred.reduce.child.java.opts=-Xmx400M
kylin.job.use-remote-cli=false
kylin.job.remote-cli-hostname=sandbox
kylin.job.remote-cli-username=root
kylin.job.remote-cli-password=hadoop
kylin.job.remote-cli-working-dir=/tmp/kylin
kylin.source.hive.redistribute-flat-table=false
kylin.engine.mr.yarn-check-interval-seconds=10
kylin.storage.hbase.compression-codec=gzip
kylin.engine.mr.max-reducer-number=5
kylin.storage.hbase.region-cut-gb=0.1
kylin.storage.hbase.max-region-count=5
kylin.storage.hbase.hfile-size-gb=2
kylin.job.lock=org.apache.kylin.job.lock.MockJobLock
kylin.engine.mr.uhc-reducer-count=3
kylin.dictionary.forest-trie-max-mb=500
kylin.cube.algorithm=random
kylin.query.security-enabled=true
kylin.server.query-metrics-percentiles-intervals=60, 360, 3600
kylin.source.hive.keep-flat-table=false
kylin.engine.spark.rdd-partition-cut-mb=100
kylin.engine.spark-conf.spark.yarn.submit.file.replication=1
kylin.engine.spark-conf.spark.executor.memoryOverhead=384
kylin.engine.spark-conf.spark.driver.memoryOverhead=256
kylin.engine.spark-conf.spark.executor.memory=768M
kylin.engine.spark-conf.spark.executor.cores=1
kylin.engine.spark-conf.spark.executor.instances=1
kylin.engine.spark-conf.spark.driver.cores=1
kylin.engine.spark-conf.spark.yarn.am.extraJavaOptions=-Dhdp.version=current
kylin.engine.spark-conf-mergedict.spark.executor.memory=1G
kylin.engine.spark-conf-mergedict.spark.memory.fraction=0.2
kylin.engine.spark-conf.spark.sql.shuffle.partitions=1
kylin.engine.flink-conf.jobmanager.heap.size=2G
kylin.engine.flink-conf.taskmanager.heap.size=4G
kylin.engine.flink-conf.taskmanager.numberOfTaskSlots=1
kylin.engine.flink-conf.yarn.queue=
kylin.engine.flink-conf.yarn.nodelabel=
kylin.server.cluster-name=C:\Users\path\Desktop\kylin\examples\test_case_data\sample_local
kylin.metadata.url.identifier=C:\Users\path\Desktop\kylin\examples\test_case_data\sample_local

The server runs OK, but when I try to upload data (in csv format) from the model tab I get the following error:

Caused by: java.net.UnknownHostException: sandbox.hortonworks.com

0

There are 0 answers