Disable hive support in sparklyr

777 views Asked by At

Is there any way to disable the hive support in sparklyr?

Just like in SparkR:

sparkR.session(master="local[*]", enableHiveSupport=FALSE)
2

There are 2 answers

0
apastor On BEST ANSWER

That option was introduced in sparklyr 1.3.0 after breaking Ron's answer solution on version 1.2.0 as described on issue #2460.

library(sparklyr)

config <- spark_config()
config$sparklyr.connect.enablehivesupport <- FALSE

sc <- spark_connect(master = "local", config = config)
0
Ron On

You can disable Hive in sparklyr by setting the SQL catalog implementation to in-memory.

# get the default config
conf <- spark_config()
# set the catalog implementation, defaults to hive, but we want it in-memory
conf$spark.sql.catalogImplementation <- "in-memory"
sc <- spark_connect(master = "local", config = conf)