Spark Shell Error: Unable to load native-hadoop library and IllegalAccessError

30 views Asked by At

trying to run Spark Shell

23/11/13 20:14:50 WARN util.Utils: Your hostname resolves to a loopback address: 127.0.1.1; using \[SANITIZED\] instead (on interface wlp2s0)

23/11/13 20:14:50 WARN util.Utils: Set SPARK_LOCAL_IP if you need to bind to another address

Setting default log level to "WARN".

To adjust logging level, use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).

23/11/13 20:14:56 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

java.lang.IllegalAccessError: class org.apache.spark.storage.StorageUtils$ (in unnamed module @0x2ba867a6) cannot access class sun.nio.ch.DirectBuffer (in module java.base) because module java.base does not export sun.nio.ch to unnamed module @0x2ba867a6

\<console\>:14: error: not found: value spark

import spark.implicits.\_

^

\<console\>:14: error: not found: value spark

import spark.sql

^

Environment:

Spark Version: 3.2.4

Hadoop Version: 2.7.1

Scala Version: 2.12.15

Java Version: 17.0.8.1

Ubuntu Version: 22.04

I am encountering issues while trying to run Spark Shell

0

There are 0 answers