Error while creating delta tables in apache spark

100 views Asked by At

I'm reading data from csv file and loading the data into delta table

I'm getting the following error while creating delta table

py4j.protocol.Py4JJavaError: An error occurred while calling o33.save.
: org.apache.spark.SparkClassNotFoundException: [DATA_SOURCE_NOT_FOUND] Failed to find the data source: delta. Please find packages at `https://spark.apache.org/third-party-projects.html`.

here is my code

from pyspark.sql import SparkSession

spark = SparkSession.builder \
    .appName("DeltaExample") \
    .getOrCreate()

csv_file = "F:\\Data\Test\\file.csv"

df_data = spark.read.csv(csv_file, header=True, inferSchema=True,sep=',')
df_data.write.format("delta").save("F:\\Data\\out\\test_delta")

can anyone suggest how to fix this error and store data into delta table i've installed spark successfully, my spark version is 3.5.0

0

There are 0 answers