i am getting error while using window functions in pyspark

126 views Asked by At

i am trying to run the below code

employees = (spark.read.format('csv')
             .option('sep', '\t')
             .schema('''EMP_ID INT,F_NAME STRING,L_NAME STRING,
                        EMAIL STRING,PHONE_NR STRING,HIRE_DATE STRING,
                        JOB_ID STRING,SALARY FLOAT,
                        COMMISSION_PCT STRING,
                        MANAGER_ID STRING,DEP_ID STRING''')
             .load('C:/data/hr_db/employees')
)

spec = Window.partitionBy('DEP_ID')

emp = (employees
         .select('JOB_ID', 'DEP_ID', 'SALARY')
         .withColumn('Total Salary', sum('SALARY').over(spec))
         .orderBy('DEP_ID')
)

emp.show()

and getting the below error

File "C:\spark-2.4.4-bin-hadoop2.7\python\lib\py4j-0.10.7-src.zip\py4j\protocol.py", line 328, in get_return_value
py4j.protocol.Py4JJavaError: An error occurred while calling o60.showString.java.lang.IllegalArgumentException: Unsupported class file major version 56

could you please anyone help me on this error?

0

There are 0 answers