ALTER TABLE table ADD IF NOT EXISTS PARTITION (state = '34' , city = '123') is not adding the partition in temp/local folder

2.4k views Asked by At

I am working on hive table partitioning and using spark client to trigger the request. I have created the table and inserted data with partition.When I execute select statement I can see the data, but when I add Partition

spark.sql("ALTER TABLE temp_table6 ADD IF NOT EXISTS PARTITION (state = '34' , city = '123')")

second time onwards I am not able to get the data. Since spark client looking for partitioned folder in temp location temp/temp_table6, spark is throwing an exception like below

py4j.protocol.Py4JJavaError: An error occurred while calling o93.showString.
: org.apache.hadoop.mapred.InvalidInputException: Input path does not exist: file:/tmp/temp_table6/state=34/city=123

From second time onwards partitioned data not getting created under temp folder.

0

There are 0 answers