I am working on hive table partitioning and using spark client to trigger the request. I have created the table and inserted data with partition.When I execute select statement I can see the data, but when I add Partition
spark.sql("ALTER TABLE temp_table6 ADD IF NOT EXISTS PARTITION (state = '34' , city = '123')")
second time onwards I am not able to get the data.
Since spark client looking for partitioned folder in temp location temp/temp_table6
, spark is throwing an exception like below
py4j.protocol.Py4JJavaError: An error occurred while calling o93.showString.
: org.apache.hadoop.mapred.InvalidInputException: Input path does not exist: file:/tmp/temp_table6/state=34/city=123
From second time onwards partitioned data not getting created under temp folder.