thirdCateBrandres.createOrReplaceTempView("tempTable2")
sql("insert overwrite table temp_cate3_Brand_List select * from tempTable2")
The code as above, thirdCateBrandres is a spark DataFrame, registered as a temp table,then write to table temp_cate3_Brand_List, the table has 3 billion row with 7 fields, data size is about 4GB in ORC+SNAPPY format .
These codes took about 20 minutes.
How can I speed up the program?