How do we create records and insert data into Hive table

I am collecting following values from the XML file by using scala XML API.

  1. name
  2. dept
  3. address

I will be getting millions of records with this information.

so I have to go through each tag and collect the above information and make a record.

I am doing this as follows:

val parse_record = BufferList[String]
val parse_record += (name,dept,address)

after this how do we union all these records and once I we have processed and created all the records, how do we insert into hive table.

for example,

I have ended up with 4 records and I want convert that it into dataframe and insert it into hive table

parse_record1(name,dept,address)
parse_record2(name,dept,address)
parse_record3(name,dept,address)
parse_record4(name,dept,address

Could someone please guide me on how to implement this. I am using spark 2.3.1 and scala 2.11.

0 Answers