Exception while importing data to dynamodb using data pipeline

256 views Asked by At

I setup data pipeline to import data into dynamodb database. I created a script which which would match the syntax of the dynamodb export. Below is the sample output,

DateAdded^C{"s":"2009-11-10T18:19:00-07:00"}^BASIN^C{"s":"B001AMUFSE"} DateAdded^C{"s":"2009-11-10T18:19:00-07:00"}^BASIN^C{"s":"B0013IWYHU"} DateAdded^C{"s":"2009-11-10T18:17:00-07:00"}^BASIN^C{"s":"B000P62ND6"}

when I feed this file to data pipeline I'm getting error

org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"item":{"DateAdded^C{\"s\":\"2009-11-10T18:19:00-07:00\"}^BASIN^C{\"s\":\"B001AMUFSE\"}":null}}

Is anything wrong with the input format, and here is my script line which converts to above synatx

line = 'DateAdded\x03{o}"s":"{}"{c}\x02ASIN\x03{o}"s":"{}"{c}\n'.format(record.date_added, record.asin, o = '{', c = '}')
1

There are 1 answers

0
Sachin Doiphode On BEST ANSWER

Got the answer, Instead of \n for new line I needed to use \x0A as a line feed.