I have an SQL database with some columns of type TEXT [] (array of strings).
I'm importing the database into AVRO format and saving it to HDFS using the sqoop command:
sqoop import-all-tables
-Dmapreduce.job.user.classpath.first=true
--connect jdbc:postgresql://localhost/project
--username postgres
--warehouse-dir /project
--as-avrodatafile
--compression-codec=snappy
--outdir output/avsc
--m 1
And getting the error:
23/05/08 09:33:51 ERROR orm.ClassWriter: Cannot resolve SQL type 2003
23/05/08 09:33:51 ERROR orm.ClassWriter: No Java type for SQL type 2003 for column genres
I tried to fix it by adding the --map-column-hive 'genres'=ARRAY\<STRING\> flag. Then sqoop printed this flah must be used with --hive-import.
The resulting command
sqoop import-all-tables
-Dmapreduce.job.user.classpath.first=true
--connect jdbc:postgresql://localhost/project
--username postgres
--warehouse-dir /project
--as-avrodatafile
--compression-codec=snappy
--outdir output/avsc
--m 1
--map-column-hive 'genres'=ARRAY\<STRING\>
--hive-import
is giving the error:
Hive import is not compatible with importing into AVRO format.
Is there a way to import SQL tables with arrays into AVRO format?
I'm using Hive 1.2.1000.2.6.5.0-292