export parquet format data to mysql using sqoop

Question

I import some data from mysql to hdfs, and saved with parquet format:

drwxr-xr-x   - root supergroup          0 2016-12-16 22:19 /wybdaily/test/zsqoop/.metadata-00000
drwxr-xr-x   - root supergroup          0 2016-12-16 22:20 /wybdaily/test/zsqoop/.signals-00000
-rw-r--r--   1 root supergroup       2056 2016-12-16 22:20 /wybdaily/test/zsqoop/19dfc82b-eb01-4395-a847-aa11390306fe.parquet

then I try to export these data back to mysql, exception occured:

16/12/16 22:21:09 WARN spi.Registration: Not loading URI patterns in org.kitesdk.data.spi.hive.Loader
16/12/16 22:21:09 ERROR sqoop.Sqoop: Got exception running Sqoop: org.kitesdk.data.DatasetNotFoundException: Descriptor location does not exist:     hdfs://iZbp1ig01gathmdv6uxz3wZ:8020/wybdaily/test/zsqoop/.metadata
org.kitesdk.data.DatasetNotFoundException: Descriptor location does not exist: hdfs://iZbp1ig01gathmdv6uxz3wZ:8020/wybdaily/test/zsqoop/.metadata
    at org.kitesdk.data.spi.filesystem.FileSystemMetadataProvider.checkExists(FileSystemMetadataProvider.java:562)
    at org.kitesdk.data.spi.filesystem.FileSystemMetadataProvider.find(FileSystemMetadataProvider.java:605)
    at org.kitesdk.data.spi.filesystem.FileSystemMetadataProvider.load(FileSystemMetadataProvider.java:114)
    at org.kitesdk.data.spi.filesystem.FileSystemDatasetRepository.load(FileSystemDatasetRepository.java:197)
    at org.kitesdk.data.Datasets.load(Datasets.java:108)
    at org.kitesdk.data.Datasets.load(Datasets.java:140)
    at org.kitesdk.data.mapreduce.DatasetKeyInputFormat$ConfigBuilder.readFrom(DatasetKeyInputFormat.java:92)
    at org.kitesdk.data.mapreduce.DatasetKeyInputFormat$ConfigBuilder.readFrom(DatasetKeyInputFormat.java:139)
    at org.apache.sqoop.mapreduce.JdbcUpdateExportJob.configureInputFormat(JdbcUpdateExportJob.java:192)
    at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:432)
    at org.apache.sqoop.manager.MySQLManager.upsertTable(MySQLManager.java:145)
    at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:74)
    at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:100)
    at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
    at org.apache.sqoop.Sqoop.main(Sqoop.java:236)

then I rename the '.metadata-00000' to '.metadata' the job also failed.

Anyone can tell me how to export parquet format data to mysql using sqoop1? An example or some docs. thanks!

Answer 1

by using hcatalog we can export parquet hive table in to RDBMS.

sqoop export --connect jdbc:oracle:thin:@//192.178.100.15:1521 --username user_name --password XXXXXX --table DB.Table_name --hcatalog-table "Table_name" --hcatalog-database "db_name"