I have a CSV file I'm trying to load into a MySQL table:
bkpgln,date,aircraft,tailnum,poe,time_out,pod,time_in,stops,inst_app,landings,nitelndgs,nav,se_retrgear,a_sel,a_mel,xc,day,night,act_inst,sim_inst,simulator,dualrecd,pic,p2,duration,remarks
054801,2013-08-19,C182,N12345,IPT,17:12:00,UNV,17:54:00,,,,,0.7,,,,,,,,,,,,,,CAP 13-1-9999/A3/121 John Doe. CREW P/U (me).
[...more lines like this...]
My load script fails, it seems, because it won't accept a NULL (empty?) value for an integer column, even though in the table definition NULLs are allowed. Is this something I can fix easily?
$ ./load-csv-with-scriptella
Dec 20, 2013 3:01:31 PM <WARNING> XML configuration warning in file:/Users/gknauth/test/db-rebuild/etl.xml(3:79): Attribute "encoding" must be declared for element type "connection".
Dec 20, 2013 3:01:31 PM <WARNING> XML configuration warning in file:/Users/gknauth/test/db-rebuild/etl.xml(5:128): Attribute "encoding" must be declared for element type "connection".
Dec 20, 2013 3:01:31 PM <INFO> Execution Progress.Initializing properties: 1%
Dec 20, 2013 3:01:31 PM <INFO> Execution Progress.Initialized connection id=in, CsvConnection, Dialect{CSV 1.0}, properties {}: 3%
Dec 20, 2013 3:01:31 PM <INFO> Execution Progress.Initialized connection id=out, JdbcConnection{com.mysql.jdbc.JDBC4Connection}, Dialect{MySQL 5.1.72}, properties {}: 5%
Dec 20, 2013 3:01:31 PM <INFO> Execution Progress./etl/query[1] prepared: 10%
Dec 20, 2013 3:01:31 PM <INFO> Registered JMX mbean: scriptella:type=etl,url="file:/Users/gknauth/test/db-rebuild/etl.xml"
Dec 20, 2013 3:01:31 PM <WARNING> Unable to rollback transaction for connection CsvConnection: Transactions are not supported by CsvConnection
Dec 20, 2013 3:01:31 PM <SEVERE> Script /Users/gknauth/test/db-rebuild/etl.xml execution failed.
Location: /etl/query[1]/script[1]
JDBC provider exception: Unable to execute statement
Error statement:
INSERT INTO logbook
(bkpgln,date,aircraft,tailnum,poe,time_out,pod,time_in,stops,inst_app,landings,nitelndgs,nav,se_retrgear,a_sel,a_mel,xc,day,night,act_inst,sim_inst,simulator,dualrecd,pic,p2,duration,remarks)
VALUES
(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)
. Parameters: [054801, 2013-08-19, C182, N12345, IPT, 17:12:00, UNV, 17:54:00, , , , , 0.7, , , , , , , , , , , , , , CAP 13-1-9999/A3/121 John Doe. CREW P/U (me).]
Error codes: [HY000, 1366]
Driver exception: java.sql.SQLException: Incorrect integer value: '' for column 'inst_app' at row 1
Here's my etl.xml
:
<!DOCTYPE etl SYSTEM "http://scriptella.javaforge.com/dtd/etl.dtd">
<etl>
<connection id="in" driver="csv" url="logbook-update.csv" encoding="UTF-8"/>
<connection id="out" driver="mysql" url="jdbc:mysql://localhost/flying"
classpath="/Users/gknauth/lib/jar/mysql-connector-java-5.1.23-bin.jar" user="username" password="password" encoding="UTF-8"/>
<query connection-id="in">
<script connection-id="out">
INSERT INTO logbook
(bkpgln,date,aircraft,tailnum,poe,time_out,pod,time_in,stops,inst_app,landings,nitelndgs,nav,se_retrgear,a_sel,a_mel,xc,day,night,act_inst,sim_inst,simulator,dualrecd,pic,p2,duration,remarks)
VALUES
(?bkpgln,?date,?aircraft,?tailnum,?poe,?time_out,?pod,?time_in,?stops,?inst_app,?landings,?nitelndgs,?nav,?se_retrgear,?a_sel,?a_mel,?xc,?day,?night,?act_inst,?sim_inst,?simulator,?dualrecd,?pic,?p2,?duration,?remarks)
</script>
</query>
</etl>
I figured out the answer to my own question. Here was my fix: