I'm trying to use Multiple Database Tables and BigQuery Multi Table Data Fusion plugin to import multiple table in one pipeline
But when I try to execute I get the following error
java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: BigQuery Multi Table has no outputs. Please check that the sink calls addOutput at some point.
I'm using Data Fusion version 6.1.4 Multiple Database Tables version 1.2.0 and BigQuery Multi Table version 0.14.8.
Any suggestion on what may be the problem?
Edit:
following the configuration of multiple table database source
{
"name": "Multiple Database Tables",
"plugin": {
"name": "MultiTableDatabase",
"type": "batchsource",
"label": "Multiple Database Tables",
"artifact": {
"name": "multi-table-plugins",
"version": "1.2.0",
"scope": "USER"
},
"properties": {
"splitsPerTable": "1",
"referenceName": "multiTable",
"connectionString": "${secure(connection)}",
"jdbcPluginName": "netezza",
"user": "${secure(username)}",
"password": "${secure(password)}",
"whiteList": "categoria_l,cliente_l,regione_l"
}
},
"outputSchema": [
{
"name": "etlSchemaBody",
"schema": ""
}
]
},
After further test the problem is that the source response is empty because data fusion is not reading view from source database but only tables
It seems like the
Multiple Database Tables
source produced no records ("Out 0"). I'd check there first. You can do a quick check using the Preview mode. Plugin doc here.Related answer here.