Does anyone know how Dataform handles multiple rows with the same uniqueKey in incremental tables?
For example if i have a SQLX file like this:
config {
type: "incremental",
database: "adh-silver",
schema: "financials",
name: "24so_gl_transaction_actual",
uniqueKey: ["pk_transaction_id"],
}
SELECT
pk_transaction_id,
transaction_number,
transaction_date,
financial_period,
amount,
last_modified,
created_time
FROM
${ref("adh-silver", "financials", "24so_gl_transaction_historic")}
${
when(
incremental(),
`WHERE last_modified > (SELECT MAX(last_modified) FROM ${self()})`
)
}
If there are multiple rows, in an incremental update, with the same "pk_transaction_id", how does Dataform know which to merge last?
In other tools it is possible to define a column to order the merges, for example with the "last_modified" column. Is there such an option in Dataform?
Thanks in advance for any help!