I am pretty new to Databricks Delta Live tables and I built a small use case to test how it works. Therefore I've created 2 tables, which I want to join in the notebook and then write the joined table into a 3rd table. Bc I want to test the APPLY CHANGES TO function I want to merge the joined result into this 3rd table. This table is currently empty and I tried this code in my notebook, but it does not work. The error is: Update 5a190f is FAILED.
The script looks like this:
WITH cte AS (
SELECT
distinct a.PK,
a.Value AS first_name,
b.VALUE AS last_name
FROM xx_xxx_xxx_dev.sandbox.tablea AS A
JOIN xx_xxx_xxx_dev.sandbox.tableb AS B ON a.PK = b.FK
)
APPLY CHANGES INTO xx_xxx_xxx_dev.sandbox.joined_table
FROM cte
KEY (PK)
;
I have the feeling I am missing a lot of things here, so any help is highly appreciated. The configuration of the pipeline is this:
I've tried first to create the table if not exist in the notebook, but this didn't work, so I've created the table in the schema which is referenced in the pipeline settings as target schema. I played with semicolons, but I do not have any clue tbh if there have to be some between the 2 blocks of cte and apply changes part. The tables which are joined are just regular tables in the schema.