How to create an incremental pipeline in fabric

91 views Asked by At

I want to create a pipeline to avoid keep loading the data that is already in the warehouse. I first created scripts to clear my dimension and fact tables (fact_Transactions). I then, inserted the dataflows to load the dimension and fact tables, and then tried to add a script to remove the old data. However, my script is always failing.

This is the script a colleague told me to try:

DELETE
FROM fact_Transactions
Where fk_date IN
(
SELECT
CAST(LEFT(fact_Transactions.fk.date,4) + '-'
    RIGHT(LEFT(fact_Transactions.fk.date,6),2) + '-'
    RIGHT(fact_Transactions.fk.date,2 ) AS DATE) as 
          dw_proper_date
FROM fact_Transactions
);

However, neither I understand it very well, neither it runs successfully.

0

There are 0 answers