SSIS staging truncate warehouse

134 views Asked by At

Daily we get the data in excel formats we load the data into staging and then go to SSIS package and take excel as connection manager and perform transformations and move the data to warehouse. since we are taking data from excel only then why to create a stage and truncate it, since we taking excel as source and every manipulation is done with in it? Can someone please explain Real time scenario? I have seen many websites and couldn't understand what the concept is all about like staging, source(excel),lookup target(warehouse) Why to create to stage since everything is being done SSIS package only ?

1

There are 1 answers

0
Amira Bedhiafi On

The staging area is mainly used to quickly extract data from its data sources, minimizing the impact of the sources. After data has been loaded into the staging area, the staging area is used to combine data from multiple data sources, transformations, validations, data cleansing.

You can use a staging design pattern :

  • Incremental load
  • Truncate Insert
  • Using Delimiters with HashBytes for Change Detection

You can find out about the Package design pattern for loading a data warehouse