Talend execution order in Subjob with tLoop

49 views Asked by mlaus At 29 February 2024 at 13:13

I have the following problem of understanding.

I have the following etl process Subjob which mainly consists of the following key parts:

(1) is a Loop which iterates every 1st of month from a given start month. Lets say the start date is the 15st oct. 2023. It will iterate from 1st Oct. 2023 until the 1st feb. 2024, in this case 5 months.
(2) is a tJava component which writes 2 variables (CurrentIterationDate, CurrentIterationDateMinus3Years): The current date of the iteration as integer (20231001, 20231101, … 20240201) and the same date 3 years ago (20201001, 20201101, … 20210201) into a 2nd global variable. This is to create a time period (20201001 to 20231001, 20201101 to 20231101, .. 20210201 to 20240201) for each iteration.
(3) and (4) are the data sources which collect some data based on the time periods mentioned in point 2. The time period is written in SQL where clause like:

where DATE >=  globalVariable(CurrentIterationDateMinus3Years)
and DATE <= globalVariable(CurrentIterationDate)

(5) is a .csv file import, the name is based on the tLOOP_3_CURRENT_ITERATION from the tLoop (1). In the 1st iteration the file name which ends with 1 will be loaded, in the 2nd iteration the filename which ends with 2 and so on. 1 File for every month.
(6) is a tJava component which initializes the 1st values (as the 1st iteration should do) of the loop. I know this is bad practice, I did this only to avoid an error which come up during execution which said that the job can’t load the .csv file (5). It seems for me like the components (5) and (4) are start doing there jobs before the loop initialized the variables. With no. (4) I am not fully sure because I Initialized the date variables in the same time in component (6)

So my problem of understanding is how this etl process works. There are mainly 2 questions to me:

Which components are executed first, respectively what is the order of execution of all the components?
If the components like (4) and (5) start before the tLoop begins his 1st iteration, how can I use a tLoop to load files in iterations based on the number of the current iteration and number of the file?

TechQA.