How to upload the range of files into database in talend

Question

Currently I have 14 files in my directory, I want to upload them into 7 per batch means want to upload 7 files first and then remaining 7 files. Can you please tell me the flow.

Answer 1

Job design should be like this .

1- in tJava_1 i have done dummy logs :

System.out.println(((String)globalMap.get("tFileList_1_CURRENT_FILEPATH"))); 

2- tfileproperties (i will need the mtime variable to get when the file was created) has a configuration as file :

((String)globalMap.get("tFileList_1_CURRENT_FILEPATH"))

3- tBufferOutput component will store you the data

4- tBufferInput component to get the data stored in tBufferOutput (don't forget to get the schema )

5- tSortRow component to sort by mtime

6- tJava_2

System.out.println(((Integer)globalMap.get("tBufferInput_1_NB_LINE")));  

// to get 7 first files
int j=((Integer)globalMap.get("tBufferInput_1_NB_LINE"))-7 ; 

globalMap.put("j",j) ; 

7- tloop would have While configuration :

Declare int i=0

Condition : i<(int)globalMap.get("j")

itération : i++

/////////////////////////////////////////////////////////////

Other way is to do it like this :

we can do with combination of tFileList and tFileExist components.

1- Let us Assume A1 is the Source.

2-Create A2 and A3 two other folders.

3- Iterate A1 folders with tFileList and use tFileExists -A2 directory path and global variable "tFileList_1_CURRENT_FILE to check the files in A2.

4- After TfileExistsuse condition tFileExist_2_EXISTS then copy the file using tFileCopy to Folder A3.

5- At end of run make sync between A1 and A2.

6- After processing new files in A3 archive/delete them

7- Like this every run new files will be stored in A3.