Every time I make use of ECL, I have to define a record structure for any file in use (CSV, XML, JSON). However, is there a way of making use of these files on ECL without defining the record structure. Since I would be dealing with a larger set of files, probably tens of thousands of them, (and all of them do not share the same record structure), it would be hard for me to define a record structure for all the files I am working on.
We have defined record structures in each ECL file ,but this caters to specific data and applications.We are unaware of any method to avoid creation of multiple record structures and this will become a cumbersome task when count of files becomes much greater for ex: let's say 10,000 or 1 million .
The short answer is YES, you always have to define the layout of the fields in whatever dataset you want to use. You can do that with either a separate RECORD structure definition, or you can use curly braces ({}) to define it inline within the DATASET declaration. Either way, it is required.
You could "cheat" and simply declare all the files as FLAT with a 1-character string as the only field, like this:
But how are you going to actually work with the data when you don't know the field layout? That "trick" essentially gets you nowhere (and I only use it to just get a quick look at exactly what is in every byte of a file I'm unfamiliar with).
You may be working with data from many disparate sources, but I would expect that each source will have a single standard layout for each of their files, so you only need to pre-define the standard layouts and use those standard RECORD structures in your explicit DATASET declarations for each file (or SuperFile).