I am working on a project to extract Master Data from a Database that gathers data from many departments, and this should be done using Talend. I did some researches to understand what is Master Data exactly and I have some modest experience in working with Talend, the problem is how to detect unchangeable data (Master Data) ? I can check dates, but what dates exactly? taking example of a sales department, I work based on order date or delivery date and see if the columns keep their values for a long time, but that doesn't sounds right because:
- First of all, the values can be updated but not dates.
- Secondly, this rule will work only if the database deals with transactions, but if a column has a distinct values this solution won't work, even though if it is a Master Data.
- It is easy to deal with it manually and analyze type of column and then decide if it is a master data or not, but this process should be automated.
I hope my idea is clear and I did clarify the problem I am facing, any peace of advise can really help me a lot, if you have any other method in mind which is better, be my guest!