Turn flat file into relational models

Question

I get flat files once a week that have a few 100k rows. I would like to turn them into a relational model system in Rails since there are about five columns that are fairly static and would make sense for a different model that would then be linked back into the main table as a foreign key.

Is there was a quick way to check if an entry already exists, and if so, look it up and put the foreign key in the main model, and if not, create a new entry in the second model and then reference it in the main model.

I can turn the above paragraph into code but wanted to know if there is a simple 'few lines' Ruby or Rails implementation.

Answer 1

Just for the sake of merging my comment and @anton-z 's.

You can use activerecord-import for doing bulk operations, and ActiveRecord's find_or_create_by to do the checking.