Large text file search in larger text file

76 views Asked by At

Been using EmEditor for most of my document handling and its been great over the years. However recently came into another request and now need to use a semi-large searchfile.txt (1GB) to search for alphanumerical values inside an even larger textfile.txt (+500GB). Its basically impossible even though EmEditor claims to be able to "Easily handle files up to 16TB". Clearly there's a limit.

I don't want to split the files and run several times, as that will just be way to boring and requires a lot of job my end. If splitting files is required it would be great if that could be done automatically (script?). Have tried indexing in Windows and that is incredibly fast but its limited to 1 (one) of the 10+ million lines inside the 1GB searchfile.txt, so that won't work.

Its also a possibility I will get this request every few weeks from now on and they are talking about text-files well above 1TB. Maybe as large as 10TB (need to buy a new drive for sure). So any suggestion on how to proceed?

Have tried EmEditor and indexing in Windows search. Also searched online to a solution but so far drew blank.

0

There are 0 answers