FileHelpers library escape separator when reading Stream

Question

need help to figure out how handling following scenario.

We have a .NET application using FileHelpers library to parse CSV files. We don't use physical files, instead we have a SQL server database where CSV files are stored in a VARBINARY field. What we have to do is reading binary content using a stream reader and parse it with a proper multirecord engine.

Our CSV files use semicolon (;) as a separator character, but file records may contain escaped separator, where the escape character is exclamation mark (!). Escaped separator characters must be ignored when parsing file. This means that following records must be parsed the same way:

Hello;World;Foo //no escaped separators here

Hello!;;World;Foo!; //two escaped separators here

Both records of previous examples are valid and must produce an object whose properties are "Hello", "World" and "Foo".

Using the answer given to a previous question on this same site (FileHelper escape delimiter) I tried to subscribe public event BeforeReadRecord of MultiRecordEngine class, using an event handler that simply replace all occurences of "!;" with empty string.

This works fine when using MultiRecordEngine.ReadString method: event BeforeReadRecord is fired every time a record is read, event handler is called as expected and escaped separator characters are replaced with empty strings.

Unfortunately, when using MultiRecordEngine to read from a stream, event BeforeReadRecord is not fired at all and event handler code is never executed. I guess there is no way to take advantage of BeforeReadRecord event when reading from stream.

According to FileHelpers library documentations (http://filehelpers.sourceforge.net/FileHelpers.MultiRecordEngineMembers.html) it seems there is no proper event to use in this scenario.

Does anyone know a way to handle escaped separators when reading from a stream ?

Thanks for helping.

Enrico.

Answer 1

Whether you are using ReadFile(string fileName), ReadString(string source) or ReadStream(TextReader reader), it uses the same underlying function ReadStream(IRecordReader reader) to parse the data. The engine must know that it is supposed to notify about reads via MustNotifyRead property. This property has the following code:

return BeforeReadRecord != null ||
       AfterReadRecord != null ||
       RecordInfo.NotifyRead;

This basically means that you must have the BeforeRecordRecord or AfterRecordRecord event subscribed or have INotifyRead against the class that you are reading before it attempts any notifications via OnBeforeReadRecord(e); which only has the following code:

if (RecordInfo.NotifyRead)
    ((INotifyRead)e.Record).BeforeRead(e);

if (BeforeReadRecord != null)
    BeforeReadRecord(this, e);

return e.SkipThisRecord;

This basically means, if you have INotifyRead on the record class, it calls the INotifyRead.BeforeRead() method and/or the BeforeReadRecord event if anything is subscribed.

However, because you are using a dynamic selector, I don't think it checks against the selected record class but against the first type of record in your constructor so you should really implement it on all of them if you try implementing INotifyRead. If you do, and you want to ignore the base class properties, you should set the appropriate ignore attribute at class level.