Documenting Internal Structures of v4 CTree Database.. parsing binary?

Question

I have a very old (circa 1993, v4 Faircom CTree Database) that was used by a now defunct test equipment manufacturer. I have no access to original configuration but there only appear to be two tables (two files) and then two other matching files which I assume are their sibling index files.

Im attempting to parse out these old files in a newer C# program to load the data for other purposes so Im looking at the binary.... I see how they are structured, there is a header and a variety of pointers (this was written and compiled with Turbo C) and I even see my data in there as text.

binary snapshot picture #1

Taking what is shown above, you can see that the offset to the record data is the second word it the main file header. Now we get into the record specific structure...

binary snapshot picture #2

4 Records shown here - each a different color

1st Word - two matching bytes signifying the record type? (FA,FB,FC,FD,FE,FF=end)

2nd Word - IS the length of the record (after 6 header bytes).

3rd Word - Unknown (sometimes matches the 2nd word, sometimes zero, somtimes something else)

Actual questions for this post:

1. It appears that I will need to ascertain the fields for the records myself by parsing through a handful of different data files to figure out lengths. Does anyone know if there are free tools out there that can 'understand' Faircom database structures? I have contacted Faircom and they will help me if I buy a support contract, but I wasn't totally expecting to spend money on this, but I may have to.
2. I tried to find C headers to get to the binary emitted in these old versions, but I just can't find the stuff this low. Anyone have any pointers or knowledge around Faircom DB internals or super low level reference info. Understanding what those 0xFA-0xFF enums would be handy too.

Answer 1

Okay, just to sorta wrap this up, here is where I landed.. in the end, I just parsed the binary myself into relevant records because I did not have the original schema files which would have been needed to use any of the Faircom stuff. The task of recreating the files would have taken just as much time as parsing the binary in C# since that is pretty straightforward.

I did see that the 0xFA records appeared to be 'real' records while the 0xFD records signified that they were 'deleted'. The original program did have the ability to 'revert' a file if you have changed it and not yet saved, so the faircom database structures allowed that to be pretty turnkey back in 1994.

Other general observations which may or may not be specific to my schema... I had the following data types... String, StringArray, ByteArray, Number

I created a little SchemaDefinition that looks like this..

schema = new CTreeSchema();
schema.Columns.Add(new CTreeColumn("Ordinal", CTreeColumnType.Number, 4));
schema.Columns.Add(new CTreeColumn("Unknown3", CTreeColumnType.Number, 4));
schema.Columns.Add(new CTreeColumn("Unknown4", CTreeColumnType.Number, 4));
schema.Columns.Add(new CTreeColumn("Name", CTreeColumnType.String, 0));
schema.Columns.Add(new CTreeColumn("ExtendedData", CTreeColumnType.StringArray, 0));

and then I parsed it using something like this...

public CTreeRecord(CTreeSchema schema, byte[] bytes)
    {
        _internalBytes = bytes;
        RecordTypeId = bytes[0];
        int byteIndex = 6;
        foreach(var i in schema.Columns)
        {
            int endIndex = 0;
            switch (i.Type)
            {
                case CTreeColumn.CTreeColumnType.String:
                    if (i.Length == 0)
                    {
                        //null terminated
                        endIndex = Array.IndexOf<byte>(bytes, 0x00, byteIndex);
                        i.Value = Encoding.ASCII.GetString(bytes.Skip(byteIndex).Take(endIndex - byteIndex).ToArray());
                        byteIndex += (1+ endIndex - byteIndex);
                    }
                    if (i.Length > 0)
                    {
                        //specific length
                        throw new NotSupportedException("Static Length String columns not supported.");
                        //byteIndex += i.Length;
                    }
                    break;
                case CTreeColumn.CTreeColumnType.StringArray:
                    List<string> headerStrings = new List<string>();
                    endIndex = Array.IndexOf<byte>(bytes, 0x00, byteIndex);
                    byte[] arrayBytes = bytes.Skip(byteIndex).Take(endIndex - byteIndex).ToArray();
                    byteIndex += (1 + endIndex - byteIndex);
                    List<byte[]> headerBytes = SplitByteString(arrayBytes, 0x01, false);
                    foreach (byte[] b in headerBytes)
                    {
                        headerStrings.Add(Encoding.ASCII.GetString(b));
                    }
                    i.Value = headerStrings;
                    break;
                case CTreeColumn.CTreeColumnType.ByteArray:
                    List<byte[]> byteArray = new List<byte[]>();
                    for (int a = 0; a < i.Length; a++)
                    {
                        endIndex = Array.IndexOf<byte>(bytes, 0x00, byteIndex);
                        byteArray.Add(bytes.Skip(byteIndex).Take(endIndex - byteIndex).ToArray());
                        byteIndex += (1 + endIndex - byteIndex);
                    }
                    i.Value = byteArray;
                    break;
                case CTreeColumn.CTreeColumnType.Number:
                    switch (i.Length)
                    {
                        case 2:
                            i.Value = BitConverter.ToUInt16(bytes, byteIndex);
                            break;
                        case 4:
                            i.Value = BitConverter.ToUInt32(bytes, byteIndex);
                            break;
                    }
                    byteIndex += i.Length;
                    break;
                case CTreeColumn.CTreeColumnType.Date:
                    throw new NotSupportedException("Date columns not supported.");
                    break;
            }
            Columns.Add(i);
        }
    }

    private List<byte[]> SplitByteString(byte[] bytes, byte splitValue, bool removeEmptyEntries)
    {
        List<byte[]> splitBytes = new List<byte[]>();
        int currentIndex = 0;
        while (currentIndex < bytes.Length)
        {
            int splitIndex = Array.IndexOf<byte>(bytes, splitValue, currentIndex);
            if (splitIndex >= 0)
            {
                //found one
                int currentLength = splitIndex - currentIndex;
                if (!(currentLength == 0 && removeEmptyEntries))
                {
                    splitBytes.Add(bytes.Skip(currentIndex).Take(currentLength).ToArray());
                    currentIndex += (1 + currentLength);
                }
                else
                {
                    currentIndex++;
                }
            }
            else
            {
                //not found, just take until end now..
                splitBytes.Add(bytes.Skip(currentIndex).Take(bytes.Length - currentIndex).ToArray());
                currentIndex += (bytes.Length - currentIndex);
            }
            
        }
        return splitBytes;
    }

All in all, it is pretty ugly and very specific to this application, but if someone has to deal with C Faircom database files, this might provide some sanity.