Circular Dependencies with Preon Binary Parser

Question

I've got a pair of binary files to parse that have dependencies on one another. I'm using the Preon library.

The files are named rec.table and rec.offset. One is an offset index into the other (lines of rec.offset file 32 bit integers, representing byte offsets into rec.table). The number of lines in rec.offset depends on one of the pieces of data available in rec.table (the recordCount field). I need to provide the size of the offset entry list, and I need that list to be used to access record entries, as seen in the @BoundList situations below.

I have the following set up:

package bytecodeparsing;

import org.codehaus.preon.annotation.BoundList;
import org.codehaus.preon.annotation.BoundNumber;

public class RecTable {
    // Using this instead of 'outer' due to circular dependencies
    @SuppressWarnings("unused") // Actually used in Preon expression below
    private RecOffsets recOffsets;

    @BoundNumber int version;
    @BoundList(size="4") byte[] pad; // 4 byte pad to ignore.

    @BoundNumber long recordCount;


    // offset is in RecOffsets, which must then be an outer class of this, but that class needs to know how many entries there
    // are, which is the recordCount value here. Circular dependencies?

    @BoundList(offset="recOffsets.offsets[index]", size="recordCount") Rec[] recs;


    public void setDependencies(RecOffsets recOffsets) {
        this.recOffsets = recOffsets;
    }
}

and:

package bytecodeparsing;

import org.codehaus.preon.annotation.BoundList;
import org.codehaus.preon.annotation.BoundNumber;

public class RecOffsets {

    @SuppressWarnings("unused") // Actually used in Preon expression below
    private RecTable recTable; // maybe use this instead of 'outer' due to circular dependencies?

    @BoundList(size="RecTable.recordCount+1") long[] offsets;

    public void setDependencies(RecTable recTable) {
        this.recTable = recTable;
    }
}

When I run this, I get an error similar to the one reported in the post "Parsing variable record lengths in Preon":

org.codehaus.preon.CodecConstructionException: Failed to construct codec.
    at org.codehaus.preon.codec.ArrayCodecFactory.getSizeExpression(ArrayCodecFactory.java:152)
    at org.codehaus.preon.codec.ArrayCodecFactory.create(ArrayCodecFactory.java:93)
    at org.codehaus.preon.codec.CompoundCodecFactory.create(CompoundCodecFactory.java:59)
    at org.codehaus.preon.DefaultCodecFactory$DecoratingCodecFactory.create(DefaultCodecFactory.java:266)
    at org.codehaus.preon.codec.ObjectCodecFactory.harvestBindings(ObjectCodecFactory.java:184)
    at org.codehaus.preon.codec.ObjectCodecFactory.createCodec(ObjectCodecFactory.java:129)
    at org.codehaus.preon.codec.ObjectCodecFactory.create(ObjectCodecFactory.java:112)
    at org.codehaus.preon.codec.CachingCodecFactory.create(CachingCodecFactory.java:130)
    at org.codehaus.preon.codec.CompoundCodecFactory.create(CompoundCodecFactory.java:59)
    at org.codehaus.preon.DefaultCodecFactory$DecoratingCodecFactory.create(DefaultCodecFactory.java:266)
    at org.codehaus.preon.DefaultCodecFactory.create(DefaultCodecFactory.java:137)
    at org.codehaus.preon.DefaultCodecFactory.create(DefaultCodecFactory.java:65)
    at org.codehaus.preon.DefaultCodecFactory.create(DefaultCodecFactory.java:60)
    at org.codehaus.preon.Codecs.create(Codecs.java:279)
    at bytecodeparsing.PreonParsing.create(PreonParsing.java:76)
    at bytecodeparsing.PreonParsing.<init>(PreonParsing.java:51)
Caused by: org.codehaus.preon.el.BindingException: Failed to create binding for bound data called recTable
    at org.codehaus.preon.codec.BindingsContext.selectAttribute(BindingsContext.java:101)
    at org.codehaus.preon.el.ImplicitsContext.selectAttribute(ImplicitsContext.java:52)
    at org.codehaus.preon.el.LimboWalker.vexpr(LimboWalker.java:667)
    at org.codehaus.preon.el.LimboWalker.vexpr(LimboWalker.java:488)
    at org.codehaus.preon.el.Expressions.arithmetic(Expressions.java:125)
    at org.codehaus.preon.el.Expressions.createInteger(Expressions.java:102)
    at org.codehaus.preon.codec.ArrayCodecFactory.getSizeExpression(ArrayCodecFactory.java:148)
    ... 19 more

This error refers to the usage of recTable in the size attribute of the offset array. As in the other question, I can get past this error by providing a constant size, or faking a simple @BoundNumber binding and referencing that.

In the other question I linked to above, the user has a situation where the circular dependency is across two classes, but while parsing one file, whereas I have to parse two separate files. I assume that the patch that Wilfred Springer mentioned (PREON-9) is already integrated into the github repository (at PREON-48).

How can I deal with this circular dependency? Am I missing a trick?

Answer 1

I found a partial solution to this problem. I was under the very strong impression that the size attribute for the @BoundList was required, but when I looked at Preon source I found it wasn't. Since the binary offset file in question has only the offsets, and no other data following that, it is safe for me to leave size unspecified. Preon deals with unknown list size nicely :)

I am hoping that I do not discover another circular dependency like this in the other half dozen files I am parsing.