Parsing input file with DFDL page by page

491 views Asked by At

I have a very simple row based document in which each row comprises a record. I want to have a DFDL that parses it into chunks that contains fixed number of rows(say, for example 3 records at each chunk).

Original File:

record1
record2
record3
record4
record5
record6
record7

After DFDL Parse:

1) [record1, record2, record3]
2) [record4, record5, record6]
3) [record7]

I am currently able to get all records at once with the following DFDL, but it creates a serious problem when the size of document gets bigger, that' s why i want to get these records page by page. Is it possible to do this? Does anyone have any idea how can this be done?

<?xml version="1.0" encoding="UTF-8"?><xsd:schema xmlns:dfdl="http://www.ogf.org/dfdl/dfdl-1.0/" xmlns:ibmDfdlExtn="http://www.ibm.com/dfdl/extensions" xmlns:ibmSchExtn="http://www.ibm.com/schema/extensions" xmlns:recSepFieldsFmt="http://www.ibm.com/dfdl/RecordSeparatedFieldFormat" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
        <xsd:import namespace="http://www.ibm.com/dfdl/RecordSeparatedFieldFormat" schemaLocation="../IBMdefined/RecordSeparatedFieldFormat.xsd"/>
    <xsd:annotation>
        <xsd:appinfo source="http://www.ogf.org/dfdl/">
            <dfdl:format byteOrder="{$dfdl:byteOrder}" encoding="{$dfdl:encoding}" escapeSchemeRef="recSepFieldsFmt:RecordEscapeScheme" occursCountKind="fixed" ref="recSepFieldsFmt:RecordSeparatedFieldsFormat"/>
        </xsd:appinfo>
    </xsd:annotation>

    <xsd:element dfdl:encoding="{$dfdl:encoding}" ibmSchExtn:docRoot="true" name="MM1">
        <xsd:complexType>
            <xsd:sequence dfdl:separator="%CR;%LF;%WSP*;" dfdl:terminator="">
                                                                                                                                                                                                                                                                    <xsd:element dfdl:alignment="1" dfdl:escapeSchemeRef="" dfdl:lengthKind="delimited" dfdl:occursCountKind="implicit" maxOccurs="unbounded" name="body" type="xsd:string">
                </xsd:element>
            </xsd:sequence>
        </xsd:complexType>
    </xsd:element>

</xsd:schema>

Thanks

1

There are 1 answers

0
Steve Hanson On

<xsd:element ibmSchExtn:docRoot="true" name="MM1">
    <xsd:complexType>
        <xsd:sequence dfdl:separator="" dfdl:terminator="">
            <xsd:element dfdl:occursCountKind="implicit" maxOccurs="unbounded" name="Chunk">
             <xsd:complexType>
                 <xsd:sequence dfdl:separator="%CR;%LF;%WSP*;" dfdl:terminator="">
                    <xsd:element dfdl:lengthKind="delimited" dfdl:occursCountKind="implicit" maxOccurs="3" name="Body" type="xsd:string">
                    </xsd:element>
                 </xsd:sequence>
              </xsd:complexType>
           </xsd:element>
        </xsd:sequence>
    </xsd:complexType>
</xsd:element>