Is there an smart way to write a fixed length flat file?

Question

Is there any framework/library to help writing fixed length flat files in java?

I want to write a collection of beans/entities into a flat file without worrying with convertions, padding, alignment, fillers, etcs

For example, I'd like to parse a bean like:

public class Entity{
    String name = "name"; // length = 10; align left; fill with spaces
    Integer id = 123; // length = 5; align left; fill with spaces
    Integer serial = 321 // length = 5; align to right; fill with '0'
    Date register = new Date();// length = 8; convert to yyyyMMdd
}

... into ...

name      123  0032120110505
mikhas    5000 0122120110504
superuser 1    0000120101231

...

Answer 1

If you are still looking for a framework, check out BeanIO at http://www.beanio.org

Answer 2

The library Fixedformat4j is a pretty neat tool to do exactly this: http://fixedformat4j.ancientprogramming.com/

Answer 3

Spring Batch has a FlatFileItemWriter, but that won't help you unless you use the whole Spring Batch API.

---

But apart from that, I'd say you just need a library that makes writing to files easy (unless you want to write the whole IO code yourself).

Two that come to mind are:

Guava

Files.write(stringData, file, Charsets.UTF_8);

Commons / IO

FileUtils.writeStringToFile(file, stringData, "UTF-8");

Answer 4

Don't know of any frame work but you can just use RandomAccessFile. You can position the file pointer to anywhere in the file to do your reads and writes.

Answer 5

A simple way to write beans/entities to a flat file is to use ObjectOutputStream.

public static void writeToFile(File file, Serializable object) throws IOException {
    ObjectOutputStream oos = new ObjectOutputStream(new FileOutputStream(file));
    oos.writeObject(object);
    oos.close();
}

---

You can write to a fixed length flat file with

FileUtils.writeByteArrayToFile(new File(filename), new byte[length]);

You need to be more specific about what you want to do with the file. ;)

Answer 6

I've just find a nice library that I'm using:
http://sourceforge.net/apps/trac/ffpojo/wiki

Very simple to configurate with XML or annotations!

Answer 7

You're not likely to encounter a framework that can cope with a "Legacy" system's format. In most cases, Legacy systems don't use standard formats, but frameworks expect them. As a maintainer of legacy COBOL systems and Java/Groovy convert, I encounter this mismatch frequently. "Worrying with conversions, padding, alignment, fillers, etcs" is primarily what you do when dealing with a legacy system. Of course, you can encapsulate some of it away into handy helpers. But most likely, you'll need to get real familiar with java.util.Formatter.

For example, you might use the Decorator pattern to create decorators to do the conversion. Below is a bit of groovy (easily convertible into Java):

class Entity{
    String name = "name"; // length = 10; align left; fill with spaces
    Integer id = 123; // length = 5; align left; fill with spaces
    Integer serial = 321 // length = 5; align to right; fill with '0'
    Date register = new Date();// length = 8; convert to yyyyMMdd
}

class EntityLegacyDecorator {
     Entity d
     EntityLegacyDecorator(Entity d) { this.d = d }

     String asRecord() {
         return String.format('%-10s%-5d%05d%tY%<tm%<td',
                               d.name,d.id,d.serial,d.register)
   }
 }

def e = new Entity(name: 'name', id: 123, serial: 321, register: new Date('2011/05/06'))

assert new EntityLegacyDecorator(e).asRecord() == 'name      123  0032120110506'

This is workable if you don't have too many of these and the objects aren't too complex. But pretty quickly the format string gets intolerable. Then you might want decorators for Date, like:

class DateYMD {
     Date d
     DateYMD(d) { this.d = d }
     String toString() { return d.format('yyyyMMdd') }
 }

so you can format with %s:

    String asRecord() {
         return String.format('%-10s%-5d%05d%s',
                               d.name,d.id,d.serial,new DateYMD(d.register))
   }

But for significant number of bean properties, the string is still too gross, so you want something that understands columns and lengths that looks like the COBOL spec you were handed, so you'll write something like this:

 class RecordBuilder {

    final StringBuilder record 

    RecordBuilder(recordSize) {
        record = new StringBuilder(recordSize)
        record.setLength(recordSize)
     }

    def setField(pos,length,String s) { 
       record.replace(pos - 1, pos + length, s.padRight(length))
    }

    def setField(pos,length,Date d) { 
      setField(pos,length, new DateYMD(d).toString())
    }

   def setField(pos,length, Integer i, boolean padded) { 
       if (padded) 
          setField(pos,length, String.format("%0" + length + "d",i))
       else 
          setField(pos,length, String.format("%-" + length + "d",i))
    }

    String toString() { record.toString() }
}

class EntityLegacyDecorator {

     Entity d

     EntityLegacyDecorator(Entity d) { this.d = d }

     String asRecord() {
         RecordBuilder record = new RecordBuilder(28)
         record.setField(1,10,d.name)
         record.setField(11,5,d.id,false)
         record.setField(16,5,d.serial,true)
         record.setField(21,8,d.register)
         return record.toString()
     }

}

After you've written enough setField() methods to handle you legacy system, you'll briefly consider posting it on GitHub as a "framework" so the next poor sap doesn't have to to it again. But then you'll consider all the ridiculous ways you've seen COBOL store a "date" (MMDDYY, YYMMDD, YYDDD, YYYYDDD) and numerics (assumed decimal, explicit decimal, sign as trailing separate or sign as leading floating character). Then you'll realize why nobody has produced a good framework for this and occasionally post bits of your production code into SO as an example... ;)

Answer 8

uniVocity-parsers goes a long way to support tricky fixed-width formats, including lines with different fields, paddings, etc.

Check out this example to write imaginary client & accounts details. This uses a lookahead value to identify which format to use when writing a row:

    FixedWidthFields accountFields = new FixedWidthFields();
    accountFields.addField("ID", 10); //account ID has length of 10
    accountFields.addField("Bank", 8); //bank name has length of 8
    accountFields.addField("AccountNumber", 15); //etc
    accountFields.addField("Swift", 12);

    //Format for clients' records
    FixedWidthFields clientFields = new FixedWidthFields();
    clientFields.addField("Lookahead", 5); //clients have their lookahead in a separate column
    clientFields.addField("ClientID", 15, FieldAlignment.RIGHT, '0'); //let's pad client ID's with leading zeroes.
    clientFields.addField("Name", 20);

    FixedWidthWriterSettings settings = new FixedWidthWriterSettings();
    settings.getFormat().setLineSeparator("\n");
    settings.getFormat().setPadding('_');

    //If a record starts with C#, it's a client record, so we associate "C#" with the client format.
    settings.addFormatForLookahead("C#", clientFields);

    //Rows starting with #A should be written using the account format
    settings.addFormatForLookahead("A#", accountFields);

    StringWriter out = new StringWriter();

    //Let's write
    FixedWidthWriter writer = new FixedWidthWriter(out, settings);

    writer.writeRow(new Object[]{"C#",23234, "Miss Foo"});
    writer.writeRow(new Object[]{"A#23234", "HSBC", "123433-000", "HSBCAUS"});
    writer.writeRow(new Object[]{"A#234", "HSBC", "222343-130", "HSBCCAD"});
    writer.writeRow(new Object[]{"C#",322, "Mr Bar"});
    writer.writeRow(new Object[]{"A#1234", "CITI", "213343-130", "CITICAD"});

    writer.close();

    System.out.println(out.toString());

The output will be:

C#___000000000023234Miss Foo____________
A#23234___HSBC____123433-000_____HSBCAUS_____
A#234_____HSBC____222343-130_____HSBCCAD_____
C#___000000000000322Mr Bar______________
A#1234____CITI____213343-130_____CITICAD_____

This is just a rough example. There are many other options available, including support for annotated java beans, which you can find here.

Disclosure: I'm the author of this library, it's open-source and free (Apache 2.0 License)

Answer 9

Try FFPOJO API as it has everything which you need to create a flat file with fixed lengths and also it will convert a file to an object and vice versa.

@PositionalRecord
public class CFTimeStamp {

    String timeStamp;

    public CFTimeStamp(String timeStamp) {
        this.timeStamp = timeStamp;
    }

    @PositionalField(initialPosition = 1, finalPosition = 26, paddingAlign = PaddingAlign.RIGHT, paddingCharacter = '0')
    public String getTimeStamp() {
        return timeStamp;
    }

    @Override
    public String toString() {
        try {
            FFPojoHelper ffPojo = FFPojoHelper.getInstance();
            return ffPojo.parseToText(this);
        } catch (FFPojoException ex) {
            trsLogger.error(ex.getMessage(), ex);
        }
        return null;
    }
}