How to read FNC1 characters in Java JTextArea

Question

I have a hand held scanner that can read GS1-DataMatrix codes(like the ones in the supermarket). I can scan codes in Notepad++ and I can see FNC1 characters are transmited(the GS before 2210, 1D in HEX - first image)

Now I'm trying to read the same GS1 code from Java but isn't working, the FNC1 is not seen by Java. In Java I only see "01095011010209171719050810ABCD12342110". I transformed the string to HEX but the result is the same, FNC1 is not in HEX either(second image).

This is the test code:

package gs1.datamatrix;

import java.awt.Font;
import java.io.UnsupportedEncodingException;
import java.math.BigInteger;
import java.util.logging.Level;
import java.util.logging.Logger;
import javax.swing.JButton;
import javax.swing.JFrame;
import javax.swing.JTextArea;
import javax.swing.event.DocumentEvent;
import javax.swing.event.DocumentListener;
import javax.swing.text.BadLocationException;
import javax.swing.text.Document;

public class GS1DataMatrix {
    public static void main(String[] args) {
        JFrame f=new JFrame();//creating instance of JFrame  
        Font font = new Font("Courier New", Font.PLAIN, 16);

        JTextArea jtf2 = new JTextArea(); // used to hold the HEX data
        jtf2.setBounds(10,250,900, 200);
        jtf2.setFont( font.deriveFont( 24.0f) );
        jtf2.setLineWrap(true);
        f.add(jtf2);//adding button in JFrame  

        JTextArea jtf1 = new JTextArea(); // scan area for the DataMatrix scanner
        jtf1.setBounds(10,10,900, 200);
        jtf1.setFont( font.deriveFont( 24.0f) );
        jtf1.getDocument().addDocumentListener(new DocumentListener() {
            @Override
            public void insertUpdate(DocumentEvent e) {                update(e);            }
            @Override
            public void removeUpdate(DocumentEvent e) {                update(e);            }
            @Override
            public void changedUpdate(DocumentEvent e) {                update(e);            }
            public void update(DocumentEvent e) {
                try {
                    Document doc = (Document)e.getDocument();
                    String hex = String.format("%040x", new BigInteger(1, doc.getText(0, doc.getLength()).getBytes("UTF8"))); // transform to HEX
                    jtf2.setText(java.util.Arrays.toString(hex.split("(?<=\\G..)"))); // split hex data by 2 characters
                    jtf1.selectAll();
                } catch (Exception ex) {
                    Logger.getLogger(GS1DataMatrix.class.getName()).log(Level.SEVERE, null, ex);
                }
            }
        });
        f.add(jtf1);//adding button in JFrame  

        f.setSize(1000,500);
        f.setLayout(null);
        f.setVisible(true);
    }
}

First image: this is how Notepad++ reads FNC1(GS special character on black background):

Second image: this is Java result:

Third image: Notepad++ hex dump showing FNC1 as 1D in HEX at every scan:

Later edit I think there has been a confusion caused by my original post: I don't parse images, the scanner has build in hardware that does this for me and I only receive a text and some special characters(FNC1's).

Answer 1

Some guesses after reading around a bit:

• FNC1 does not have a standard representation. This stackoverflow answer suggests that there is no way to directly encode FNC1 in the default Latin-1 encoding used for transmission. As a workaround, most readers seem to default to the ASCII control character "Group Separator" (GS, 29, 0x1d).

• You are using a swing control to display and work with the data. Swing is primarily intended for displaying purposes, not for correct data handling purposes.
  I assume what happens is that swing strips the non-printable GS character when it's set within the content of the JTextArea

Considering that you're not terribly explicit about how exactly your scanner transfers the data, but you mention "It's more like a keyboard", I assume the scanner transfers the data by pretending to be a keyboard. You'd be selecting the input, pressing a button on the scanner and it would send data as keypresses.

Now if that is the case, you won't be able to use Swing's DocumentListener/Document to solve this. The following Stack Overflow question basically refers to the same problem that you have (with the difference that they're using a qrcode instead of a barcode): ASCII Non printable characters in textcomponent

Now the question I linked suggests that you can use a KeyBinding or a KeyListener to fix this. Note that this will in some way break the hexadecimal representation, if you want to print the non-printable character.

UTF-8 does have a special codepoint for ASCII non-printable character representations. The "Symbol for Group Separator" is located at \u241d. An option to handle this would then be:

jtf1.getInputMap().put(KeyStroke.getKeyStroke(29), "handleGS");
jtf1.getActionMap().put("handleGS", new AbstractAction() {
    @Override
    public void actionPerformed(ActionEvent e) {
        jtf1.setText(jtf1.getText() + "\u241d");
    }
}

That way the hexadecimal representation should become:

.. , 33, 34, e2, 90, 9d, 32, 31, 31, 30]

Note that because we remapped the GS to Unicode's "SYMBOL_FOR_GS", we get e2, 90, 9d instead of 1d.