ZModem implementation in C# - Problems with bigger files caused by ZPOS

Question

I have a problem with my ZModem implementation when I try to send files with more than 16kb.

I traced the communication with WireShark and also compared my data with the one TeraTerm is sending.

TeraTerm, for instance, is capable to send every file size. The problem I am facing is that at some point the receiving device sends me a ZPOS request back.

When I try to resend data from the given Offset my data stream gets corrupt. Every further data packet results in that ZPOS response.

Here's the complete communication I traced with WireShark.

Invoke receiver with rz\r command.

0000   1b 00 30 b5 45 fb 01 a2 ff ff 00 00 00 00 09 00   ..0µEû.¢ÿÿ......
0010   00 03 00 14 00 01 03 03 00 00 00 72 7a 0d         ...........rz.

Initiate session

0000   1b 00 70 b1 f0 15 02 a2 ff ff 00 00 00 00 09 00   ..p±ð..¢ÿÿ......
0010   00 03 00 14 00 01 03 14 00 00 00 2a 2a 18 42 30   ...........**.B0
0020   30 30 30 30 30 30 30 30 30 30 30 30 30 0d 0a      0000000000000..

Response from device

0000   1b 00 90 01 5b fb 01 a2 ff ff 00 00 00 00 09 00   ....[û.¢ÿÿ......
0010   01 03 00 14 00 81 03 15 00 00 00 2a 2a 18 42 30   ...........**.B0
0020   31 30 30 30 30 30 30 32 33 62 65 35 30 0d 8a 11   100000023be50...

Then I send the file header with filename and file information.

0000   1b 00 b0 fa b3 00 02 a2 ff ff 00 00 00 00 09 00   ..°ú³..¢ÿÿ......
0010   00 03 00 14 00 01 03 0a 00 00 00 2a 18 41 04 00   ...........*.A..
0020   00 01 01 aa 16                                    ...ª.

0000   1b 00 b0 f1 a1 08 02 a2 ff ff 00 00 00 00 09 00   ..°ñ¡..¢ÿÿ......
0010   00 03 00 14 00 01 03 20 00 00 00 73 61 6d 70 6c   ....... ...sampl
0020   65 2e 74 78 74 00 32 31 35 30 31 20 31 33 33 37   e.txt.21501 1337
0030   37 35 30 31 33 33 32 18 6b 83 e5                  7501332.k.å

The response from the device tells me it is ready to receive files.

0000   1b 00 90 01 5b fb 01 a2 ff ff 00 00 00 00 09 00   ....[û.¢ÿÿ......
0010   01 03 00 14 00 81 03 15 00 00 00 2a 2a 18 42 30   ...........**.B0
0020   39 30 30 30 30 30 30 30 30 61 38 37 63 0d 8a 11   900000000a87c...

After that I send the ZData header and all subpackets.

0000   1b 00 b0 f1 a1 08 02 a2 ff ff 00 00 00 00 09 00   ..°ñ¡..¢ÿÿ......
0010   00 03 00 14 00 01 03 0b 00 00 00 2a 18 41 18 4a   ...........*.A.J
0020   00 00 00 00 46 ae                                 ....F®

When I try to upload a bigger file I get a ZRPOS response back during the upload. Regarding to the ZModem specification, the receiver tells me to resume at the given offset.

0000   1b 00 90 01 5b fb 01 a2 ff ff 00 00 00 00 09 00   ....[û.¢ÿÿ......
0010   01 03 00 14 00 81 03 15 00 00 00 2a 2a 18 42 30   ...........**.B0
0020   39 30 30 34 30 30 30 30 30 62 35 64 31 0d 8a 11   900400000b5d1...

But every further subpacket gets the same response from the receiving device.

Here's the implementation I made that is responsible to send the file.

private void SendZDATAPackets(byte[] src, CRC16CCIT crcCalculator)
{
    // Chunk size
    var chunkSize = 1024;

    // Create ZDATA header
    var zdataHeaderQueue = new Queue<byte>();
    var zdataHeaderCommand = Utils.BuildDataCommand(HeaderType.ZDATA, 0, 0, 0, 0);

    foreach (var c in zdataHeaderCommand)
    {
        zdataHeaderQueue.Enqueue((byte)c);
    }

    // Send ZDATA header
    SendCommand(zdataHeaderQueue.ToArray());

    ResponseHeader zdataResponse = null;

    // Slice binary data into chunks.
    for (int i = 0; i < src.Length; i += chunkSize)
    {
        // Slice data
        var dataSlice = src
            .Skip(i)
            .Take(chunkSize)
            .ToArray();

        // Send ZCRCG, if its the last part of the data send ZCRCE
        var requiredSequence = (i + dataSlice.Length) < src.Length ? ZDLESequence.ZCRCG : ZDLESequence.ZCRCE;

        if (zdataResponse?.ZHeader == HeaderType.ZRPOS)
        {
            /*
             * A ZRPOS header resets the sender's file offset to the correct position.
             * If possible, the sender should purge its output buffers and/or networks
             * of all unprocessed output data, to minimize the amount of unwanted data
             * the receiver must discard before receiving data starting at the correct
             * file offset. The next transmitted data frame should be a ZCRCW frame
             * followed by a wait to guarantee complete flushing of the network's memory.
             */
            SerialPort.DiscardOutBuffer();

            dataSlice = src
                .Skip(zdataResponse.RequestedOffset)
                .Take(chunkSize)
                .ToArray();

            requiredSequence = ZDLESequence.ZCRCW;
        }

        // Create data queue for the sliced data.
        var queue = new Queue<byte>(dataSlice);

        // Calculate CRC
        var dataForCrcCalculation = dataSlice.Concat(new byte[] { (byte)requiredSequence }).ToArray();
        var crc = crcCalculator.ComputeChecksumAsBytes(dataForCrcCalculation);

        // Add crc
        queue.Enqueue((byte)ControlBytes.ZDLE);
        queue.Enqueue((byte)requiredSequence);
        queue.Enqueue(crc[0]);
        queue.Enqueue(crc[1]);

        // Send data
        var zdataCommand = queue.ToArray();
        zdataResponse = SendCommand(zdataCommand);
    }
}

Does anyone had the same problem during the implementation and could point me in some direction to solve this problem?

Thanks!

Answer 1

In case anyone is in need of a native ZModem implementation in C# I managed to make it work now.

The problem was that I had no ZDLE encoding for the bytes I was sending. ZModem requires to encode the data packets as well as the CRC attached at the end to escape certain characters. However, if you use CRC16 you don't need encoding for CRC.

I created a GitHub repository so anyone can use it. The functionality however is limited to upload data to a remote device. Implementing a download should be no problem though.

https://github.com/datoml/zmodem-dotnet-standard