Using LZ4HC algorithm in an ARM Cortex-M3 processor

Question

I have this embedded system with a flash memory placed on the board to store a huge number of data. The main controller is an ARM Cortex-M3 processor and I'm supposed to compress the data placed on a part of flash and put the compressed data on another part of the flash.

Now since the amount of SRAM is limited in these kind of systems how exactly can I use the LZ4HC algorithm? I can't compress the whole data at once like we do in PC and I guess I have to do this on a little chunk of data or block by block (for example every 512 or 4096 bytes of data). I'm just not sure how. I couldn't totally understand the functions.
Is that even possible to do this block by block?
I couldn't find any example. And the open source code does not come with a good documentation. Actually I think there is no documentation.

Answer 1

I'd recommend you use a library specific for embedded systems. These libraries usually use low amount of memory and are designed to compress small chunks of data per cycle.

If you mustn't use LZ4HC or if you want to implement your own library, a good start point is heatshrink, a LZSS based library for embedded systems.

There is also a LZ4 decompression implementation in assembly for ARM processors here.

Answer 2

Yes, you can do it a chunk at a time. Yes, there is documentation. See, for example how to handle a huge file in small amount of memory. lz4hc.h defines a maximum compression level of 12.