How to zstd compress a chunk of bytes from an input file?

Question

Not enough example on zstd compression. I am using zstandard 0.8.1, trying to compress 2 bytes at a time. Came across https://anaconda.org/rolando/zstandard on using write_to(fh), but not sure how to use it. Below is my partial code trying to read a chuck bytes from a file, then compresses each chuck, cctx = zstd.ZstdCompressor(level=4) with open(path, 'rb') as fh: while True: bin_data = fh.read(2) #read 2 bytes if not bin_data: break compressed = cctx.compress(bin_data) fh.close()

with open(path, 'rb') as fh:
    with open(outpath, 'wb') as outfile:
        outfile.write(compressed)
        ...

But how shall i use the write_to()?

Answer 1

I think I found the right way to use zstd 0.8.1 module for streaming chunk of bytes:

with open(filename, 'wb') as fh:
    cctx = zstd.ZstdCompressor(level=4)

    with cctx.write_to(fh) as compressor:
        compressor.write(b'data1')
        compressor.write(b'data2')        

with open(filename, 'rb') as fh:
    cctx = zstd.ZstdCompressor(level=4)

    for chunk in cctx.read_from(fh, read_size=128, write_size=128):
        #do something