multiple compressor for single array

Question

Is it possible to have different compressors, e.g. lossy and lossless for individual chunks?

In a scenario, where you have a mask of importance, where you want to keep signal with lossless compression or even with no compression, but have other parts of the signal with lossy compression for efficiency and space.

For example we have:

import zarr
z = zarr.zeros((32, 32), chunks=(4, 4))

important region that we want to keep is A(4:11,4:11), where we want to go lossless, e.g. zlib, then for the rest we use quantize from numcodecs for lossy. So we will have high precision for interesting part within mask and have lossy compression for out of mask and have two different compressor for different parts of a single array at chunk level.

Answer 1

This is not currently possible. The compressor interface would have to receive coordinates for encode(). Then you could implement a compressor that would decide to lose information on encoding depending on coordinates. Since compressors operate on chunks, you'd have to choose the chunking so it aligns with the boundaries where you want to change fidelity.

Overall I think you'll have an easier time just writing a wrapper that combines several zarr stores for the different fidelities and overlay them on access and write.