Memory error using ncks to set time as record dimension

645 views Asked by At

I am trying to use ncks to set the time as record dimension in a large (14 GB) file and getting the following error

$ ncks -O --mk_rec_dmn time vorticity_1979_1.40625deg.nc test.nc
nco_def_var_chunking(): ERROR Total requested chunk size = 14926479360 exceeds netCDF 
maximium-supported chunk size = 4294967295
nco_err_exit(): ERROR Short NCO-generated message (usually name of function that triggered 
error): nco_def_var_chunking()
nco_err_exit(): ERROR Error code is -127. Translation into English with nc_strerror(-127) is 
"NetCDF: Bad chunk sizes."
nco_err_exit(): ERROR NCO will now exit with system call exit(EXIT_FAILURE)
2

There are 2 answers

0
Manmeet Singh On

I was able to solve this using the following command

ncks -6 -O --mk_rec_dmn time vorticity_1979_1.40625deg.nc test.nc
1
Charlie Zender On

Not sure why it automatically chose too large of a chunk size. Changing format to netCDF-64 bit works because then chunking is not an issue. You could also, I think, keep the format as netCDF4 by explicitly setting the chunk size to that of e.g., a single timestep with

ncks -O --cnk_dmn time,1 --mk_rec_dmn time vorticity_1979_1.40625deg.nc test.nc