I'm using the xz zipping utility on a PBS cluster; I've just realised that the time I've allowed for my zipping jobs won't be long enough, and so would like to restart them (and then, presumably, I'll need to include the .xz that has already been created in the new archive file?). Is it safe to kill the jobs, or is this likely to corrupt the .xz files that have already been created?
1
There are 1 answers
Related Questions in UNIX
- Reading & Writing to the same file from terminal
- `df` command not capturing entire output in perl
- Why is it that when I pass certain directory names to `ls`, sometimes it does not list their contents?
- Detect Mouse Clicks; Terminate Program on Scroll Wheel Movement
- grep expression behaving weird (unix/mac) while reading a conf file
- Unix sub path creation and copy files
- Ignoring folders in The Silver Searcher `ag`
- struct nameidata-Linux Kernel Module
- telegraf service not able to retrieve data using the journalctl commands
- Order of options in Unix sed for editing files in-place
- Trying to echo line variable along with another variable inside a while read loop
- failed to handshake with xxx: authentication error?
- UDP socket client not able to receive data
- Invalid SCA token in unix
- How can I keep randomized UNIX timestamp in specific hour range in SQL?
Related Questions in PBS
- Python script submitting jobs to PBS and then wait until all the jobs are finished
- Configuration is missing run command in eclipse remote debugging
- Submit multiple jobs in PBS script
- On pbs how to move a running job to another node to continue?
- pbsdsh: What happened to PBS_VNODE (and other mysteries)
- Simultaneous installation of slurm and pbs pro
- Is there a way to have PBS notify a service on a job status change?
- Call a bash variable within a R using conda environement
- Is there a way to change the default directory for PBS output and error files while retaining the default names?
- Pass a variable that contains a comma as a -v option to qsub
- How can I measure over time memory and time usage for my Java model run by a PBS script on a cluster?
- Is there any way by which we can save the output and error files of a PBS script with current time as filename?
- stdout stderr not captured from binary executed in singularity container
- How to include aws credentials and configs while submitting PBS script jobs?
- How does GNU parallel detect available memories?
Related Questions in XZ
- How to use XZ / liblxma-dev to compress data using LZMA_FILTER_LZMA1 filters
- Cleanest way skip bytes in .RAW image file
- Where in the .XZ file is stored uncompressed file size?
- How to use Packer with xz images
- "liblzma library and headers are required" when installing R-4.2.2 and dependencies under $HOME
- Removing null bytes from a file results in larger output after XZ compression
- How to Extract tar.xz with password on Google colab?
- tar.xz - how to check uncompressed files size without decompress whole archive
- How to do random access of specific blocks in an .xz file?
- Determining Anaconda XZ Package Content
- unzip xz file into memory and read csv with pandas
- Python 3.8 lzma decompress huge file incremental input and output
- Compressing files through vroom and piping to pixz
- `xz-java` library: Is there any way to use «Extreme» presets?
- how to extract the size of the uncompressed file from xz api?
Popular Questions
- How do I undo the most recent local commits in Git?
- How can I remove a specific item from an array in JavaScript?
- How do I delete a Git branch locally and remotely?
- Find all files containing a specific text (string) on Linux?
- How do I revert a Git repository to a previous commit?
- How do I create an HTML button that acts like a link?
- How do I check out a remote Git branch?
- How do I force "git pull" to overwrite local files?
- How do I list all files of a directory?
- How to check whether a string contains a substring in JavaScript?
- How do I redirect to another webpage?
- How can I iterate over rows in a Pandas DataFrame?
- How do I convert a String to an int in Java?
- Does Python have a string 'contains' substring method?
- How do I check if a string contains a specific word?
Trending Questions
- UIImageView Frame Doesn't Reflect Constraints
- Is it possible to use adb commands to click on a view by finding its ID?
- How to create a new web character symbol recognizable by html/javascript?
- Why isn't my CSS3 animation smooth in Google Chrome (but very smooth on other browsers)?
- Heap Gives Page Fault
- Connect ffmpeg to Visual Studio 2008
- Both Object- and ValueAnimator jumps when Duration is set above API LvL 24
- How to avoid default initialization of objects in std::vector?
- second argument of the command line arguments in a format other than char** argv or char* argv[]
- How to improve efficiency of algorithm which generates next lexicographic permutation?
- Navigating to the another actvity app getting crash in android
- How to read the particular message format in android and store in sqlite database?
- Resetting inventory status after order is cancelled
- Efficiently compute powers of X in SSE/AVX
- Insert into an external database using ajax and php : POST 500 (Internal Server Error)
I am not sure about the implications of using
xzin a cluster, but in general killing anxzprocess (or any decent compression utility) should only affect the file being compressed at the time the process terminates. More specifically:Any output files from input files that have already been compressed should not be affected. The resulting
.xzcompressed files should remain perfectly usable.Any input files that have not been processed yet should not be altered at all.
The input file that was being compressed at the time of termination should not be affected.
Provided that the process is terminated using the
SIGTERMsignal, rather than a signal than cannot be caught likeSIGKILL,xzshould clean-up after itself before exiting. More specifically, it should not leave any partial output files around.If
xzis killed violently, the worst that should (as opposed to might) happen is for a partial compressed file to remain on the disk, right along its corresponding input file. You may want to ensure that such files are cleaned up properly - a good way is to havexzwork in a separate directory from the actual storage area and move files in and out for compression.That said, depending on the importance of the compressed data, you may still want to incorporate measures to detect and deal with any corrupt files. There can be a lot of pathological situations where things do not happen as they are supposed to...