Untar gzip to different directories

Question

I have a .tar.gz file which may have the following files:

folder1/folder2/folder3/imp_folder1/file11.jpg
folder1/folder2/folder3/imp_folder1/file12.jpg
folder1/folder2/folder3/imp_folder2/file21.jpg
folder1/folder2/folder3/imp_folder3/file31.jpg
...
...

I want to untar it to the following directories:

/new_folder1/new_folder2/imp_folder1/file11.jpg
/new_folder1/new_folder2/imp_folder1/file12.jpg
/new_folder1/new_folder2/imp_folder2/file21.jpg
/new_folder1/new_folder2/imp_folder3/file31.jpg
...
...

Basically, "folder1/folder2/folder3/" should be replaced by "/new_folder1/new_folder2/". And, if the "imp" directories are not present, then I have to create them

Right now I have an implementation that loops through all the members in the tar and creates the folder names and then does the following

input_file  = tar.extractfile (member)
with open (image_path_local, 'w') as output_file:
     output_file.write(input_file.read())
input_file.close()

This process is too slow. Since there are many files(in order of 100k) what will be the fastest way to achieve this?

Answer 1

You need to use the --transform option for tar. This posting discussed the usage of that option for a similar problem.

Here is a demo of the option's usage:

#!/bin/sh

### Create test data
TESTDIR="TEST_2"

mkdir "${TESTDIR}"
for i in ab DF jkL
do
    echo "${i}" >"${TESTDIR}/testFile_${i}.txt"
done

echo "\n Show test directory contents:"
ls -l "${TESTDIR}"

echo "\n Report tar file creation:"
tar cvf "${TESTDIR}.tar" "${TESTDIR}"

### Demonstrate mapping of content path string

echo "\n Report tar file extraction: (NOTE: no name change visible)"
tar --transform='s+^TEST_2+TEST-3+' -xvpf "${TESTDIR}.tar"

echo "\n Show extracted in new location:"
ls -lR TEST-3

The session output for that script is as follows:

ericthered@OasisMega1:/0__WORK$ ./test_47.sh

 Show test directory contents:
total 12
-rw-rw-r-- 1 ericthered ericthered 3 Nov 12 17:22 testFile_ab.txt
-rw-rw-r-- 1 ericthered ericthered 3 Nov 12 17:22 testFile_DF.txt
-rw-rw-r-- 1 ericthered ericthered 4 Nov 12 17:22 testFile_jkL.txt

 Report tar file creation:
TEST_2/
TEST_2/testFile_jkL.txt
TEST_2/testFile_DF.txt
TEST_2/testFile_ab.txt

 Report tar file extraction: (NOTE: no name change visible during extract)
TEST_2/
TEST_2/testFile_jkL.txt
TEST_2/testFile_DF.txt
TEST_2/testFile_ab.txt

 Show extracted in new location:
TEST-3:
total 12
-rw-rw-r-- 1 ericthered ericthered 3 Nov 12 17:22 testFile_ab.txt
-rw-rw-r-- 1 ericthered ericthered 3 Nov 12 17:22 testFile_DF.txt
-rw-rw-r-- 1 ericthered ericthered 4 Nov 12 17:22 testFile_jkL.txt
ericthered@OasisMega1:/0__WORK$