Node.js ArchiverJS library not writing to output filestream in AWS Lambda

871 views Asked by At

I'm trying to zip the contents of a directory using node-archiver as suggested here. The function is running in AWS Lambda in the Node.js runtime environment.

My function is as follows:

function zipDirectory(source, outputTarget) {
    var archive = archiver("zip");
    const stream = fs.createWriteStream(outputTarget, { flags: 'w' });

    stream.on("close", () => {
        console.log(archive.pointer() + ' total bytes');
        console.log('archiver has been finalized and the output file descriptor has closed.');
    });

    return new Promise((resolve, reject) => {
        archive.pipe(stream);
        archive.on("error", err => reject(err))
        archive.directory(source, false);
        archive.finalize();
        stream.close();
        resolve();
    });
}

The result is that the zip is created but the file size is zero:

INFO    0 total bytes
INFO    archiver has been finalized and the output file descriptor has closed.

I'm also unable to extract the archive with the following error.

INFO    Error: end of central directory record signature not found

Notes:

  1. The source directory definitely exists, is populated, and the path is correct.
  2. Everything is being written to the /tmp/ directory to which AWS Lambda allows us read and write access.
  3. I tried uploading the zip binary with the Lambda deployment package and running it as a child process, but apparently, that is not supported because it's a 32-bit package. If you have alternatives that I could use to zip directories outside of node that would work in Lambda's runtime, please do tell.

Appreciate any help, thank you!

1

There are 1 answers

0
doberoi96 On BEST ANSWER

Alright, figured my mistake out.

  1. archive.finalize() returns a promise and the function was returning before the archiving was complete.
  2. Calling stream.close() is unnecessary as archive.finalize() does that for you, so the stream was being closed regardless of whether the archiving was complete or not.

The correct code looks like this:

function zipDirectory(source, outputTarget) {
    var archive = archiver("zip");
    const stream = fs.createWriteStream(outputTarget, { flags: 'w' });

    stream.on("close", () => {
        console.log(archive.pointer() + ' total bytes');
        console.log('archiver has been finalized and the output file descriptor has closed.');
    });

    return new Promise(async (resolve, reject) => {
        archive.pipe(stream);
        archive.on("error", err => reject(err))
        archive.directory(source, false);
        await archive.finalize();
        resolve();
    });
}