How to pipe stdout/stderr to file from worker in Deno?

581 views Asked by At

Suppose I have the following snippet to create a worker in Deno and start it.

const worker = new Worker("./worker.ts", { type: "module", deno: true });
worker.postMessage({ command: "START" });

Is there a way to pipe stdout/stderr from the worker to a file, similarly to how Deno.run works? For example, is something like this possible?

const file = await Deno.open('/path/to/file.log', { write: true });
const worker = new Worker("./worker.ts", {
  type: "module",
  deno: true,
  stdout: file.rid,
  stderr: file.rid
});
worker.postMessage({ command: "START" });

Or, is it possible to use a stream?

import { WritableStream } from "https://denopkg.com/keroxp/deno-streams/writable_stream.ts"
const stream = new WritableStream<number>({
    write: chunk => {
        ...
    }
})
const worker = new Worker("./worker.ts", {
  type: "module",
  deno: true,
  stdout: stream,
  stderr: stream
});
worker.postMessage({ command: "START" });
1

There are 1 answers

0
Marcos Casagrande On

Currently transfering streams is not supported:

https://github.com/whatwg/streams/blob/main/transferable-streams-explainer.md

Once it is, you'll be able to transfer a WritableStream/ReadableStream/TransformStream to the worker using the following syntax.

const ws = new WritableStream({
  // implement
});
worker.postMessage(ws, [ws]);

As a temporary solution, until it's supported, you can use MessageChannel to represent stdout & stderr

main

const worker = new Worker(new URL("./worker.js", import.meta.url).href, {
  type: "module",
  deno: true
});

const stdout = new MessageChannel();
const stderr = new MessageChannel();

worker.postMessage({ command: 'stdout' }, [stdout.port1])
worker.postMessage({ command: 'stderr' }, [stderr.port1])
worker.postMessage({ command: "START" });

stdout.port2.onmessage = (stderr) => {
    // write to your stream, or whatever you need
    console.log('stdout:', stderr.data);
}
stderr.port2.onmessage = (stderr) => {
    console.log('stderr:', stderr.data);
}

worker.onmessage = (e) => {
    console.log('onMessage: Message received from worker');
}

worker

let stdout;
let stderr;

function delay(ms) {
    return new Promise(resolve => setTimeout(resolve, ms));
}

async function start() {
    while(true) {
        await delay(1000);
        stdout.postMessage(`some data: ${new Date()}`);
        stderr.postMessage(new Error('foo'));

        self.postMessage('other messages');
    }
}

self.onmessage = (evt) => {
    if(evt.data.command === 'stdout')
        [stdout] = evt.ports;
    else if(evt.data.command === 'stderr')
        [stderr] = evt.ports;
    else if(evt.data.command === 'START')
        start();
};