For some context, I need to quickly render a small video (6.5 seconds long, at 15 FPS), and send it in a Discord channel with discord.js. I don't want to ever have to write anything to disk, because that'll slow it down and I just don't need it downloaded. So far, I was able to write the video to disk, but now I want to skip that step and send the video Buffer straight to discord.js. I was also able to output the video from ffmpeg to Discord as an audio file, but when I try to use the .mp4 format, I get a "Conversion failed!" error.
I rendered the individual frames for the video using the canvas module, and export all of them as pngs using canvas.toBuffer("image/png");
, and then push all the frames to an array. Then I combine the frames to one Buffer using Buffer.concat()
, and then create a ReadableStream from the nodejs stream module. I also needed to write a custom WritableStream class that implements the _write
method. Here's the class:
class MyWritable extends Stream.Writable {
constructor() {
super();
this.buffer = Buffer.from([]);
}
_write(chunk, encoding, callback) {
this.buffer = Buffer.concat([this.buffer, chunk]);
callback();
}
}
And here's how I implement everything using fluent-ffmpeg:
const allFrames = Buffer.concat(framesArray);
const readable = Stream.Readable.from(allFrames);
const writable = new MyWritable();
const output = await (new Promise((resolve, reject) => {
ffmpeg()
.input(readableStream)
.inputOptions([
`-framerate 15`,
])
.input("path_to_audio_file.mp3")
.videoCodec("libx264")
.format("mp4")
.outputOptions([
"-pix_fmt yuv420p"
])
.duration(6.5)
.fps(15)
.writeToStream(writable)
.on("finish", () => {
// this is never reached, but it should resolve the promise with all the data that was written to the WritableStream (which should be the video)
resolve(writable.buffer);
})
.on("error", (err) => {
// this is never reached also
reject(err);
})
}))
// then I try to send the output buffer to Discord as an attachment, but I don't ever get here anyway
And here's the error I get:
node:events:491
throw er; // Unhandled 'error' event
^
Error: ffmpeg exited with code 1: Conversion failed!
at ChildProcess.<anonymous> (/Users/isaac/Documents/Github/DiscordBot/node_modules/fluent-ffmpeg/lib/processor.js:182:22)
at ChildProcess.emit (node:events:513:28)
at ChildProcess._handle.onexit (node:internal/child_process:293:12)
Emitted 'error' event on FfmpegCommand instance at:
at emitEnd (/Users/isaac/Documents/Github/DiscordBot/node_modules/fluent-ffmpeg/lib/processor.js:424:16)
at endCB (/Users/isaac/Documents/Github/DiscordBot/node_modules/fluent-ffmpeg/lib/processor.js:544:13)
at handleExit (/Users/isaac/Documents/Github/DiscordBot/node_modules/fluent-ffmpeg/lib/processor.js:170:11)
at ChildProcess.<anonymous> (/Users/isaac/Documents/Github/DiscordBot/node_modules/fluent-ffmpeg/lib/processor.js:182:11)
at ChildProcess.emit (node:events:513:28)
at ChildProcess._handle.onexit (node:internal/child_process:293:12)
Like I said earlier, if I change the format to an audio format, it exports perfectly fine. But I have no idea where the problem is with converting to a video. Thanks in advance.