I'm currently trying to use the vaapi hwaccelleration on FFMPEG.
In my command, I have hwaccel
on vaapi
, hwaccel_output_fomrat
on vaapi
, -hwaccel_device
on /dev/dri/renderD128
, as well as -vf
as format=nv12, hwupload
and as video codec -c:v
on h264_vaapi
.
When I now try to start it, I'm getting the error
grep stderr: [hwupload @ 0x30bb660] A hardware deveice reference is required to upload frames to.
[Parsed_hwupload_1 @ 0x30bb560] Query format failed for 'Parsed_hwupload_1': Invalid argument
Can I somewhere define a hardware device reference? I thought it's what I do with hwaccel_device
, but seems like not. So what can I do to get this working?
You'll need to initialize your hardware accelerator correctly, as shown in the documentation below (perhaps we should create a wiki entry for this in time?):
Assume the following snippet:
Where:
(a). VAAPI is available, and we will bind the DRM node
/dev/dri/renderD128
to the encode session, and(b). We are taking a udp input, where
$ingest_ip:$port_ip
corresponds to a known UDP input stream, matching the IP and port pairing respectively, with a defined fifo size (as indicated by the'?fifo_size=n'
parameter).(c). Encoding to an output udp stream packaged as an MPEG Transport stream (see the muxer in use, mpegts), with the necessary parameters matching the output IP and port pairing respectively.
(d). Defined video bitrates (
$video_bitrate$unit
, where $unit can be either K or M, as you see fit) and audio bitrates ($audio_bitrate$unit
, where $unit should be in K, for AAC LC-based encodings) as shown above, with appropriate encoder settings passed to the vaapi encoders. For your reference, there are four available video encoders in FFmpeg as at the time of writing, namely:With the omission of the mjpeg encoder (as its' not of interest in this context), and each of these encoders' documentation can be accessed via:
Where
$encoder_name
matches the encoders on the list above.For VAAPI, the following notes apply:
i.
-init_hw_device vaapi=intel:/dev/dri/renderD128
initializes a hardware device named vaapi (that can be called up later via the-hwaccel_device
and-filter_hw_device
as demonstrated above) bound to the DRM render node/dev/dri/renderD128
. Theintel:
prefix can be dropped, but its' often useful to identify what render node was used by a vendor name in an environment where more than one VAAPI-capable device exist, such as a rig with an Intel IGP and an AMD GPU.ii. Take note of the format constraint defined by
-hwaccel_output_format vaapi
. This is needed to satisfy the condition in 1.iii. We then pick up the named hardware acceleration implementation, vaapi, and call it up for both the hardware accelerator device (
-hwaccel_device
) and the device to which we will upload the hardware frames via the hwupload filter (-filter_hw_device
). Omitting the latter will result in encoder initialization failure, as you observed.iv. Now, inspect the video filter syntax closely:
This video filter chain converts any unsupported video frames to the VAAPI hardware format, applying a known constraint prior to uploading the frames to the device via hwupload. This is done for safery reasons; you cannot assume that the decoded format will be accepted by the encoder. Performance in this mode will vary, based on source, the decoder device and the VAAPI driver in use.
v. Now, for the video encoder (defined by
-c:v $encoder_name
), pass your arguments as needed. You can modify the example I provided in the snippet above, though its' wise to refer to the encoder documentation as explained earlier should you need further tuning.Bonus: Dealing with the Intel-based QSV encoders:
I'm including this section for future reference, for these who use Intel's open source MSDK for FFmpeg's QSV enablement and the associated encoders. See the snippet below:
You can see the similarities.
The QSV encoders use VAAPI-style mappings (as explained above), but with an extra constraint placed for the hwupload filter: The
hwupload=extra_hw_frames=10
parameter must be used, or the encoder's initialization will fail.One of the reasons I cannot recommend QSV's encoders, despite their supposedly better output quality, are their fragile mappings, that often exit with some of the most unhelpful errors often unrelated to how the encoder failed. Where possible, stick to VAAPI. QSV's usefulness (where applicable) is for low power encoding, as is the case with Intel's Apollolake and anemic Cannonlake initial offerings.
Hope this documentation will be of use to you.