Build FFmpeg QSV

This build documentation was tested under clear Ubuntu 18.04 (with gcc-7.4.0) and Ubuntu 19.04 (gcc-8.3.0) but it should work on another OS distributions with various versions of gcc and clang.

Prepare environment and dependencies

Install all required common packages:

You can get it in two ways:

• Build from sources:

  • Intel Media Driver for VAAPI (aka iHD) (gmmlib and LibVA required)
  • Intel Media SDK

  do not forget to export environment variables:

  export LIBVA_DRIVERS_PATH=/path/to/iHD_driver
  export LIBVA_DRIVER_NAME=iHD
  export LD_LIBRARY_PATH=/path/to/msdk/lib
  export PKG_CONFIG_PATH=/path/to/msdk/lib/pkgconfig
  

• Starting from Ubuntu 19.04 Intel media stack components are available for installation via apt-get (see: Intel media stack on Ubuntu).

  sudo apt-get install libva-dev libmfx-dev intel-media-va-driver-non-free
  export LIBVA_DRIVER_NAME=iHD
  

For build ffplay (as part of ffmpeg) you also need to install additional dependencies:

sudo apt-get install libsdl2-dev

Build FFmpeg

Get ffmpeg sources

git clone https://github.com/ffmpeg/ffmpeg
cd ffmpeg

Configure and build FFmpeg install

Configure ffmpeg for use with vaapi and MediaSDK. Main key is --enable-libmfx

./configure --arch=x86_64 --disable-yasm --enable-vaapi --enable-libmfx
make

If you need a debug version of ffmpeg you can try

./configure --arch=x86_64 --disable-yasm --enable-vaapi --enable-libmfx  \
            --enable-debug=3 --disable-stripping --extra-cflags=-gstabs+ \
            --disable-optimizations
make

If you want to install FFmpeg on your machine:

make install

Usage examples

Download stream:

wget https://fate-suite.libav.org/h264-conformance/AUD_MW_E.264

Decode

H264 video decode and save as a raw file

ffmpeg -hwaccel qsv -c:v h264_qsv -i AUD_MW_E.264  \
       -vf hwdownload,format=nv12 -pix_fmt yuv420p \
       AUD_MW.yuv

Encode

Encode a 10 frames of 720p raw input as H264 with 5Mbps using VBR mode:

ffmpeg -loglevel debug -init_hw_device qsv=hw     \
       -filter_hw_device hw -f rawvideo -pix_fmt  \
       yuv420p -s:v 176x144 -i AUD_MW.yuv -vf     \
       hwupload=extra_hw_frames=64,format=qsv     \
       -c:v h264_qsv -b:v 5M -frames:v 10         \
       -y ./AUD_MW_E.h264

Let's analyze cmd in more details: with -loglevel debug FFmpeg provides more logs in cmds. -init_hw_device qsv=hw initialized mfx session and mapped as available to filters such as hwupload (-filter_hw_device hw).

Also note on: -vf hwupload=extra_hw_frames=64,format=qsv, this ensure that MFX runtime receives supported pixel format. Without this init failure and you get output something like this:

[h264_qsv @ 0x55ddc5d21a40] Selected ratecontrol mode is unsupported
[h264_qsv @ 0x55ddc5d21a40] Low power mode is unsupported
[h264_qsv @ 0x55ddc5d21a40] Current frame rate is unsupported
[h264_qsv @ 0x55ddc5d21a40] Current picture structure is unsupported
[h264_qsv @ 0x55ddc5d21a40] Current resolution is unsupported
[h264_qsv @ 0x55ddc5d21a40] Current pixel format is unsupported
[h264_qsv @ 0x55ddc5d21a40] some encoding parameters are not supported by the QSV runtime. Please double check the input parameters.
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
Conversion failed!

The extra argument extra_hw_frames=64 passed to hwupload has to do with the MFX runtime requiring a fixed frame pool size to be per-allocated.

Transcode

H264 decode && H265 encode with 5Mbps using VBR

ffmpeg -hwaccel qsv -c:v h264_qsv -i AUD_MW_E.264 \
       -c:v hevc_qsv -b:v 5M AUD_MW_E.hevc

1:N transcoding

ffmpeg -hwaccel qsv -c:v h264_qsv -i AUD_MW_60FPS.264  \
       -filter_complex "split=2[s1][s2];               \
                        [s1]scale_qsv=1280:720[o1];    \
                        [s2]fps=fps=30[o2]"            \
       -map -[o1] -c:v h264_qsv -b:v 5M 5M.mp4         \
       -map -[o2] -c:v h264_qsv -b:v 4M 4M60FPS.h264

For more details, you can also read Advanced Video options on FFmpeg wiki.

Advanced examples

• RTMP streaming with FFmpeg and MediaSDK

Troubleshouting

ERROR: libmfx not found

Usually it happens, if you forgot to specify pkg-config path (default for manual build is /opt/intel/mediasdk/lib/pkgconfig, and for apt-get is /usr/lib/x86_64-linux-gnu/pkgconfig):

export PKG_CONFIG_PATH=/path/to/mediasdk/lib/pkgconfig

QSV requires a fixed frame pool size

It happens, if you forgot to specify extra_hw_frames=64 (where 64 is pool size) in cmd -vf hwupload=extra_hw_frames=64,format=qsv

[AVHWFramesContext @ 0x55ed5ffce240] QSV requires a fixed frame pool size
[AVHWFramesContext @ 0x55ed5ffce240] Error creating an internal frame pool
[Parsed_hwupload_0 @ 0x55ed5ffc9c00] Failed to configure output pad on Parsed_hwupload_0
Error reinitializing filters!
Failed to inject frame into filter network: Invalid argument
Error while processing the decoded data for stream #0:0

Useful links

• Hardware/Quick Sink FFmpeg wiki: https://trac.ffmpeg.org/wiki/Hardware/QuickSync
• FFmpeg documentation: https://ffmpeg.org/ffmpeg-all.html
  • QSV-encoders: https://ffmpeg.org/ffmpeg-all.html#QSV-encoders
• FFmpeg upstream repository: https://git.ffmpeg.org/ffmpeg.git

  Note: contributions makes thru mailing lists: https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

• Github mirror: https://github.com/FFmpeg/FFmpeg

  Note: it does not accept PRs

• Fork repository for verification patches before submission to upstream: https://github.com/intel-media-ci/ffmpeg