GStreamer 1.0 plugins for i.MX platforms
This is a set of GStreamer 1.0 plugins for NXP's i.MX platform, which make use of the i.MX multimedia capabilities.
Currently tested on: i.MX6DL, i.MX6Q, i.MX8m, i.MX8m mini, i.MX8 QuadMax
These plugins are licensed under the LGPL v2.
There is one decoder element for each format the i.MX SoC supports. The list of supported formats depends on the i.MX variant. For example, if h.264 is supported, there will be a
imxvpudec_h264element. The same is true of the encoder elements.
For the full list of formats that could theoretically be supported, check out the libimxvpuapi library version 2.1.2 or later.
Video transform, video sink, and compositor elements are available that perform their operations using hardware blitters.
The following element types are available (not all are possible with all 2D blitter APIs):
NOTE: Compositor elements are only available with GStreamer 1.16 or later. Compositor support in GStreamer 1.14 was not yet in gst-plugins-base and had serious bugs.
NOTE: 2D blitter video sink elements do not work on i.MX8 machines, since these no longer use the older MXC framebuffer driver (which these video sink elements rely on).
The following blitters are supported by these elements:
All elements use internal "uploader" code that uploads frames into DMA memory if necessary. If incoming frames are not aligned in a way that is compatible with what the blitters require, internal frame copies are automatically done. These frame copies are CPU-based, so performance may suffer, but otherwise, such frames could not be processed at all.
mxc_v4l2V4L2 driver on the i.MX6 for various capture devices such as the OmniVision OV5640 is unfortunately severely broken, and requires numerous workarounds in userspace. Furthermore, it contains several i.MX specific extras for working with physical addresses. For that reason, GStreamer's
v4l2sinkelements do not work on the i.MX6 for these devices.
To fix this, gstreamer-imx contains two elements that access V4L2 and apply workarounds for the driver issues. These elements are:
v4l2srcequivalent for capturing video frames
v4l2sinkequivalent for displaying video frames
NOTE: These elements exist only as a workaround for these driver bugs, since fixing that driver is impractical. They are not meant to be used on any newer i.MX platform, since starting with the i.MX8, a different (and better) driver is used.
-Dv4l2command line switch can be passed to meson to enable / disable this V4L2 plugin. If set to
true(the default value), it will be enabled.
falsewill disable it.
imxuniaudiodec: audio decoder plugin based on NXP's unified audio (UniAudio) architecture
imxmp3audioenc: MP3 audio encoder plugin based on NXP's MP3 encoder
All DMA buffer allocators that are used by the elements implement the GstPhysMemoryAllocator interface. This makes it possible to integrate them with other GStreamer elements, as long as said elements can make use of the GstPhysMemory functionality to extract physical addresses. In addition, if the i.MX Soc BSP supports ION based DMA buffer allocation, the elements will use an allocator based on GstDmaBufAllocator. This allocator also implemnts the GstPhysMemoryAllocator interface, so with this, it is possible to access the underlying DMA buffer by its DMA-BUF file descriptor and also by its physical address, making integration even easier.
This is also why the old
imxeglvivsinkelement from the earlier "v1" branch is not present anymore - it is now unnecessary, since glimagesink and waylandsink both are full replacements.
Furthermore, if applications or other GStreamer elements wish to access some of the gstreamer-imx specific functionality (particularly its allocators and interfaces), the gstimxcommon library can be used for this purpose, since its API is public.
videoparsersbadplugin from the
gst-plugins-badpackage in GStreamer is needed, since this plugin contains video parsers like
mpegvideoparse(for MPEG1 and MPEG2), and
You must also use a Linux kernel with i.MX additions for i.MX specific subsystems such as the VPU, IPU, PxP etc. Such kernels are typically referred to as "imx-kernels".
At least version 0.54.0 is required. (Older versions can (partially) work, but build scripts may require changes/workarounds.)
First, set up the necessary cross compilation configuration for meson.
Then, create a build directory for an out-of-tree build:
make build cd build
Now set up the build by running meson:
You might want to look into the
--buildtype=plainflag if the compiler flags Meson adds are a problem. This is particularly useful for packagers. Read here for more.
Also, you might be interested in the
-Dlibdirarguments to control where to install the resulting binaries. Again, this is particularly useful for packagers.
Finally, build and install the code by running ninja:
The following meson configuration options are available. Use them with the
-Dflag. For example, to set the
vpuoption to "true", specify
See the Meson documentation about build option types to understand what values can be passed to these options.
Since most options are of type
feature, this is the type in use unless a different type is explicitly defined in the list below.
vpu: VPU en/decoder elements.
g2d: 2D blitter elements based on the Vivante G2D API.
g2d-based-on-dpu: Special option for use with i.MX8 QuadMax and QuadPlus SoCs. G2D is emulated via the DPU on these machines. The DPU behaves somewhat differently. It is recommended to set this option to
trueon these SoCs for better performance. On all other SoCs, this must be set to
false(the default value). Type:
ipu: 2D blitter elements based on the NXP Image Processing Unit (IPU).
pxp: 2D blitter elements based on the NXP Pixel Pipeline (PxP).
imx-headers-path: Path to extra imx kernel headers. These are used for IPU and PxP code. The build scripts attempt to autodetect this path, so specifying this typically is not necessary. Type:
imx2d-videosink: Enables/disables building 2D blitter video sink elements. Default value is
true. Setting this to
falsemakes sense on i.MX8 machines, since rendering to the framebuffer is not possible on those. Type:
imx2d-compositor: Enables/disables building 2D blitter compositor elements. Type:
v4l2: Enables/disables building the custom Video4Linux2 source / sink elements. See the Video4Linux2 section above for details. Type:
package-name: GStreamer package name to use in the plugins. Type:
package-origin: GStreamer package origin to use in the plugins. Type: