Vulkan Video adds hardware extensions for portable multicore codecs
Khronos is releasing a set of hardware extensions called Vulkan Video to standardise video encoding and decoding.
The Provisional Vulkan Video acceleration extensions support the Vulkan philosophy of providing flexible, fine-grained control over video processing scheduling, synchronization, and memory utilization to the application.
The extensions use the existing Vulkan framework enables efficient, low-latency, low-overhead use of processing resources, including distributing stream processing tasks across multiple CPU cores and video codec hardware, all with application portability across multiple platforms and devices ranging from small embedded devices to high performance servers.
Alongside the hardware extensions, Khronos plans to add support in the Vulkan SDK with layers for validation and higher-level abstractions that will speed the development of video applications where simple frame-in-frame-out and black-box decoding and encoding is sufficient. This will be complemented by open source samples for a range of application use cases on Windows and Linux.
The Provisional Vulkan Video extensions closely integrate hardware accelerated video processing with Vulkan’s existing graphics, compute and display functionality. Developers are encouraged to provide feedback so the finalized Vulkan Video 1.0 extensions can be finely tuned to provide new capabilities for a wide range of applications.
GPUs typically contain dedicated video decode and encode acceleration engine(s) that are independent from other graphics and compute engines. In fact, some physical devices may support only video decode and/or video encode operations. Consequently Vulkan Video adds video decode and encode queues, the presence of which can be queried by using VkQueueFlagBits.
Video codecs are continuously evolving, enabling ever more efficient video compression and decompression through increasingly advanced and domain-specific video coding tools but resulting in new codecs and codec extensions. Vulkan Video has been designed with flexible support for a wide variety of existing and future codecs by being divided into universal ‘core’ extensions expected to be relevant to all codecs, and codec specific extensions. The core extensions include video queue functionality that are video codec-independent.
This provisional Vulkan Video release also includes three extensions that extend base structures defined by the core video KHR extensions to support H.264-decode, H.264-encode, and H.265-decode:
These extensions do not define API calls, but simply extend data structures. There is an H.265-encode extension currently in development, and VP9 decode and AV1 decode/encode extensions are expected to follow soon after in a later release.
Video coding experts often analyze video bitstreams to investigate coding artifacts and improve video quality using codec-specific syntax elements in the bitstream using the codec specification that defines behaviuoral descriptions of syntax and tools. Vulkan Video makes it easy to recognize API fields corresponding to codec syntax elements or codec-defined terms, without bloating the Vulkan specification with descriptions already well documented in the codec standard specifications.
Codec-specific standard C-headers define structures with explicit and derived codec syntax fields in the naming and style convention of the corresponding codec standard specification. These Std structures are used as fields in the EXT extension structures.
Nvidia has released beta Vulkan drivers that implement Vulkan Video, and a sample Vulkan Video decoding application vk_video_decoder to enable developers to prototype and experiment against the current provisional extensions.
The SDK validation layer support will be added for the finalized Vulkan Video 1.0 extensions. For this provisional release, validation layers will only be verified to work with Vulkan Video extensions disabled.
Khronos says it will now work to finalize the Vulkan Video 1.0 specifications, SDK and conformance tests, so focus can then shift towards supporting additional codecs and more advanced video features.
Related Vulkan articles
- Partnership aims to bring safety certifiable GPU compute
- Graphics and compute libraries for Deos RTOS
- ARM’s Bifrost steps up graphics, bridges to machine learning
Other articles on eeNews Europe
- Percepio, Lauterbach team for debug
- Quantum laser links for aircraft WiFi
- Xilinx, Mavenir team on OpenRAN Massive MIMO
- Arduino uses Mbed for low code IoT ML development
If you enjoyed this article, you will like the following ones: don't miss them by subscribing to :
