Preface
Bitmovin isn’t the only organization whose sole purpose is to shape the future of video – a few senior developers at Bitmovin along with me are active members of the Moving Pictures Expert Group (MPEG). Personally, I have been a member and attendant of MPEG for 15+ years and have been documenting the progress since early 2010. This year yielded countless changes for consumers, developers, academics, and organizations alike, with most business operations shifting to a digital-first orientation. MPEG was no different, with three out of four our annual working sessions were strictly virtual. In addition to this general industry-wide shift, MPEG made further rearrangements to our structure to help streamline the standardization processes, especially since hosting a 400 person zoom session could end up being more distracting than productive.
The 131st meeting was our first sample of the new process and yielded the ratification of h.266 Versatile Video Coding (VVC) codec. Our latest session was even more productive with VVC quality verification tests, 3D geometry-based cloud compression, LCEVC, and Omnidirectional Media format methods moving to the final stages of standardization.
MPEG Continues to Progress – The 132nd Meeting is the First Under the New Structure
The 132nd MPEG meeting was the first meeting with the new structure as introduced previously. That is, ISO/IEC JTC 1/SC 29/WG 11 — the official name of MPEG under the ISO structure — was disbanded after the 131st MPEG meeting and some of the subgroups of WG 11 (MPEG) have been elevated to independent MPEG Working Groups (WGs) and Advisory Groups (AGs) of SC 29 rather than subgroups of the former WG 11. Thus, the MPEG community is now an affiliated group of WGs and AGs that will continue meeting together according to previous MPEG meeting practices and will further advance the standardization activities of the MPEG work program.
The 132nd MPEG meeting was the first meeting with the new structure as follows (incl. Convenors and position within WG 11 structure):
- AG 2 MPEG Technical Coordination (Convenor: Prof. Jörn Ostermann; for overall MPEG work coordination and prev. known as the MPEG chairs meeting; it’s expected that one can also provide inputs to this AG without being a member of this AG),
- WG 2 MPEG Technical Requirements (Convenor Dr. Igor Curcio; former Requirements subgroup),
- WG 3 MPEG Systems (Convenor: Dr. Youngkwon Lim; former Systems subgroup),
- WG 4 MPEG Video Coding (Convenor: Prof. Lu Yu; former Video subgroup),
- WG 5 MPEG Joint Video Coding Team(s) with ITU-T SG 16 (Convenor: Prof. Jens-Rainer Ohm; former JVET),
- WG 6 MPEG Audio Coding (Convenor: Dr. Schuyler Quackenbush; former Audio subgroup)
- WG 7 MPEG Coding of 3D Graphics (Convenor: Prof. Marius Preda, former 3DG subgroup)
- WG 8 MPEG Genome Coding (Convenor: Prof. Marco Mattaveli; newly established WG),
- AG 3 MPEG Liaison and Communication (Convenor: Prof. Kyuheon Kim; (former Communications subgroup), and
- AG 5 MPEG Visual Quality Assessment (Convenor: Prof. Mathias Wien; former Test subgroup).
The 132nd MPEG meeting was an online meeting with more than 300 participants who continued to work efficiently on standards for the future needs of the industry. As a group, MPEG started to explore new application areas that will benefit from standardized compression technology in the future. A new web site has been created and can be found at http://mpeg.org/.
The official press release can be found here and comprises the following items:
- Versatile Video Coding (VVC) Ultra-HD Verification Test Completed and Conformance and Reference Software Standards Reach their First Milestone
- MPEG Completes Geometry-based Point Cloud Compression Standard (G-PCC)
- MPEG Evaluates Extensions and Improvements to MPEG-G and Announces a Call for Evidence on New Advanced Genomics Features and Technologies
- MPEG Issues Draft Call for Proposals on the Coded Representation of Haptics
- MPEG Evaluates Responses to MPEG IPR Smart Contracts CfP
- MPEG Completes Standard on Harmonization of DASH and CMAF
- MPEG Completes 2nd Edition of the Omnidirectional Media Format (OMAF)
- MPEG Completes the Low Complexity Enhancement Video Coding (LCEVC) Standard
In this report, I’d like to focus on VVC, G-PCC, DASH/CMAF, OMAF, and LCEVC.
Versatile Video Coding (VVC) Ultra-HD Verification Test Completed and Conformance & Reference Software Standards Reach their First Milestone
MPEG completed a verification testing assessment of the recently ratified Versatile Video Coding (VVC) standard for ultra-high definition (UHD) content with standard dynamic range, so that it can be used in newer streaming and broadcast television applications. The verification test was performed using rigorous subjective quality assessment methods and showed that VVC provides a compelling gain over its predecessor — the High-Efficiency Video Coding (HEVC) standard. In particular, the verification test was performed using the VVC reference software implementation (VTM) and the recently released open-source encoder implementation of VVC (VVenC):
- Using its reference software implementation (VTM), VVC showed bit rate savings of roughly 45% over HEVC for comparable subjective video quality.
- Using VVenC, additional bit rate savings of more than 10% relative to VTM were observed, which at the same time runs significantly faster than the reference software implementation.
Additionally, the standardization work for both conformance testing and reference software for the VVC standard reached its first major milestone, i.e., progressing to the Committee Draft ballot in the ISO/IEC approval process. The conformance testing standard (ISO/IEC 23090-15) will ensure interoperability among the diverse applications that use the VVC standard, and the reference software standard (ISO/IEC 23090-16) will provide an illustration of the capabilities of VVC and a valuable example showing how the standard can be implemented. The reference software will further facilitate the adoption of the standard by being available for use as the basis of product implementations.
Bitmovin has always been among the early adopters of new video coding standards such as for HEVC and AV1. Thus, it is expected that we will also come up with early implementations for VVC in the context of ABR encoding, most likely in collaboration with our research partners such as the Alpen-Adria-Universität Klagenfurt (https://athena.itec.aau.at/).
MPEG Completes Geometry-based Point Cloud Compression Standard
MPEG promoted its ISO/IEC 23090-9 Geometry-based Point Cloud Compression (G‐PCC) standard to the Final Draft International Standard (FDIS) stage. G‐PCC addresses lossless and lossy coding of time-varying 3D point clouds with associated attributes such as colour and material properties. This technology is particularly suitable for sparse point clouds. ISO/IEC 23090-5 Video-based Point Cloud Compression (V‐PCC), which reached the FDIS stage in July 2020, addresses the same problem but for dense point clouds, by projecting the (typically dense) 3D point clouds onto planes, and then processing the resulting sequences of 2D images using video compression techniques. The generalized approach of G-PCC, where the 3D geometry is directly coded to exploit any redundancy in the point cloud itself, is complementary to V-PCC and particularly useful for sparse point clouds representing large environments.
Point clouds are typically represented by extremely large amounts of data, which is a significant barrier to mass-market applications. However, the relative ease of capturing and rendering spatial information compared to other volumetric video representations makes point clouds increasingly popular for displaying immersive volumetric data. The current draft reference software implementation of a lossless, intra-frame G‐PCC encoder provides a compression ratio of up to 10:1 and lossy coding of acceptable quality for a variety of applications with a ratio of up to 35:1.
By providing high immersion at currently available bit rates, the G‐PCC standard will enable various applications such as 3D mapping, indoor navigation, autonomous driving, advanced augmented reality (AR) with environmental mapping, and cultural heritage.
Bitmovin is actively conducting research and development with respect to the adaptive streaming of point cloud data in collaboration with our research partners such as the Alpen-Adria-Universität Klagenfurt. In this context, we have recently co-authored a paper on “From Capturing to Rendering: Volumetric Media Delivery With Six Degrees of Freedom” that has been published in the IEEE Communications Magazine.
MPEG Finalizes the Harmonization of DASH and CMAF
MPEG successfully completed the harmonization of Dynamic Adaptive Streaming over HTTP (DASH) with Common Media Application Format (CMAF) featuring a DASH profile for the use with CMAF (as part of the 1st Amendment of ISO/IEC 23009-1:2019 4th edition).
CMAF and DASH segments are both based on the ISO Base Media File Format (ISOBMFF), which helps enable a smooth integration of both technologies. Most importantly, this DASH profile defines (a) a normative mapping of CMAF structures to DASH structures and (b) how to use Media Presentation Description (MPD) as a manifest format.
Additional tools added to this amendment include
- DASH events and timed metadata track timing and processing models with in-band event streams,
- a method for specifying the resynchronization points of segments when the segments have internal structures that allow container-level resynchronization,
- an MPD patch framework that allows the transmission of partial MPD information as opposed to the complete MPD using the XML patch framework as defined in IETF RFC 5261, and
- content protection enhancements for efficient signaling.
It is expected that the 5th edition of the MPEG DASH standard (ISO/IEC 23009-1) containing this change will be issued at the 133rd MPEG meeting in January 2021. An overview of DASH standards/features can be found in the Figure below.
CMAF enables low latency DASH thanks to the introduction of chunks (in CMAF) and HTTP/1.1 chunked transfer encoding (CTE). Bitmovin’s R&D efforts with respect to low latency streaming resulted in ACTE (ABR for Chunked Transfer-Encoding), a bandwidth prediction scheme for low-latency chunked streaming.
MPEG Completes 2nd Edition of the Omnidirectional Media Format
MPEG completed the standardization of the 2nd edition of the Omnidirectional Media Format (OMAF) by promoting ISO/IEC 23009-2 to Final Draft International Standard (FDIS) status including the following features:
- “Late binding” technologies to deliver and present only that part of the content that adapts to the dynamically changing users’ viewpoint. To enable an efficient implementation of such a feature, this edition of the specification introduces the concept of bitstream rewriting, in which a compliant bitstream is dynamically generated that, by combining the received portions of the bitstream, covers only the users’ viewport on the client.
- Extension of OMAF beyond 360-degree video. This edition introduces the concept of viewpoints, which can be considered as user-switchable camera positions for viewing content or as temporally contiguous parts of a storyline to provide multiple choices for the storyline a user can follow.
- Enhances the use of video, image, or timed text overlays on top of omnidirectional visual background video or images related to a sphere or a viewport.
Bitmovin’s research efforts related to 360-degree video streaming resulted in the first prototypes of viewport adaptive streaming awarded by DASH-IF excellence in DASH award.
MPEG Completes the Low Complexity Enhancement Video Coding Standard
MPEG is pleased to announce the completion of the new ISO/IEC 23094-2 standard, i.e., Low Complexity Enhancement Video Coding (MPEG-5 Part 2 LCEVC), which has been promoted to Final Draft International Standard (FDIS) at the 132nd MPEG meeting.
- LCEVC adds an enhancement data stream that can appreciably improve the resolution and visual quality of reconstructed video with an effective compression efficiency of limited complexity by building on top of existing and future video codecs.
- LCEVC can be used to complement devices originally designed only for decoding the base layer bitstream, by using firmware, operating system, or browser support. It is designed to be compatible with existing video workflows (e.g., CDNs, metadata management, DRM/CA) and network protocols (e.g., HLS, DASH, CMAF) to facilitate the rapid deployment of enhanced video services.
- LCEVC can be used to deliver higher video quality in limited bandwidth scenarios, especially when the available bit rate is low for high-resolution video delivery and decoding complexity is a challenge. Typical use cases include mobile streaming and social media, and services that benefit from high-density/low-power transcoding.
The next meeting will be again an online meeting in January 2021.
Click here for more information about MPEG meetings and their developments
Check out the following links for other great reads!
A little lost about the formats and standards described above? Check out some other great educational content to learn more!
- Bitmovin’s Video Developer Network (No Sign-up Required!)
- [Blog Post] MPEG 131 Meeting Takeaways
- [Blog Series] Cloud-Agnostic Encoding Solutions for Dolby Vision and Dolby Atmos
- [Blog Series] Super-Resolution with Machine Learning
- [E-Book] Ultimate Guide to Container Formats
