Preface
“Shaping the Future of Video” is not just a catchy slogan that we like to throw around, rather it’s Bitmovin’s company vision. Not only do we keep a close eye on the industry trends, but we’re actively taking part in standardization activities to improve the quality of video technologies. I have been a member and attendant of the Moving Pictures Experts Group for 15+ years and have been documenting the progress since early 2010.
Virtual Meetings Continue – Yet Productivity Prevails
As the global pandemic rages on, most in-person events remain in virtual form, this stayed true for the 131st MPEG meeting. However, as most home office employees will tell you nowadays, productivity is at an all-time high – with multiple progressions across video and audio compression technologies taking major steps towards widespread application, despite certain controversies in the working group. In fact, MPEG was still able to make progress on an unprecedented amount of developments, including, but not limited to: allowing the VVC codec to be finalized, Video-based Point Cloud Compression to move to the FDIS stage, and further support HDR formats.
Future Compression Technology Moves Forward – VVC, Point Cloud Compression, and MPEG-5 Progress
Just in the middle of the SC29 (i.e., MPEG’s parent body within ISO) restructuring process, MPEG successfully ratified — jointly with ITU-T’s VCEG within JVET — its next-generation video codec among other interesting results from the 131st MPEG meeting:
Standards progressing to final approval ballot (FDIS)
- MPEG Announces VVC – the Versatile Video Coding Standard
- Point Cloud Compression – MPEG promotes a Video-based Point Cloud Compression Technology to the FDIS stage
- MPEG-H 3D Audio – MPEG promotes Baseline Profile for 3D Audio to the final stage
Call for Proposals
- Call for Proposals on Technologies for MPEG-21 Contracts to Smart Contracts Conversion
- MPEG issues a Call for Proposals on extension and improvements to ISO/IEC 23092 standard series
Standards progressing to the first milestone of the ISO standard development process
- Widening support for storage and delivery of MPEG-5 EVC
- Multi-Image Application Format adds support of HDR
- Carriage of Geometry-based Point Cloud Data progresses to Committee Draft
- MPEG Immersive Video (MIV) progresses to Committee Draft
- Neural Network Compression for Multimedia Applications – MPEG progresses to Committee Draft
- MPEG issues Committee Draft of Conformance and Reference Software for Essential Video Coding (EVC)
The corresponding press release of the 131st MPEG meeting can be found here: https://mpeg-standards.com/meetings/mpeg-131/. This report focused on video coding featuring VVC as well as PCC and systems aspects (file format, DASH).
MPEG Announces VVC – the Versatile Video Coding Standard
MPEG is pleased to announce the completion of the new Versatile Video Coding (VVC) standard at its 131st MPEG meeting. The document has been progressed to its final approval ballot as ISO/IEC 23090-3 and will also be known as H.266 in the ITU-T.
VVC is the latest in a series of very successful standards for video coding that have been jointly developed with ITU-T, and it is the direct successor to the well-known and widely used High-Efficiency Video Coding (HEVC) and Advanced Video Coding (AVC) standards. VVC provides a major benefit in compression over HEVC. Plans are underway to conduct a verification test with formal subjective testing to confirm that VVC achieves an estimated 50% bit rate reduction versus HEVC for equal subjective video quality. Test results have already demonstrated that VVC typically provides about a 40%-bit rate reduction for 4K/UHD video sequences in tests using objective metrics (i.e., PSNR, VMAF, MS-SSIM). Application areas especially targeted for the use of VVC include
- ultra-high-definition 4K and 8K video,
- video with a high dynamic range and wide colour gamut, and
- video for immersive media applications such as 360° omnidirectional video.
Furthermore, VVC is designed for a wide variety of types of video such as camera captured, computer-generated, and mixed content for screen sharing, adaptive streaming, game streaming, video with scrolling text, etc. Conventional standard-definition and high-definition video content are also supported with similar gains in compression. In addition to improving coding efficiency, VVC also provides highly flexible syntax supporting such use cases as (i) subpicture bitstream extraction, (ii) bitstream merging, (iii) temporal sub-layering, and (iv) layered coding scalability.
The current performance of VVC compared to HEVC-HM is shown in the figure below (taken from https://bit.ly/mpeg131) which confirms the statement above but also highlights the increased complexity. Please note that VTM9 is not optimized for speed but functionality (i.e., compression efficiency).
MPEG also announces completion of ISO/IEC 23002-7 “Versatile supplemental enhancement information for coded video bitstreams” (VSEI), developed jointly with ITU-T as Rec. ITU-T H.274. The new VSEI standard specifies the syntax and semantics of video usability information (VUI) parameters and supplemental enhancement information (SEI) messages for use with coded video bitstreams. VSEI is especially intended for use with VVC, although it is drafted to be generic and flexible so that it may also be used with other types of coded video bitstreams. Once specified in VSEI, different video coding standards and systems-environment specifications can re-use the same SEI messages without the need for defining special-purpose data customized to the specific usage context.
At the same time, the Media Coding Industry Forum (MC-IF) announces a VVC patent pool fostering an initial meeting on September 1, 2020. The aim of this meeting is to identify tasks and to propose a schedule for VVC pool fostering with the goal to select a pool facilitator/administrator by the end of 2020. MC-IF is not facilitating or administering a patent pool.
At the time of writing this blog post, it is probably too early to make an assessment of whether VVC will share the fate of HEVC or AVC (w.r.t. patent pooling). AVC is still the most widely used video codec but with AVC, HEVC, EVC, VVC, LCEVC, AV1, (AV2), and probably also AVS3 — did I miss anything? — the competition and pressure is certainly increasing. Bitmovin is committed to supporting novel video codecs within its product portfolio and specifically enabled efficient multi-codec support (view Bitmovin’s analysis here). We will further evaluate, optimize, and integrate novel video codecs upon becoming available in order to offer the best possible Quality of Experience (QoE) for our customers and their end-users.
MPEG promotes a Video-based Point Cloud Compression Technology to the FDIS stage
At the 131st MPEG meeting, the group promoted its Video-based Point Cloud Compression (V-PCC) standard to the Final Draft International Standard (FDIS) stage. V-PCC addresses lossless and lossy coding of 3D point clouds with associated attributes such as colors and reflectance. Point clouds are typically represented by extremely large amounts of data, which is a significant barrier for mass-market applications. However, the relative ease to capture and render spatial information as point clouds compared to other volumetric video representations makes point clouds increasingly popular to present immersive volumetric data. With the current V-PCC encoder implementation providing compression in the range of 100:1 to 300:1, a dynamic point cloud of one million points could be encoded at 8 Mbit/s with good perceptual quality. Real-time decoding and rendering of V-PCC bitstreams have also been demonstrated on current mobile hardware.
The V-PCC standard leverages video compression technologies and the video ecosystem in general (hardware acceleration, transmission services, and infrastructure) while enabling new kinds of applications. The V-PCC standard contains several profiles that leverage existing AVC and HEVC implementations, which may make them suitable to run on existing and emerging platforms. The standard is also extensible to upcoming video specifications such as Versatile Video Coding (VVC) and Essential Video Coding (EVC).
The V-PCC standard is based on Visual Volumetric Video-based Coding (V3C), which is expected to be re-used by other MPEG-I volumetric codecs under development. MPEG is also developing a standard for the carriage of V-PCC and V3C data (ISO/IEC 23090-10) which has been promoted to DIS status at the 130th MPEG meeting.
By providing high-level immersion at currently available bandwidths, the V-PCC standard is expected to enable several types of applications and services such as six Degrees of Freedom (6 DoF) immersive media, virtual reality (VR) / augmented reality (AR), immersive real-time communication and cultural heritage.
Bitmovin collaborates with the Alpen-Adria-Universität Klagenfurt in the context of the ATHENA project where the paper on the “Objective and Subjective QoE Evaluation for Adaptive Point Cloud Streaming” won the best paper award at QoMEX 2020. This work was a collaboration between Ghent University, Alpen-Adria-Universität Klagenfurt, Bitmovin, Ozyegin University, Networked Media, and AIT Austrian Institute of Technology.
MPEG Systems related News
Finally, I’d like to share news related to MPEG systems and the carriage of video data as depicted in the figure below (taken from https://bit.ly/mpeg131). In particular, the carriage of VVC (and also EVC) has been now enabled in MPEG-2 Systems (specifically within the transport stream) and in the various file formats (specifically within the NAL file format). The latter is used also in CMAF and DASH which makes VVC (and also EVC) ready for HTTP adaptive streaming (HAS).
What about DASH and CMAF?
CMAF maintains the so-called technology under consideration document which contains — among other things — a proposed VVC CMAF profile. Additionally, there are two exploration activities related to CMAF, i.e., (i) multi-stream support and (ii) storage, archiving, and content management for CMAF files.
DASH works on potential improvement for the first amendment to ISO/IEC 23009-1 4th edition related to CMAF support, events processing model, and other extensions. Additionally, there’s a working draft for a second amendment to ISO/IEC 23009-1 4th edition enabling bandwidth change signaling track and other enhancements. Furthermore, ISO/IEC 23009-8 session-based DASH operations has been advanced to Draft International Standard (see also the last report).
An overview of the current status of MPEG-DASH can be found in the figure below.
The next meeting will be again an online meeting in October 2020.
Finally, MPEG organized a Webinar presenting results from the 131st MPEG meeting. The slides and video recordings are available here: https://bit.ly/mpeg131.
Click here for more information about MPEG meetings and their developments
Check out the following links for other great reads!
A little lost about the formats and standards described above? Check out some other great educational content to learn more!
- Bitmovin’s Video Developer Network (No Sign-up Required!)
- [Blog Post] MPEG 130 Meeting Takeaways
- [Blog Series] Live Low Latency Streaming
- [Blog Series] Super-Resolution with Machine Learning
- [E-Book] Ultimate Guide to Container Formats
To see these formats tested against existing Per-Title Encoding products, by industry expert Jan Ozer, Check out the following Whitepaper: Choosing the Best Per-Title Encoding Technology
