The 117th MPEG meeting concluded on Jan 20, 2017 in Geneva, Switzerland and featured some hot topics such as; Omnidirectional Media Application Format, Global Media Technology Standards and going beyond HEVC.
The entire MPEG press release can be found here, but in this blog post I will focus on the following topics from the meetup:
- MPEG issues Committee Draft of the Omnidirectional Media Application Format (OMAF)
- MPEG Workshop “Global Media Technology Standards for an Immersive Age”
- Preliminary Call for Evidence on video compression with capability beyond HEVC
MPEG issues Committee Draft of the Omnidirectional Media Application Format (OMAF)
Real-time entertainment services deployed over the open, unmanaged Internet – streaming audio and video – account now for more than 70% of the evening traffic in North American fixed access networks and it is assumed that this figure will reach 80 percent by 2020. More and more such bandwidth hungry applications and services are pushing onto the market including immersive media services such as virtual reality and, specifically 360-degree videos. However, the lack of appropriate standards and, consequently, reduced interoperability is becoming an issue and, thus, MPEG has started a project referred to as Omnidirectional Media Application Format (OMAF). The first milestone of this standard has been reached and the committee draft (CD) has been approved at the 117th MPEG meeting. Such application formats “are essentially superformats that combine selected technology components from MPEG (and other) standards to provide greater application interoperability, which helps satisfy users’ growing need for better-integrated multimedia solutions” [MPEG-A].” In the context of OMAF, the following aspects are defined:
- Equirectangular projection format (note: others might be added in the future)
- Metadata for interoperable rendering of 360-degree monoscopic and stereoscopic audio-visual data
- Storage format: ISO base media file format (ISOBMFF)
- Codecs: High Efficiency Video Coding (HEVC) and MPEG-H 3D audio
OMAF is the first specification which is defined as part of a bigger project currently referred to as ISO/IEC 23090 — Immersive Media (Coded Representation of Immersive Media). It currently has the acronym MPEG-I and we have previously used MPEG-VR which is now replaced by MPEG-I (that still might chance in the future). It is expected that the standard will become Final Draft International Standard (FDIS) by Q4 of 2017. Interestingly, it does not include AVC and AAC, probably the most obvious candidates for video and audio codecs which have been massively deployed in the last decade and probably still will be a major dominator (and also denominator) in upcoming years. On the other hand, the equirectangular projection format is currently the only one defined as it is broadly used already in off-the-shelf hardware/software solutions for the creation of omnidirectional/360-degree videos. Finally, the metadata formats enabling the rendering of 360-degree monoscopic and stereoscopic video is highly appreciated. A solution for MPEG-DASH based on AVC/AAC utilizing equirectangular projection format for both monoscopic and stereoscopic video is shown as part of our solution for VR and 360-degree video.
MPEG Workshop “Global Media Technology Standards for an Immersive Age”
On January 18, 2017 MPEG successfully held a public workshop on “Global Media Technology Standards for an Immersive Age” hosting a series of keynotes from Bitmovin, DVB, Orange, Sky Italia, and Technicolor. Stefan Lederer, CEO of Bitmovin discussed today’s and future challenges with new forms of content like 360°, AR and VR. The slides are available here and MPEG took their feedback into consideration in an update of its 5-year standardization roadmap. David Wood (EBU) reported on the DVB VR study mission and Ralf Schaefer (Technicolor) presented a snapshot on VR services. Gilles Teniou (Orange) discussed video formats for VR pointing out a new opportunity to increase the content value but also raising a question what is missing today. Finally, Massimo Bertolotti (Sky Italia) introduced his view on the immersive media experience age.
Overall, the workshop was well attended and as mentioned above, MPEG is currently working on a new standards project related to immersive media. Currently, this project comprises five parts. The first part comprises a technical report describing the scope (incl. kind of system architecture), use cases, and applications. The second part is OMAF (see above) and the third/forth parts are related to immersive video and audio respectively. Part five is about point cloud compression (see below).
Preliminary Call for Evidence on video compression with capability beyond HEVC
A preliminary “Call for Evidence on video compression with capability beyond HEVC” was issued at the 117th MPEG meeting. This call will be made jointly with ITU-T SG16/Q6 (VCEG). It is addressed to interested parties which are in possession of technology providing better compression capability than the existing standard, either for conventional video material, or for other domains such as HDR/WCG or 360-degrees (“VR”) video. As test cases, the call defines rate points and materials in all of these latter categories, anchors with HEVC encodings are also provided. Submissions are expected for the July 2017 meeting, where assessment will be made based on objective criteria (such as rate savings judged by PSNR quality) as well as subjective quality evaluation (experts viewing). The final version of the Call for Evidence is planned to be issued by the April meeting, where it is anticipated that encodings with the Joint Exploration Model (JEM) algorithm may be included as an additional reference point for comparison. The JEM is being developed by the Joint Video Exploration Team (JVET) of MPEG and VCEG, and is already known to provide bit rate reductions in the range of 20-30% for relevant test cases, as well as subjective quality benefits. Based on the outcome of the call, and promising evidence that potential technology exists, MPEG and VCEG may produce a formal Call for Proposals later in the year, with the intent to enter a more rigid standardization phase for the next generation of video compression standards beyond HEVC. A preliminary target date for completion of a new standard on the subject is late 2020.
What else happened at the MPEG meeting?
- Committee Draft MORE (note: type in ‘man more’ on any unix/linux/max terminal and you’ll get ‘less – opposite of more’;): MORE stands for “Media Orchestration” and provides a specification that enables the automated combination of multiple media sources (cameras, microphones) into a coherent multimedia experience. Additionally, it targets use cases where a multimedia experience is rendered on multiple devices simultaneously, again giving a consistent and coherent experience.
- Technical Report on HDR/WCG Video Coding: This technical report comprises conversion and coding practices for High Dynamic Range (HDR) and Wide Colour Gamut (WCG) video coding (ISO/IEC 23008-14). The purpose of this document is to provide a set of publicly referenceable recommended guidelines for the operation of AVC or HEVC systems adapted for compressing HDR/WCG video for consumer distribution applications
- CfP Point Cloud Compression (PCC): This call solicits technologies for the coding of 3D point clouds with associated attributes such as color and material properties. It will be part of the immersive media project introduced above.
- MPEG-H 3D Audio verification test report: This report presents results of four subjective listening tests that assessed the performance of the Low Complexity Profile of MPEG-H 3D Audio. The tests covered a range of bit rates and a range of “immersive audio” use cases (i.e., from 22.2 down to 2.0 channel presentations). Seven test sites participated in the tests with a total of 288 listeners.
Finally, let’s have a look what happened related to MPEG-DASH…
MPEG-DASH and CMAF: Friend or Foe?
For MPEG-DASH and CMAF it was a meeting “in between” official standardization stages. MPEG-DASH experts are still working on the third edition which will be a consolidated version of the 2nd edition and various amendments and corrigenda. In the meantime, MPEG issues a white paper on the new features of MPEG-DASH which I would like to highlight here.
- Spatial Relationship Description (SRD): allows to describe tiles and region of interests for partial delivery of media presentations. This is highly related to OMAF and VR/360-degree video streaming.
- External MPD linking: this feature allows to describe the relationship between a single program/channel and a preview mosaic channel having all channels at once within the MPD.
- Period continuity: simple signaling mechanism to indicate whether one period is a continuation of the previous one which is relevant for ad-insertion or live programs.
- MPD chaining: allows for chaining two or more MPDs to each other, e.g., pre-roll ad when joining a live program.
- Flexible segment format for broadcast TV: separates the signaling of the switching points and random access points in each stream and, thus, the content can be encoded with a good compression efficiency, yet allowing higher number of random access point, but with lower frequency of switching points.
- Server and network-assisted DASH (SAND): enables asynchronous network-to-client and network-to-network communication of quality-related assisting information.
- DASH with server push and WebSockets: basically addresses issues related to HTTP/2 push feature and WebSocket.
CMAF issued a study document which captures the current progress and all national bodies are encouraged to take this into account when commenting on the Committee Draft (CD). To answer the question in the headline above, it looks more and more like as DASH and CMAF will become friends — let’s hope that the friendship lasts for a long time.
The next MPEG meeting will be held in Hobart, April 3-7, 2017. Feel free to contact us for any questions or comments.
