WWDC16: HLS Supports Fragmented MP4 - and Becomes MPEG-DASH Compatible!

WWDC16: HLS Supports Fragmented MP4 – and Becomes MPEG-DASH Compatible!

15 June 2016 . 4 min read

Did you know our video player guarantees playback quality on any screen through our modular architecture, including low-latency, configurable ABR and Stream Lab, the world’s first stream QoE testing service? Check out the Bitmovin Player to learn more.

Apple introduces fragmented MP4 in HLS – a small step for Apple but a huge step for video streaming services

HTTP Live Streaming (HLS) is a very important part of our service, (as you can see from our latest HTML5 Player release) so when we heard the news from WWDC16, that Apple’s HLS protocol will be extended to support fragmented MP4, I think it’s safe to say that we were even more excited than the Apple technicians themselves!
Although Apple still forces content providers to make use of HLS streaming on iOS for assets longer than 10 minutes, the step to introduce fragmented MP4 can be a huge advance from an encoding perspective. Up to now, it has been necessary to encode content in different formats, to maximize browser coverage and reach most of today’s end-user devices. This increases the storage footprint of your content by 2x and reduces also CDN efficiency as content cannot be effectively reused across devices.
In more detail, to enable playback on e.g,. iOS devices, you need to multiplex your content into MPEG-2 Transport Stream, which is (was) required by HLS. You also need to multiplex your content also into fragmented MP4 format to enable native HTML5 playback with MPEG-DASH on e.g., Google Chrome, Firefox, Android and other devices. But with the step towards the usage of fragmented MP4s in combination with HLS, it will be sufficient to multiplex your encoding once to fragmented MP4 and simply use the produced segments for both technologies, HLS and MPEG-DASH – only the manifest files will be different.
So, what will a HLS m3u8 look like with fMP4 segments, as they are used by MPEG-DASH as well. Here is the example directly from the WWDC16 keynote, pretty straight forward:

Apple already published a HLS test asset, using fragmented MP4 segments, which is available at their website, where additional guides and documentation about HLS can also be found.

MPEG-CENC with HLS

But that’s not all! Apple goes even further and will also support DRM-protected fMP4 segments using MPEG-CENC (Common Encryption) in HLS, which is used by MPEG-DASH as well, and supported within all major browsers. Here you can find a recent introduction to DRM on the web, and here you can find a compatibility overview for DRM-systems and browsers, seems we have to update them soon with some more possibilities.

Offline HLS Playback & Offline FairPlay

HLS now also supports offline playback which can be used to persist videos on the user device that can be watched later when no internet connection is available, e.g., in the airplane, train, car, etc. The application has full control of which renditions will be downloaded. This is especially useful for media files that contain different audio tracks and subtitles as the application could download only the tracks that are important for the user. Moreover, offline playback could also be used with FairPlay content that can now be played back without a connection to the license server.

In-Playlist Timed Metadata

HLS now also supports a new metadata format which is called In-playlist timed metadata. It has similar features to ID3 tags but unlike ID3, the metadata information is available in the manifest and not packaged in the media stream. This makes the information available when you download the manifest, which means you get the entire information in advance and therefore it can be used for things like navigation control. It also contains bindings for SCTE-35 tags that can now be carried within In-playlist timed metadata.

Conclusion

Apple announced some very nice HLS features at WWDC16 that could have a huge impact on the media industry. In summary, the most important new features are:

Support for Fragmented MP4

makes HLS content compatible with DASH and vice versa
encode and multiplex your content once and use it with HLS or DASH
increase CDN efficiency while reusing the same content for HLS and DASH
reduces the storage footprint of your encoded content by half
minimal changes in HLS playlists

Support for MPEG Common Encryption (CENC)

supports FairPlay streaming with cbcs mode
first step to harmonize all DRM technologies
in the future you should be able encode and encrypt your content once and use it with Widevine, PlayReady, PrimeTime and FairPlay

Offline HLS Playback

configurable downloading of media renditions
can be used for watch later functionally in airplanes, trains and other places where no internet connection is available

Offline FairPlay Playback

encrypted content can be played back without internet connection
assets must be marked in the backend to allow offline playback

In-Playlist Metadata

same feature support as ID3 tags
entire information when you download the manifest and therefore it can be used for things like navigation control
contains bindings for SCTE-35 tags

We at Bitmovin are now working on some demos using HLS with fragmented MP4 with our player and encoding, to show the huge impact to the streaming ecosystem and its potentials, so stay tuned!
More insights about Apple’s streaming protocol in general can be found in one of our recent blog posts and a detailed overview of streaming technologies and covered platforms in our support section.
Update 17 June 2016: The Bitmovin team has already prepared a demonstration of FMP4 in HLS.

See it in Action!

Below you can see the Bitmovin HTML5 Adaptive Streaming Player in action, playing Fragmented MP4 through an HLS manifest.
Please note that this demo will work on Mac iOS 10 and above, and for PC users, all recent browser versions are compatible, including Edge, FireFox, Chrome and Safari.

All the best,
Reinhard & the Bitmovin Team!

Providing a Premium Audio Experience in HLS with the Bitmovin Encoder

Developers

Open-Source vs. Commercial Players: Understanding the True Cost of Ownership

Developers

Device testing

Whitepaper

Partnership

Marketplace

WHITEPAPER

PARTNERS