Halve your Encoding, Packaging and Storage Costs – HLS with fragmented MP4

By using a single package format you can reduce your encoding, packaging and storage costs by halve and decrease your CDN costs by up to 10% as fMP4 has less overhead than MPEG-TS

At this years Worldwide Developer Conference (WWDC 2016), Apple introduced fragmented MP4 (fMP4) for HLS. Although this announcement was not such a big deal for Apple, the impact on the rest of the media industry is huge. In this blog post I will try to explain why, and also address some of the frequently asked questions around the topic of fMP4 for HLS.

Why is this such a big change? After all, it’s just a container format. It’s still HLS.

Very true, but this new container format halves encoding/packaging and storage costs. In the past you were required to multiplex each rendition/bitrate/resolution into two containers, MPEG2 Transport Stream (TS) for HLS and fMP4 for DASH, or maintain “just in time” packagers that do not scale well and cost you money on every request. Now that HLS supports fMP4 it can share the same encoded segments as the DASH manifest. On top of that, there is now also the potential for a major reduction in CDN costs for some businesses. TS is less efficient than DASH, with up to 10% more overhead. This means that CDN costs for HLS content can be reduced by up to 10% in certain cases.

Why do we need two formats, HLS and DASH, isn’t one enough?

Technically speaking it should be, and everybody would prefer that. Only having one format would make all of our lives much easier. Unfortunately, due to the proprietary nature of the Apple infrastructure, HLS is required on Safari, iOS and tvOS while on the other hand you need DASH to get native HTML5 playback on all other browsers.

Isn’t it possible to playback HLS in HTML5 on all browsers?

It is – and we can do this too – but for high resolutions and bitrates (just think about 360° videos in 4k and higher resolutions) it’s obviously not as effective (you need to remultiplex every chunk in javascript) and this costs performance, battery and latency, which makes such videos unplayable on non state of the art devices.

Which devices support HLS with fMP4?

It’s supported on iOS10, macOS and tvOS. Considering that Apple users are adopting new versions of iOS quite quickly – trends show that 80% of all Apple iOS users are already using iOS10 (https://mixpanel.com/trends/#report/ios_10), it’s already a large user base that could potentially benefit from HLS with fMP4. Same applies for OSX where HLS with fMP4 is available on Safari 10 which has by far the largest market share compared with other Safari desktop versions (https://www.stetic.com/market-share/browser/).

What does this mean for SVOD, DRM use cases?

For SVOD and DRM use cases the situation is a little bit more complicated. HLS with fMP4 as well as DASH support MPEG Common Encryption (MPEG-CENC). MPEG-CENC supports two major encryption modes, AES-CTR (Counter Mode) and AES-CBC (Cipher-Block Chaining) which are incompatible. Fairplay with fMP4 HLS uses AES-CBC, while PlayReady and Widevine with fMP4 DASH are using AES-CTR. This makes it currently impossible to use a single content encoding for all DRM systems – for the moment. I think that this will change in the future and Widevine has already added AES-CBC support on Chromecast and Android N devices (https://www.widevine.com/product_news.html). When Widevine continues to broaden their AES-CBC support and Playready follows it will be possible to use single encrypted encoding for all platforms.

• Streaming – DASH and HLS manifests are compatible since the beginning because HLS could be seen as a subset of the DASH standard.
• Container Format – This was a problem in the past as HLS just supported MPEG-TS segments and HLS is required on iOS. DASH on the other hand is container format agnostic and due to the fact that all recent browsers only support fMP4 natively, DASH was used with fMP4 mainly and therefore everybody was required to generate and store both formats. With the recent changes it’s possible to use HLS and DASH with the same segments which reduces your encoding/packaging efforts and storage footprint by half.
• Encryption Mode – This is only needed for SVOD/DRM use cases but here we have still an incompatibility that needs to be resolved. As FairPlay uses CBC and Widevine and PlayReady mainly use AES-CTR this is a problem and you are still required to generate two versions of your segments – one for Fairplay and another one for Widevine and PlayReady. Nevertheless, as described Widevine has just recently announced that they now also support AES-CBC for Chromecast and Android N devices (https://www.widevine.com/product_news.html). If Widevine and PlayReady continue to broaden that support we will have single format.
• Codec – On the codec layer, both, H264 as well as H265 can be multiplexed in fMP4 and if the browser or device supports it the player can playback the content.

So I think we all agree now that HLS with fMP4 is something that is pretty useful. Content encoding, packaging and playback solutions are still very rare on the market for this workflow but Bitmovin already offers a complete end-to-end workflow through our Bitmovin API that allows you to encode and playback HLS with fMP4.

HLS fMP4 from Encoding to Playback

Let’s start with the actual encoding of your content. Our new API can be utilized easily, with the PHP, Python, and Go API clients, which are already available. We will use the PHP API Client and its example to show you how to create fMP4 HLS content.
1. Get the Bitmovin PHP API Client
You can either download it from Github or install it using composer. Please see the API client’s repository for more information about the setup.
2. Initialize the Bitmovin API Client
In order to use the API client, we have to initialize it first.

$client = new \Bitmovin\BitmovinClient('INSERT YOUR API KEY HERE');

That’s it^^. The client is now ready to use. Now, we can start preparing the configurations for your input source, output destination, and encoding, containing all the renditions, you want to create for the fMP4 HLS content.
3. Create a input configuration
For the sake of simplicity we are using an HTTP(S) input, although many other input sources such as AWS S3, Google Cloud Storage, Microsoft Azure, Aspera, and (S)FTP are also supported.

$videoUrl = 'http://example.com/path/to/your/movie.mp4';
$input = new HttpInput($videoUrl);

4. Create an output configuration
Here you define either to directly transfer your encoding results to your preferred storage type (
AWS S3, Google Cloud Storage, Microsoft Azure, (S)FTP), or you can store the encoding on your own Bitmovin storage. Direct transfer as well as storage are features of our new Bitmovin API. The first allows you to keep the turnaround time of your encoding very low, so your encoded content becomes available as quickly as possible on your own storage. The latter enables you to keep a backup of your encoding, transfer them later on to another storage and so on. If you want to do both, you can do that as well now :). As AWS S3 is a very common cloud storage, we will use it for this example.

$s3AccessKey = 'INSERT YOUR S3 ACCESS KEY HERE';
$s3SecretKey = 'INSERT YOUR S3 SECRET KEY HERE';
$s3BucketName = 'INSERT YOUR S3 BUCKET NAME HERE';
$s3Prefix = 'path/to/your/output/destination/';
$s3Output = new S3Output($s3AccessKey, $s3SecretKey, $s3BucketName, $s3Prefix);

5. Create an encoding profile configuration
An encoding profile configuration contains all the encoding related configurations for video/audio renditions as well as the encoding environment itself. It’s now possible to select which region and cloud provider should be used to encode your content. This enables you, to locate the encoding infrastructure, where your input and/or output bucket is located. This could improve your download and upload speeds and keeps your costs for egress traffic low.
Create an encoding profile configuration

$encodingProfileConfig = new EncodingProfileConfig();
$encodingProfileConfig->name = 'Test Encoding FMP4';
$encodingProfileConfig->cloudRegion = CloudRegion::AWS_EU_WEST_1;

Create an video quality configuration

$videoConfig = new H264VideoStreamConfig();
$videoConfig->input = $input;
$videoConfig->width = 1920;
$videoConfig->height = 1080;
$videoConfig->bitrate = 4800000;
$encodingProfileConfig->videoStreamConfigs[] = $videoConfig;

Create an audio quality configuration

$audioConfig = new AudioStreamConfig();
$audioConfig->input = $input;
$audioConfig->position = 1;
$audioConfig->bitrate = 128000;
$audioConfig->name = 'English';
$audioConfig->lang = 'en';
$encodingProfileConfig->audioStreamConfigs[] = $audioConfig;

You might have noticed, that you need to provide an input for each Audio/Video stream configuration. This is another feature of our new API. Now, you can provide several input files for an encoding, create all the renditions you need, and use them in your manifest afterwards. One typical use-case would be, if you have separate files for your video and audio tracks.
6. Select Output Formats
Besides HLS and DASH, we do support Smooth Streaming, MP4, TS and of course HLS fMP4 by now as well. We will only use “HLS fMP4” in this example, but you could create all of them at once with one encoding attempt too.

$outputFormats = array();
$outputFormats[] = new HlsFmp4OutputFormat();

7. Create Encoding configuration
This configuration object acts as a container for all the previous configurations from above and will be passed to the BitmovinClient in order to start the encoding.

$jobConfig = new JobConfig();
$jobConfig->output = $s3Output;
$jobConfig->encodingProfile = $encodingProfileConfig;
$jobConfig->outputFormat = $outputFormats;

8. Start the encoding
Finally, we can start the encoding. runJobAndWaitForCompletion() will return as soon as the encoding is finished and transferred/stored successfully.

$client->runJobAndWaitForCompletion($jobConfig);

By reading this line, the encoding might be finished already 🙂 If not, we can use the spare time to quickly setup a Bitmovin player example to playback the created content once it is finished.

Playback

Now that we successfully created our HLS fMP4 content, we want to play it as well. This is as simple as it is for HLS TS content, because it works exactly in the same way. The minimum player configuration would look like the following:

{
   key: "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
   source: {
       hls: "https://example.com/path/to/your/fmp4-hls-content-master-playlist.m3u8"
   }
}

A full example page would like the following:

<!DOCTYPE html>
<html>
<head>
   <meta charset="utf-8">
   <meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
   <title>V6 fMP4 HLS</title>
   <script src="https://bitmovin-a.akamaihd.net/bitmovin-player/stable/6/bitmovinplayer.min.js"></script>
</head>
<body>
<div id="unique-player-id"></div>
<script type="text/javascript">
   var player = bitmovin.player("unique-player-id");
   var conf = {
      key: "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
      source: {
          hls: "https://example.com/path/to/your/fmp4-hls-content-master-playlist.m3u8"
      }
   };
   player.setup(conf).then(function (value) {
       console.log("Successfully created bitmovin player instance");
   }, function (reason) {
       console.log("Error while creating bitmovin player instance");
   });
</script>
</body>
</html>

With this configuration for our player, and the encoded content from our encoding-service, you can now play your HLS fMP4 content everywhere, with one SINGLE format.

HLS fMP4 Demo

Now, we have HLS fMP4 content as well as a player example, which is ready to be used. Before we give that a try, let’s have a look what actually changed compared to a conventional HLS playlist.
As fragmented MP4 requires an initialization segment, which you might already know from MPEG-DASH, it has to be referenced using the EXT-X-MAP tag (see picture below). Then the player is able to play the fragmented MP4 segments properly. Therefore, we had to add the EXT-X-MAP tag in every variant playlist and use EXT-X-VERSION with value “6” or higher.

 

Does it play everywhere?

Below you can see the Bitmovin HTML5 Adaptive Streaming Player in action, playing Fragmented MP4 through an HLS manifest on every browser. This demo will work on iOS 10, macOS, tvOS and all recent browser, including Edge, FireFox, Chrome and Safari, etc.

How much storage space can be saved?

Based on the video of that blog post we created a quick storage and CDN savings case. In the past it was necessary to encode DASH and HLS TS to playback this video on all browsers and platforms. Now it’s possible to use just HLS fMP4 for everything. The table below shows the output in Bytes for each format and the combination of DASH and HLS TS. In terms of encoding output, packaging and storage you would save here 50.89% as you just need the HLS fMP4 output which is more than half of the actual DASH and HLS TS output as the HLS TS output is bigger than the DASH output due to the fact that TS is less efficient than fMP4.
HLS fMP4HLS TSDASH + HLS TSSavings

This advantage helps us also on the CDN as shown in the second row. HLS fMP4 is more efficient than HLS TS and for every user that you serve with content you would need less bits to deliver the same quality. In that example you would immediately save 3.49% on your CDN costs. This could be much more on other services but our TS output is already optimized for low overhead.

Conclusion

All in all, HLS with fMP4 is already very useful, as long as you don’t have to deal with DRM protected content. It enables you to greatly improve your storage consumption, which reduces your overall storage and CDN costs, and it enables you to use a more efficient output format across all devices. If you want to give it a try you could use our new Bitmovin API, just a request an API key for free and try it out.