Efficient Multi-Codec Support for OTT Services: H.264/HEVC/VP9 and/or AV1?

By encoding your videos using a multi-codec approach you can double the quality while still reducing your bandwidth consumption and maintaining maximum device reach.

In the spectrum of online video, you probably already know that H.264 (also known as AVC) is ubiquitous. Nearly every device and operating system supports decoding either on hardware or software. Although this compression technology is widely supported, which is a significant advantage, it is nowhere near as efficient as the next generation codecs in terms of compression rate.
According to Netflix’ experiments, H.265 (also known as HEVC) can deliver up to 50% bitrate savings when compared to previous generation codecs like H.264/AVC. Besides Apple devices equipped with iOS 11 and macOS High Sierra, H.265 is also supported by most 4K SmartTVs and Microsoft Edge for Windows 10 (with hardware decoder present in the device).
Similar to H.265, VP9 is another great option when it comes to reducing bandwidth consumption or delivering higher quality with the same bitrate. Bitrate savings can reach up to 50% compared to H.264, dramatically lowering your CDN costs. VP9 is supported on multiple platforms including Google Chrome, Firefox, Microsoft Edge and Android devices.
Roughly 83% of the internet users in the US could be reached with VP9 and HEVC. The remaining 17% would fall back to H.264 and you would still have complete coverage of every browser. The table below shows the browser market share for desktop and mobile.

Source: netmarketshare.
* Only available in Safari for iOS 11 and macOS High Sierra.
** Only available in Edge 14.14291.

Codec comparison

The figure below shows a side-by-side codec comparison. Maintaining the same visual quality, a Full HD content could be encoded at 4 Mbit/s (50% less than H.264) with H.265/HEVC and VP9.

Saving potential exemplified

As previously stated, roughly 83% of the internet users in the US could be reached with H.265 or VP9, thus benefiting both end users by consuming less bandwidth and also streaming companies by reducing CDN costs. Considering 50% of bandwidth reduction by leveraging these codecs your total saving potential would be of 42%. Under those circumstances, let’s picture the scenario described in the table below:

How to implement multi-codec streaming with Bitmovin

Now that we have established the effectiveness of a multi-codec approach on your online video strategy we can jump right into the “how to” section of the article. First of all let’s evaluate what we can do on the encoding side.
With the Bitmovin API you can encode your content with different codecs like H.264/AVC, H.265/HEVC, VP8, VP9, and recently also AV1. Ultimately the output can be MPEG-DASH, HLS, Microsoft Smooth and/or progressive MP4/WebM/TS.

To work with the Bitmovin API we have API clients for all the major programming languages. Visiting our Github page you will find all of them as well as code examples. For this particular topic of multi-codec streaming we have this Java API Client example which covers everything that is being presented in this article.

Each video and audio stream encoded using a given codec needs to be wrapped in a container. A container/muxing supports multiple streams, such as audio and video tracks. For adaptive streaming formats such as MPEG-DASH and HLS it is common to have separated muxings for audio and videos, however, for progressive formats obviously audio and video tracks needs to be muxed altogether. The table below shows the containers that can be used for each of the codecs.

As shown above, fMP4 muxings can be used to hold H.264 and H.265 segments for both HLS and MPEG-DASH. As a result these segments can be encoded only once and be referenced by a MPEG-DASH and HLS manifest, reducing your storage costs by 50%.
Adaptive streaming formats also allow us to include multiple codecs to the same manifest/playlist, that is the beauty of the solution. By encoding your content like this, you can hand over the logic of choosing the most appropriate codec to the player (you will find more details on that further down).
The examples below show how a multi-codec MPEG-DASH manifest and HLS playlist look like. For MPEG-DASH we use one AdaptationSet for each codec. On the other hand, for HLS we simply list all the variant streams with it’s codecs one below the other.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<MPD id="fc7573ef-1945-4eea-91b0-fe6e20e870ca" profiles="urn:mpeg:dash:profile:full:2011" type="static" mediaPresentationDuration="P0Y0M0DT0H0M46.067S" minBufferTime="P0Y0M0DT0H0M2.000S" bitmovin:version="1.19.0" xmlns="urn:mpeg:dash:schema:mpd:2011" xmlns:bitmovin="http://www.bitmovin.net/mpd/2015" xmlns:ns2="http://www.w3.org/1999/xlink">
    <Period id="437c9a5c-a403-499c-92ca-b24944a70b77" start="P0Y0M0DT0H0M0.000S">
        <AdaptationSet segmentAlignment="true" mimeType="video/mp4">
            <Representation id="990f791b-984f-4566-a3b1-77a0ffbe2e60" bandwidth="875000" width="854" height="480" frameRate="30" codecs="hvc1.1.c.L90.90">
                <SegmentTemplate media="video/875_h265_fmp4/segment_$Number$.m4s" initialization="video/875_h265_fmp4/init.mp4" duration="120000" startNumber="0" timescale="30000"/>
            </Representation>
            <Representation id="6c1b1d6f-8a59-4a97-9424-ee99bb17819b" bandwidth="1175000" width="1280" height="720" frameRate="30" codecs="hvc1.1.c.L93.90">
                <SegmentTemplate media="video/1175_h265_fmp4/segment_$Number$.m4s" initialization="video/1175_h265_fmp4/init.mp4" duration="120000" startNumber="0" timescale="30000"/>
            </Representation>
        </AdaptationSet>
        <AdaptationSet segmentAlignment="true" mimeType="video/webm">
            <Representation id="00f77da9-8658-4afb-9710-0dfb08e7d346" bandwidth="875000" width="854" height="480" frameRate="30" codecs="vp9">
                <SegmentTemplate media="video/875_vp9_webm/segment_$Number$.chk" initialization="video/875_vp9_webm/init.hdr" duration="120000" startNumber="0" timescale="30000"/>
            </Representation>
            <Representation id="60ad47c8-9b48-41a9-8c3d-16fe4c9e56b2" bandwidth="1175000" width="1280" height="720" frameRate="30" codecs="vp9">
                <SegmentTemplate media="video/1175_vp9_webm/segment_$Number$.chk" initialization="video/1175_vp9_webm/init.hdr" duration="120000" startNumber="0" timescale="30000"/>
            </Representation>
        </AdaptationSet>
        <AdaptationSet segmentAlignment="true" mimeType="video/mp4">
            <Representation id="e53c20bd-519d-4881-9e35-6dd1a3817eaf" bandwidth="1750000" width="854" height="480" frameRate="30" codecs="avc1.4D401F">
                <SegmentTemplate media="video/1750_h264_fmp4/segment_$Number$.m4s" initialization="video/1750_h264_fmp4/init.mp4" duration="120000" startNumber="0" timescale="30000"/>
            </Representation>
            <Representation id="8138b60d-2bc8-4eef-91b9-3ef9e27b6cbb" bandwidth="2350000" width="1280" height="720" frameRate="30" codecs="avc1.4D401F">
                <SegmentTemplate media="video/2350_h264_fmp4/segment_$Number$.m4s" initialization="video/2350_h264_fmp4/init.mp4" duration="120000" startNumber="0" timescale="30000"/>
            </Representation>
        </AdaptationSet>
        <AdaptationSet lang="en" segmentAlignment="true" mimeType="audio/mp4">
            <AudioChannelConfiguration schemeIdUri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011" value="2"/>
            <Representation id="3565543a-8524-41ac-98b4-006fcd21eaea" bandwidth="128000" audioSamplingRate="48000" codecs="mp4a.40.2">
                <SegmentTemplate media="audio/128_aac_fmp4/segment_$Number$.m4s" initialization="audio/128_aac_fmp4/init.mp4" duration="192000" startNumber="0" timescale="48000"/>
            </Representation>
        </AdaptationSet>
    </Period>
</MPD>

Example of a multi-codec MPEG-DASH manifest.

#EXTM3U
#EXT-X-INDEPENDENT-SEGMENTS
#EXT-X-VERSION:6
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audio_128",NAME="audio_128.m3u8",LANGUAGE="en",URI="audio_128.m3u8"
#EXT-X-STREAM-INF:BANDWIDTH=2101985,AVERAGE-BANDWIDTH=1796254,CODECS="avc1.4D401F,mp4a.40.2",RESOLUTION=854x480,AUDIO="audio_128"
video_h265_480p_1750.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=2739681,AVERAGE-BANDWIDTH=2372431,CODECS="avc1.4D401F,mp4a.40.2",RESOLUTION=1280x720,AUDIO="audio_128"
Video_h265_720p_2350.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=2049427,AVERAGE-BANDWIDTH=1824322,CODECS="hev1.1.6.L90.90,mp4a.40.2",RESOLUTION=854x480,AUDIO="audio_128"
video_h264_480p_1750.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=2706785,AVERAGE-BANDWIDTH=2421640,CODECS="hev1.1.6.L93.90,mp4a.40.2",RESOLUTION=1280x720,AUDIO="audio_128"
video_h264_720p_2350.m3u8

Example of a multi-codec HLS playlist.
On the playout side, as you can see in the code example below, by using Bitmovin Adaptive Player you just need to provide the dash, hls and/or progressive URLs as you would normally do. The player is responsible for identifying the most appropriate codec to deliver based on the browser/device capabilities.

var conf = {
  key: 'INSERTPROVIDEDKEYHERE',
  source: {
    dash       : 'http://path/to/mpd/file.mpd',
    hls        : 'http://path/to/hls/playlist/file.m3u8',
    progressive: [{
      url: 'http://path/to/mp4',
      type: 'video/mp4'
    }, {
      url: 'http://path/to/webm',
      type: 'video/webm'
    }]
  }
};
player.setup(conf).then(function(value) {
  // Success
}, function(reason) {
  // Error!
});

Example of a player configuration and setup.
There is also the possibility of having separated manifests/playlists for each codec you want to work with. In this case you would need to handle the logic of choosing the best source on your side – which would not be too complicated and you could have custom business rules as well.

Conclusion

As we were able to see, addressing multi-codec streaming can be a very effective measure towards reducing costs on bandwidth while delivering the same quality of experience to your viewers. Here at Bitmovin we treat this subject seriously and are constantly improving and adding new features such as Per-Title Encoding, Per-Scene Adaptation, Stream Conditions and others.
It is also important to mention that one of the complexities of multi-codec streaming is the increase of computational resources necessary to encode the same content, which usually also leads to higher turn-around times. However, by leveraging on Bitmovin Containerized Video Encoding, where we split the input file into multiple small parts to encode in parallel, this is just a matter of adding more nodes to the cluster, problem solved.
We are looking forward to helping you reduce your CDN costs or deliver higher qualities to your viewers.