Apple made waves across the video encoding and streaming communities when they announced the iPhone 15 Pro and 15 Pro Max would have a dedicated AV1 hardware decoder, making them the first Apple devices with official AV1 codec support. We’ve compiled all the details from their announcement, the HLS interest group, and product release notes to bring you everything you need to know about Apple AV1 codec support. If you’re looking for more information about AV1 playback on Android, Smart TVs and set-top boxes, you can find more information at https://bitmovin.com/av1-playback-support/. Otherwise, keep reading to learn more!

• Hints that Apple AV1 support was coming
• iPhone 15 Pro announcement
• More details about HDR, DRM, HLS and Safari support for AV1
• Apple M3 processor announcement
• Apple M4 processor iPad announcement
• AV1 Software Decoding Support?
• Ready to take advantage of AV1 Encoding?
• Related links

Hints that Apple AV1 support was coming

Prior to the iPhone 15 announcement in September 2023, there were several indications that Apple would eventually support AV1. Back in 2018, Apple joined the Alliance for Open Media, the organization responsible for creating and promoting AV1 encoding and many took it as a sign that Apple would eventually support AV1. More recently, updates to Apple’s AVFoundation core media framework showed the addition of a new global variable “kCMVideoCodecType_AV1“, and earlier in 2023, the Safari 16.4 Beta release notes actually showed AV1 support was coming, but it was removed without comment shortly after and never added to Safari 16. AV1 WebCodecs support did eventually become available as an experimental  option in the Safari Technology Preview, but enabling it didn’t seem to have any effect.

Still with all of these hints being dropped, the announcements of Apple’s M series of processors and the most recent update to the HLS draft specification in May 2023 all came and went with no mention of AV1. Everyone who was paying close attention and anticipating Apple AV1 support was left disappointed, especially knowing how much weight their decision carried for the rest of the streaming ecosystem. Overall AV1 adoption has been slower than many had hoped and expected, and Apple’s lack of support was often cited as a reason to wait and avoid updating video encoding stacks. 

iPhone 15 Pro announcement

This all changed on September 12, 2023, when Apple announced their new A17 Pro mobile processor would include support for AV1 hardware decoding. You can watch the full replay here, with the section about the 15 Pro’s new processor beginning at 1:01:20. VP of the Apple Silicon Engineering Group, Sribalan Santhanam presented the new A-series processor and shared details about the industry’s first 3 nm chip, including a 6-core CPU and a new Pro-class, 6-core GPU. It also has a 16-core neural engine that can process up to 35 trillion operations per second and run machine learning models on the device, without sending personal data to the cloud. It also includes a dedicated engine for Apple’s own ProRes codec in addition to the big one for video streaming services, the AV1 hardware decoder. 

“We also included a dedicated AV1 decoder, enabling more efficient and high-quality video experiences for streaming services.”

Sribalan Santhanam – VP, Apple Silicon Engineering Group

More details about HDR, DRM, HLS and Safari support for AV1

After the presentation, co-author of the HLS specification Roger Pantos shared more details via the hls-interest mailing list. He confirmed that indeed, that both the iPhone 15 Pro and 15 Pro Max would be the first Apple devices with hardware decoding support for AV1 video content. The dedicated hardware meant that in addition to Standard Dynamic Range (SDR) content, it would also support High Dynamic Range (HDR10) as well as content that was protected by FairPlay Streaming DRM, things that software decoders typically cannot handle well or securely. Playback would be supported in Apple’s native AVPlayer or AVSampleBufferDisplayLayer, including using Media Source Extensions (MSE), or Managed Media Source (MMS) as Apple calls their new version, under an experimental setting on iOS Safari.

HLS playback of AV1 will work without any new signaling requirements, just the regular CODEC and VIDEO-RANGE attributes. The SCORE attribute can also be used to force the playback client to prefer AV1 over other encodings, but renditions encoded with AVC and/or HEVC should still be included for older devices and AirPlay support. The WebKit blog provided more information about Safari 17.0, confirming support for the AV1 video codec was added on devices with hardware decoding support. They also shared this html code snippet for presenting single-file progressive video that has been encoded with AV1, HEVC and VP9, which allows the browser to choose the best option for playback. It should be noted that outside of very short clips, adaptive streaming with HLS is preferred over progressive streaming in order to provide the best quality of experience and bandwidth efficiency.

The ‘type’ attribute signals the type of container being used and ‘codecs’ parameter string lets the browser know which codec was used and other characteristics like profile, level, color space, bit depth and dynamic range. This informs the browser and lets it decide whether it supports those attributes or needs to fall back on an older codec. It’s also possible to use a simpler codecs=”av01”, but it’s best to provide as much detail as possible if you can. More information on the AV1 codecs parameter string from the Alliance for Open Media can be found here, and details about codec and profile parameters are available in this IETF doc. 

While not directly related to the Apple AV1 news, Safari 17.0 also added a new media player stats overlay similar to YouTube’s “stats for nerds”. This is a nice addition for video developers doing any troubleshooting and will be very helpful as people begin experimenting with adding AV1 encoding. It’s available to anyone who checks the “Show features for web developers” box in the advanced settings of Safari.  

Apple M3 processor announcement

In late October 2023, Apple announced their newest generation of desktop processors would include AV1 hardware decoders. This includes the M3, M3 Pro and M3 Max chips, meaning all new models of Macbooks, iMacs and desktop computers with an M3 processor will support AV1 video playback. Some were disappointed that the M3 did not also include support for AV1 encoding, but for video playback, the decoding is all that really matters, so this will be another nice wave of new devices that streaming services can target with AV1 encoded video. 

Apple M4 processor iPad announcement

Announced in May 2024, the new iPad Pro is powered by Apple’s latest system on a chip, the M4. The media engine of the M4 supports multiple codecs, including H.264, HEVC, ProRes and now AV1, making it the most advanced media processor ever in an iPad. With this, Apple continues their march toward full AV1 support. Will the Vision Pro 2 be next?

Apple AV1 Dolby Vision Support

Usually around the time of Apple’s World Wide Developer Conference there are some new updates or features around HLS and AVPlayer. During WWDC24, Apple shared a “What’s new in HTTP Live Streaming 2024” doc with several interesting new additions. For AV1 specifically, they called out support for using Dolby Vision Profile 10, which is Dolby’s 10-bit AV1 aware profile. Apple now supports 3 different Dolby Vision profiles: 10, 10.1 and 10.4. Profile 10 is “true” Dolby Vision, 10.1 is their backward compatible version of HDR10 and 10.4 their backward compatible version of Hybrid Log Gamma (HLG). For profiles 10.1 and 10.4, you need to use a SUPPLEMENTAL-CODECS attribute and the correct VIDEO-RANGE. For these, 10.1 should use ‘db1p’ and PQ, and 10.4 should use ‘db4h’ and HLG. The full example codec string they provided is: CODECS=”av01.0.13M.10.0.112″,SUPPLEMENTAL-CODECS=”dav1.10.09/db4h”,VIDEO-RANGE=HLG.

AV1 Software Decoding Support?

When Apple released the iPhone 6s with the A9 chip, it became the first iOS device to support HEVC(H.265) hardware decoding, which included support for FairPlay Streaming with HEVC. When this happened, they also included an HEVC software decoder as part of the next iOS and macOS updates for older devices without hardware support. While the software decoding didn’t support FairPlay Streaming, it was still a big boost for HEVC support and was one of the first things we wondered about after seeing the AV1 decoder announcement.

Unfortunately when asked, Roger Pantos shared that Apple would not be shipping an AV1 video software decoder at this time. He did confirm that iOS 17 does include some AV1 codec support, but only for still images using the Alliance for Open Media’s AVIF format. For now, we can only hope that AV1 video software decoding (like Meta is already using in their iOS apps) will be coming soon.

Ready to take advantage of AV1 Encoding?

Bitmovin has been ready for AV1 adoption to spread for some time now, dating back to 2017 when we partnered with Mozilla to enable AV1 playback in the Firefox browser using the Bitmovin Player. We’ve added AV1 codec support to our Per-Title and 3-pass encoding optimizations and just recently made AV1 encoding available in our dashboard UI, so now you can perform your first AV1 encode without any code, API calls, or configuration necessary! Bitmovin’s AV1 encoding has supported DASH streaming together with Widevine content protection for a long time, but we’ve now also added support for fMP4 in HLS playlists together with FairPlay content protection to take advantage of Apple AV1 support for premium content. It’s also available in our free trial, so there’s never been a better time to check it out and begin taking advantage of the bandwidth savings and quality improvements that AV1 can provide. 

Click here to start your free trial today!

Related links

• Read the latest info about our AV1 playback support and device testing here.
• Learn how using Bitmovin’s Per-Title Encoding together with AV1 can let you stream 4K video at bitrates that had been limited to Standard Definition with older codecs. 
• Check out our AV1 hub and download our datasheet to learn all about the codec’s development, performance and how it can lower your CDN costs.

The post Everything you need to know about Apple AV1 Support appeared first on Bitmovin.

Table of Contents

This post covers some recent updates, focusing on the new Firefox AV1 support in Encrypted Media Extensions. Bitmovin has been supporting and advocating for use of the AV1 codec for several years, even though there have been gaps in playback support preventing adoption for some workflows. Slowly but surely, those gaps are being filled and the reasons not to use AV1 are going away. Keep reading to learn more.

Firefox 125 adds support for encrypted AV1

A couple of years ago, Bitmovin began testing several different combinations of AV1 encoding, muxing and DRM support across browsers and playback devices. We were somewhat surprised to learn that even though Firefox was the first major browser to support AV1 playback, they had not implemented support for encrypted AV1 as they had for other codecs. We found there was actually an open bug/request filed 5 years ago. 

Shortly after we began watching closely, there was an update…

Ouch. Once the ticket got reassigned, Bitmovin got involved and gave our feedback that for premium/studio content, this support would be needed soon. We also provided a Widevine-protected sample for them to use in testing. Fast-forward to this spring, we saw some action on the ticket and support for AV1 with Encrypted Media Extensions was officially added to Firefox 125!

This means premium content workflows can now use AV1 on all of the major desktop browsers. Apple added support to Safari last fall, including with FairPlay Streaming, but for now it’s limited to devices with AV1 hardware decoders (iPhone 15 Pro, iPad Pro, new Macs with M3 processors).

Previous Bitmovin and Firefox AV1 collaboration

Way back in 2017, before the AV1 spec was finalized, Bitmovin and Firefox collaborated on the first HTML5 AV1 playback. Because the bitstream was still under development and subject to change, Bitmovin and Mozilla agreed on a common codec string to ensure compatibility between the version in the Bitmovin encoder and the decoder in Mozilla Firefox. It was made available in Mozilla’s experimental development version, Firefox Nightly, for users to manually enable. 

Even earlier in 2017, Bitmovin demonstrated the first broadcast quality AV1 live stream at NAB, winning a Best of Show award from Streaming Media Magazine. 

Other recent AV1 playback updates

Android adds dav1d decoder

In March 2024, VideoLAN’s “dav1d” became available to all Android devices running Android 12 or higher. Apps need to opt-in to using AV1 for now, but according to Google, most devices can at least keep up with software decoding of 720p 30fps video. YouTube initially opted to begin using dav1d on devices without a hardware decoder, but may have reverted that decision, likely due to battery concerns on phones. For plug-in Android devices, dav1d is still a great option and a welcome addition to the ecosystem.

iPad Pro gets AV1 playback support with M4 processor

In early May 2024, Apple continued their march toward full AV1 support with the announcement of their new M4 chip, which will power the new iPad Pro. The Media Engine of M4 is the most advanced to come to iPad, supporting several popular video codecs, like H.264, HEVC, and ProRes, in addition to AV1.

Ready to get started with AV1?

Bitmovin has added AV1 codec support to our Per-Title and 3-pass encoding optimizations and made AV1 encoding available in our dashboard UI, so now you can perform your first AV1 encode without any code, API calls, or configuration necessary! Bitmovin’s AV1 encoding has supported DASH streaming together with Widevine content protection for a long time, but we’ve now also added support for fMP4 in HLS playlists together with FairPlay content protection to take advantage of Apple AV1 support for premium content. It’s also available in our free trial, so there’s never been a better time to check it out and begin taking advantage of the bandwidth savings and quality improvements that AV1 can provide.

Related Links

Website: Bitmovin’s AV1 hub   

Blog: State of AV1 Playback Support

Blog: Everything you need to know about Apple’s AV1 Support

Blog: 4K video at SD bitrates with AV1

The post New Firefox AV1 support for Encrypted Media Extensions appeared first on Bitmovin.

In this post, I’ll be taking a look at the current state of AV1 playback support, covering which browsers, mobile devices, smart TVs, consoles and streaming sticks are compatible with the AV1 codec right now.  I’ll also touch on some of the incredible bandwidth savings companies like Netflix are seeing with AV1 and detail the latest announcements, rumors and speculation around future AV1 playback support.

AV1: The Story So Far (2017-2023)

Back in 2017, Bitmovin debuted the world’s first AV1 live encoding at the NAB Show in Las Vegas, earning a Best of NAB award. While it was an exciting proof of concept at the time, AV1 playback support was extremely limited and large-scale production usage wouldn’t come until years later. In 2020, YouTube and Netflix began delivering AV1 to the first compatible Android devices, and last year Netflix shared details about their expanded use of AV1 for 4K streams.

Netflix also published a report that showed over the course of one month in early 2022, 21% of their streamed content benefited from the most recent improvements in codec efficiency, like Per-Title optimized AV1 and HEVC. They estimated that without those improvements, total Netflix traffic globally would have been around 24% higher, proving that you can see massive bandwidth and overall cost savings by encoding just a portion of your most popular content with AV1.

Apple adds AV1 hardware decoding support to iPhone 15 Pro and new Macbooks

Many of us who have been tracking the adoption and progress of AV1 were disappointed when the announcements for Apple’s M-series processors over the past couple years did not include AV1 hardware decoding support. But on September 12, 2023, the big moment we’ve been waiting for finally arrived when Apple announced that the A17 Pro chip in their new iPhone 15 Pro would include a dedicated AV1 decoder. This is a big line in the sand for Apple and for the wider industry and will hopefully prove to be the day that revitalized interest and momentum for AV1 adoption across the industry.

“We also included a dedicated AV1 decoder, enabling more efficient and high-quality video experiences for streaming services.”

Sribalan Santhanam – VP, Apple Silicon Engineering Group

After the presentation, co-author of the HLS spec Roger Pantos shared more details via the hls-interest mailing list: 

The iPhone 15 Pro (both screen sizes) will be the first Apple product to support hardware decode of AV1 content. This includes SDR, HDR10, and content protected by FairPlay Streaming, played back through either AVPlayer or AVSampleBufferDisplayLayer (including MSE on Safari).

There is no new signaling necessary for HLS, just the regular content-specific values for the CODECS and VIDEO-RANGE attributes in the MVP. If you wish, you can use the SCORE attribute to make the client prefer AV1 over other encodings (but please continue to provide renditions encoded with AVC and/or HEVC for compatibility with earlier devices and AirPlay).

A month later in October 2023, Apple announced their newest generation of desktop processors would include AV1 hardware decoders. This includes the M3, M3 Pro and M3 Max chips, meaning all new models of Macbooks, iMacs and desktop computers with an M3 processor will also support AV1 video playback.

Earlier in 2023, while everyone was waiting for Apple to officially support AV1, Meta took matters into their own hands, sharing how they brought AV1 to their Reels videos for Facebook and Instagram, including on iOS devices. This became possible through ongoing open source software decoding efficiency improvements, in particular with the dav1d decoder, developed by VideoLAN. Meta also said they believe for their video products, AV1 is the most viable codec for the coming years. The image below shows how they significantly improved visual quality with AV1 over VP9 and H.264, while keeping the bitrate constant.

At Bitmovin we also believe in the potential of AV1 and have explored the possibilities of software decoding on mobile devices. At a recent internal hackathon, one of our senior software engineers, Roland Kákonyi, built a custom iOS player using the dav1d decoder that was able to decode and smoothly play 1080p AV1 content. We’ll continue exploring this further as a way to fill gaps in playback coverage for devices lacking hardware support.

AV1 Playback Support News in 2024

Following 2023’s big announcements from Apple, 2024 got off to a strong start with Android, Firefox and (again) Apple adding new AV1 playback support. The barriers and arguments against adopting AV1 continue falling, slowly, but surely.

Android adds dav1d decoder

In March 2024, VideoLAN’s “dav1d” became available to all Android devices running Android 12 or higher. Apps need to opt-in to using AV1 for now, but according to Google, most devices can at least keep up with software decoding of 720p 30fps video. YouTube initially opted to begin using dav1d on devices without a hardware decoder, but may have reverted that decision, likely due to battery concerns on phones. For plug-in Android devices, dav1d is still a great option and a welcome addition to the ecosystem.

Firefox adds AV1 support in Encrypted Media Extensions

While Firefox was the first major browser to support AV1 playback, a long-standing bug (or lack of implementation) prevented DRM-protected AV1 from playing. When Apple added support to Safari for HLS + FairPlay streaming, it meant Firefox was the only major browser that still did not support premium, secure content. That changed in April 2024, when Firefox 125 added AV1 support in encrypted media extensions, meaning Widewine-protected AV1 is now supported.

iPad Pro gets AV1 playback support with M4 processor

In early May 2024, Apple continued their march toward full AV1 support with the announcement of their new M4 chip, which will power the new iPad Pro. The Media Engine of M4 is the most advanced to come to iPad, supporting several popular video codecs, like H.264, HEVC, and ProRes, in addition to AV1.

Current State of AV1 Playback support

To answer the question of current playback support as thoroughly as possible, we created several sample streams with different combinations of containers, muxings and DRM. While there will be some exceptions and omissions, especially when you go back to the 2021 and 2020 models, I’ll use the emojis below to show the general level of support you can expect from these platforms and brands right now and give the full results of our direct testing in the table at the end. 

• Fully Supported – Successful AV1 playback with all test streams, including DRM

• Partial or Documented Support – Successfully played at least one, but not all of our test streams OR the product documentation claims AV1 playback support, but has not yet been verified by Bitmovin

• Not Supported – AV1 playback not supported here currently

Browsers and Operating Systems

*Safari 17 or later, when a hardware decoder is present

**AV1 is also supported in Chrome and Firefox on macOS

Generally speaking, the Chrome browser and Android ecosystem handle AV1 well across phones, tablets, smart TVs and set-top boxes/streaming sticks. Unfortunately, the same cannot be said for Safari and iOS where support had been lacking until the iPhone 15 Pro announcement.

Firefox was the first major browser to support AV1, and recently Firefox 125 added support for AV1 in Encrypted Media Extensions, meaning Widevine-protected content is now playable.

The Edge browser on Windows 10 and later supports AV1, but you may need to install the free AV1 Video Extension from the Microsoft Store. 

For more details about the specific versions and less common browsers that support AV1, check out the table from CanIUse.com here. 

Smart TVs

As mentioned, Android handles AV1 quite nicely, which also applies to the Smart TVs running Android TV and Google TV operating systems. These include Sony Google TV models from 2021 on and many Amazon Fire TV models as far back as 2020. (FireOS is based on Android)

Samsung TVs (and phones) from late 2020 onward have AV1 hardware decoders and were mentioned by Netflix as some of the first outlets for their 4K AV1 content. 

LG has developer documentation stating AV1 is supported for their UHD TVs and projectors running WebOS 5.0 and above, although our testing on some 2020 models was unsuccessful.

Consoles and Streaming Sticks

Amazon Fire TV Stick 4K Max

Playstation 4 Pro

Xbox One

Roku Streaming Stick 4K

Playstation 4 Pro was also called out by Netflix as one of the targets for their 4K AV1 streams and it takes advantage of GPU-accelerated decoding. Netflix didn’t publicly mention delivering AV1 to Xbox One, but the same decode libraries that the PS4 Pro uses were first made available for Xbox One, so it should be possible.

The Amazon Fire TV Stick 4K Max has AV1 + DRM support, making it one of the cheapest and best options for giving older 4K TVs an AV1 upgrade. 

Roku is a little bit of a gray area at the moment. Officially, they still do not support AV1 as an adaptive streaming video codec, but newer models like the Roku Ultra that have a USB port do support AV1 playback via USB media. There does appear to be some level of support for AV1 adaptive streaming, as the YouTube “stats for nerds” overlay reveals a combination of AV1 video and opus audio playing on many of the popular recommended videos. Hopefully wider support is coming, but in the meantime, did confirm successful playback of our single file “progressive” AV1 MP4 files on the Streaming Stick 4K.

Looking Ahead: Future AV1 Playback Support

Even with gaps in support on some platforms, there is plenty of opportunity to see tangible bandwidth savings and quality improvements from AV1 right now and thankfully, the future looks even brighter. Intel, AMD, Samsung and Qualcomm have all announced additional AV1 support coming at the chip level.

Will Apple add AV1 software decoding support for older devices? 

There have been several indications that Apple would eventually support AV1. Apple joined the Alliance for Open Media, the organization responsible for creating and promoting AV1 encoding, back in 2018, which many took as a sign that Apple would eventually support it. We’re hopeful that with the addition of AV1 hardware decoding support to the iPhone Pro 15, iPad Pro and Macbooks, Apple will also add official HLS support and fallback software decoding for older devices that are capable.

Conclusion

While AV1 support and adoption has been on the rise and we’ve seen some encouraging announcements, universal support like we have with H.264 is just not there yet. That means AV1 will need to be part of a multi-codec approach for the foreseeable future, but that’s ok! Not that long ago, it took millions of views to offset the higher encoding costs of AV1, but with recent improvements, we’ve seen the break-even point drop to as low as 4,000 views! So for a whole lot of content, encoding with AV1 can already save you money right now and those savings will only increase as more supporting devices become available. 

Ready to get started with AV1 encoding? You can try it for free with a Bitmovin Trial, sign up here!

	fMP4 (DASH)	fMP4 with Widevine and Playready (DASH)	Single file “progressive” MP4 (.mp4)	Single file “progressive” MP4 + Widevine (DASH)	WebM (DASH)	WebM + Widevine (DASH)	Single file “progressive” WebM (DASH)	Single file “progressive” WebM + Widevine (DASH)
Chrome
Edge
Firefox
Safari
Android Native
Android Web
iOS
Fire TV Max
Fire TV Max Web (Silk Browser)
Roku Streaming Stick 4K
Samsung Tizen (2020 and 2021)

The post The State of AV1 Playback Support: 2024 appeared first on Bitmovin.

The AV1 Video Codec

First things first, what is AV1 and where does it come from? In September 2015 the Alliance for Open Media (AOMedia) was founded by leading companies from various industries with an association with media technology. Among them are browser vendors like Google, Mozilla, and Microsoft, hardware vendors like AMD, ARM, Intel, and NVIDIA, and content providers like Amazon and Netflix. The goal of the AOMedia is to develop an open, royalty-free, next-generation video coding format that is:

• Interoperable and open
• Optimized for the Internet
• Scalable to any modern device at any bandwidth
• Designed with a low computational footprint and optimized for hardware
• Capable of consistent, highest-quality, real-time video delivery, and
• Flexible for both commercial and non-commercial content, including user-generated content.

The new video coding format AOMedia Video 1 (AV1) is meant to replace Google’s VP9 and compete with HEVC/H265 from MPEG. The Alliance is targeting an improvement of about 50% over VP9/HEVC with only reasonable increases in encoding and playback complexity.
When comparing AV1 with HEVC, probably the biggest competitive advantage of AV1 will be that it is royalty-free, especially if we look at the still very uncertain royalty situation with HEVC. Currently, there are two patent pools with MPEG LA and MPEG Advance, plus some unknown HEVC IP owners who have not joined a pool yet. In the end, nobody will know how much you will need to pay in royalties for HEVC. This situation is obviously not satisfactory for the industry and especially, encoding, distribution, content, and hardware companies. (Download the AV1 Datasheet)

Bitmovin and AV1 Video Encoding as of 2024

We have made improvements to the core AV1 encoder in itself and have extensively benchmarked it against multiple practical use cases. The turnaround time and speed of encoding have improved by several orders of magnitude. And in regards to the quality, for the encoder version release v2.110.0, we found that AV1 can offer the same visual quality at 50% less bitrate for H.264/AVC and 30% less bitrate for H.265/HEVC respectively. That is a pretty significant gain!
In addition to the improvements to the core encoder itself, we have integrated AV1 with all the popular features that our customers have come to love. Here is a quick rundown : 

• Since encoder version 2.104.0, 3-pass encoding with AV1 is generally available. We have found that three-pass AV1 video encoding provides significantly better bitrate distribution compared to the regular 2-pass encoding.
• Since encoder version 2.109.0, Per-Title encoding with AV1 is available now. Per-Title is one of our biggest competitive advantages. We are proud to offer this also for AV1. 
• Since encoder version 2.110.0, AV1 video encoding offers three smart presets. This allows customers to choose an optimal tradeoff between the quality and speed of the AV1 encodings. 
• Since encoder version 2.187.0, AV1 video encoding can be used in HLS playlists, together with FairPlay content protection. This enables support for AV1 playback on compatible Apple devices like the iPhone 15 Pro and new laptops with Apple’s M3 processor.

Also at Bitmovin, we like to keep our promises . We promised seven years ago that we will not stop innovating around AV1 and that we will enable our customers in the best possible way with our AV1 solutions. We are excited to announce that we have kept our end of the bargain. We have developed two patent-pending technologies around AV1. We cannot delve into the details now but just to tease you out, it significantly improves the turnaround times for Per-Title and 3-pass encodings. Keep watching this space for more details about this soon!
And here is the cherry on top of all this. It’s easy to get all this awesome Per-Title ABR encoding together with the AV1 codec and DASH packaging in a SINGLE API call! Yes, it’s not a typo. We said SINGLE. Can you believe that !? What are you waiting for you? It’s easier than ever to get started with AV1. Try it and reach out to us if you have any questions! We are happy and excited to get you onboard with AV1.

How AV1 Video Encoding Development Works

The AV1 codec has its roots in the codebase of Google’s VP9/VP10 codec with an additional 77 experimental coding tools that have been added and are under consideration. Out of that 77 experimental coding tools, only 8 are currently enabled by default (adapt_scan, ref_mv, filter_7bit, reference_buffer, delte_q, tile_groups, rect_tx, cdef), but the performance of the codec is already appealing. The final goal is to get as many promising coding tools into the final version of the codec and afterward freeze the bitstream specification.
The following procedure explains the high-level process on how experiments can be added to the AV1 codec:

1. Coding tools are added as experiments into the AV1 codebase. They are controlled at build-time by flags (e.g., –enable-experimental –enable-<experiment-name>).
2. The hardware team (group of hardware members inside of AOMedia) reviews the experiments to ensure it can be implemented in hardware.
3. Each experiment needs to pass an IP review to ensure no IPs are violated.
4. Once reviews are passed the experiment can be enabled by default.

As of today, it is not sure which experiments will make it into the final codec. However, we want to highlight a few that look promising today:

Directional Deringing

It is an effective algorithm for removing ringing artifacts from a coded frame. It plugs in right at the end of the decoding process, so it is easy to integrate. Blocks are searched for an overall direction that is taken into account when applying a conditional replacement filter (CRF) to reduce the risk of blurring and only take obvious ringing patterns into account. It is currently enabled by default.

PVQ (Perceptual Vector Quantization)

This experiment was originally developed for the Daala codec and has the potential to bring a lot of gains, however, it is also quite difficult to integrate into AV1 because PVQ interacts with many other parts of a codec. Compared to the usual scalar quantization, PVQ offers a lot more flexibility to control quantization. It makes techniques like Chroma from Luma or Activity Masking easier. Activity Masking is trying to provide better resolution in low contrast areas. This can be achieved by varying the codebook which is possible with PVQ.

Chroma from Luma (CfL)

CfL is based on a rather simple idea: Take advantage of the fact that edges in the chroma plane are usually well correlated with those in the luma plane. As CfL works entirely in the frequency domain, it can be easily implemented using PVQ. Using PVQ, the chroma coefficients can be predicted from injected luma coefficients. It is a very promising tool as it is quite simple to compute and provides nice benefits with much cleaner colors.

Bitmovin AV1 VoD and Live Encoding

The Bitmovin encoding service now supports AV1 video encoding for VoD and Live. It is possible to encode AV1 with our cloud encoding service. Currently, AV1 video encoding with common encoding tools is a very time-consuming process, as can be seen in the below screenshot taken from a Lenovo T540p notebook with an i7-4800MQ, 8GB RAM running Ubuntu 14.04. It would take 8 hours and 42 minutes to encode a 1080p@24fps 40-second long sequence (Tears of Steel Teaser) with a target bitrate of 1.5Mbps.

The encoding runs with about 1.93 fpm (frames per minute) which would translate to 0.032 fps (frames per second). If you want to achieve real-time with 24 fps you would need at least 746 times the computing power on a single machine, which is not very practical in a real-world scenario. Clearly, we need another approach to encode with reasonable speeds, especially when it comes to live streaming.
Thanks to our chunk-based encoding approach that allows us to scale a single encoding among multiple instances we can encode AV1 with reasonable turnaround times and it’s also possible to use AV1 for live streams. Our chunked encoding allows us to speed up the encoding almost linearly with the number of instances that are added to the encoding cluster and this approach works with our cloud encoding the same way it works with our on-premise setups that are based on Kubernetes and Docker. Consequently, we can reach the same encoding speeds for AV1 that our customers have come to expect for H264, VP9, and HEVC encoding, which makes the codec effectively usable for media companies and content providers throughout the industry.

We also encoded the ToS teaser with our AV1 encoder in the cloud with the default configuration where we achieved 7 fps, which is about 219 times faster than what was achieved in the test with the Lenovo notebook. This is already pretty impressive however, we were not satisfied with the speed as it was still below real-time. So we tried with an enterprise set-up by just adding more instances to the encoding process. The resulting encoding speed was at 36 fps, which is about 1125 times faster than with the single Lenovo notebook.

In addition, we don’t have to compromise on quality for speed because our encoder does not need to sacrifice quality to reach a certain speed on a single instance as other encoding vendors typically do. With our approach we are not bound to the hardware restrictions of a single instance, we can add more instances to an encoding cluster to generate the quality that our customers have configured in a reasonable time or in real-time for live streams. With our chunk-based implementation of the AV1 video codec, we can encode videos with AV1 even faster than in real-time without compromising quality.

How to implement an AV1 Livestream

In most cases, to run live stream encodings you would need around 4 to 15 Mbps with traditional codecs like H264 to deliver the same quality. So AV1 could reduce your CDN and storage cost by up to 10x.
The setup of our AV1 live workflow that we will showcase consists of the following components:

• OBS RTMP mezzanine stream, 12Mbps 1080p@30fps
• Bitmovin Distributed AV1 Cloud Encoder running in Google Cloud receives an RTMP ingest and transcodes to 1.5Mbps 1080p@30fps segmented WebM. Segments will be directly transferred to a Google Cloud Storage bucket.
• The Bitmovin Distributed AV1 Cloud Encoder also generates HLS and MPEG-DASH manifests that will be transferred to the Google Cloud Storage bucket. Enabled experiments of the AV1 codec are: adapt_scan, ref_mv, filter_7bit, reference_buffer, delte_q, tile_groups, rect_tx, cdef
• Native playback on a desktop with a Bitmovin Player based on aomdec and ffplay

Our AV1 encoder generates WebM segmented output that could be used with HLS or MPEG-DASH for VoD and Live. However, as AV1 is currently not supported by any browser, we had to write our own player that is able to playback our AV1 live stream. We updated the aomdec application to be able to download and decode the AV1 chunks which can be seen in the left console window. Fortunately, decoding is not as resource intensive as the encoding, which allows you to decode the AV1 stream on normal hardware without special requirements, e.g., the same Lenovo notebook (i7-4800MQ, 8GB RAM running Ubuntu 14.04) that was not capable of encoding this video just near to realtime could easily playback AV1 in software. After the decoding step, we pipe the decoded YUV frames to ffplay to display the stream in a window as you see in the screenshot above. We plan to contribute this functionality back to aomdec after a technical cleanup of the current implementation.

A Practical Quality Comparison

Although the bitstream from AV1 is not finalized yet and much work needs to be done to further improve the quality of the codec, we wanted to get a snapshot of the current state and compare its quality with AVC/H264, HEVC/H265, and VP9. For that purpose, we made two different quality comparisons, the first one with two objective metrics, PSNR and SSIM. PSNR does not always correlate well with perceived quality but is the de-facto standard for video quality comparisons. SSIM is a perception-based quality metric that should give better results in regard to perceived quality.
For the second comparison, we chose to make a side-by-side quality comparison between AV1 and the other codecs. This quality comparison targets a practical use case where the resulting content can be used for Adaptive Bitrate Streaming (ABR). Therefore we have used a fixed Group of Pictures (GOP) size for our experiments and also used Variable Bitrate (VBR) encodings with a target bitrate. This approach is established in the industry but results can vary from scientific evaluations that purely target abstract use cases and theoretical encoder performance through the HM (HEVC reference software) and JM (AVC reference software) reference software that has no practical relevance in the industry.
Let’s first start with the objective quality comparison with PSNR. We encoded the open-source movie Sintel from the Blender Foundation with VBR to the following target bitrates: 100Kbps, 250Kbps, 500Kbps, 1Mbps, 2Mbps, 4Mbps and calculated PSNR and SSIM for the bitrate that has actually been achieved by the individual codec (typical codecs in VBR mode do not hit the target bitrate exactly).
The following encoding settings for the different codecs were used in the Bitmovin Encoding Service:

• AVC/H264:
  GOP Size: 96 frames (4 seconds), Me_range: 16, Cabac: true, B-Adapt: 2, Me: UMH, Rc-Lookahead: 50, Subme: 8, Trellis: 1, Partitions: All, BFrames: 3, ReferenceFrames: 5, Profile: High, Direct-Pred: Auto

• HEVC/H265:
  GOP Size: 96 frames (4 seconds), Sao: 1, B-Adapt: 2, CTU: 64, Profile: Main, BFrames: 4, Rc-Lookahead: 25, WeightP: 1, MeRange: 57, Ref: 4, Subme: 3, Tu-Inter-Depth: 1, Me: 3, No-WeightB: 1, Tu-Intra-Depth: 1

• VP9:
  GOP Size: 96 frames (4 seconds), Cpu-used: 1, Tile-columns: 4, Arnr-Type: Centered, Threads: 4, Arnr-maxframes: 0, Quality: Good, Frame-Parallel: 0, AQ-Mode: none, Arnr-Strength: 3, Tile-Rows: 0

• AV1:
  Build f3477635d3d44a2448b5298255ee054fa71d7ad9, Enabled experiments by default: adapt_scan, ref_mv, filter_7bit, reference_buffer, delte_q, tile_groups, rect_tx, cdef
  Passes: 1, Quality: Good, Threads: 1, Cpu-used: 1, KeyFrame-Mode: Auto, Lag-In-Frames: 25, End-Usage: VBR

The above diagram clearly shows that AV1 already outperforms all the other codecs for each bitrate setting. For bitrates from 1Mbps and higher the quality difference is already pretty big (> 0.5db which is usually clearly visible). VP9 and HEVC/H265 are very similar from a PSNR perspective, however, VP9 was the codec that overshot the target bitrate by far the most.

We also compared the four codecs with SSIM. The results can be seen in the above diagram and are quite similar to PSNR with some slight differences. AV1 is still the best performing codec over all bitrates, and AVC/H264 lags behind. However, interestingly AVC/H264 catches up with increased bitrate. An explanation for that could be that in the higher bitrates we can reach nearly the quality of the source material with all codecs, which results in only minor differences between the codecs.
Additionally, we created several side-by-side quality comparisons where we experimentally changed the target bitrate for each codec to reach an average of 500 Kbps. Below you can see the quality comparisons between the encodings comparing the quality of Bitmovin AV1 video encoding with AVC/H264, HEVC/H265, and VP9. We used the well-known Tears of Steel teaser that is 40 seconds long with a 1080p resolution for the comparison, selecting a complex scene that is hard to encode.

When comparing AV1 video encoding with AVC/H264 the quality difference is very obvious as expected. We can clearly see multiple encoding artifacts and blocking in the right part of the image that has been encoded with AVC/H264. In contrast, the left part with AV1 Video Encoding looks much cleaner without obvious encoding artifacts.

Looking at the quality difference between AV1 and VP9 it is not as obvious as with AVC/H264, but still quite visible. Especially the borders of the tiles of the sphere show encoding artifacts and the overall picture in VP9 seems to have quite some noise. We can also identify some blocking artifacts that are not visible in AV1.

HEVC/H265 visually looks a bit better than VP9, however, it still has visible encoding artifacts, especially in the lower part of the image and around the arm of the guy with the red coat. When we look closely at the arm we can see that the color is not encoded as nicely as with AV1 and shows some noise.

Conclusion

Bitmovin’s culture and vision have always been to be a technology leader and our passion for video means we consistently tackle the most complex video problems. Why? Because it’s fun and challenging and our team loves a challenge!
Besides that, there are already use cases for an AV1 video encoding where you could use it as your mezzanine format to preserve a high-quality version of your video at a low bit rate that can be used to create your adaptive bitrate renditions or other formats. Using AV1 for that use case would decrease your storage footprint and speed up transfer times inside of your data center or for upload to the cloud.
Furthermore, with the companies behind AOMedia, like AMD, ARM, Intel, NVIDIA, Google, Microsoft, Mozilla, Netflix, and Amazon, it should not take too long to get broad support for AV1. AMD, Intel, and NVIDIA cover the desktop market quite nicely, and ARM and Intel the mobile market. Additionally, the major browser vendors, Google, Microsoft, and Mozilla will make sure that the codec finds its way into the browsers soon after the bitstream freeze. Google, Netflix, and Amazon will make sure that AV1 content will be available quickly and that will further drive adoption and hardware support.
AV1 is the next generation video codec and it’s on track to deliver a 30% improvement over VP9 & HEVC – Learn More

More AV1 Resources:

• 4K Video at SD Bitrates with AV1
• Bitmovin’s AV1 Encoding Gift Guide
• Cool New Video Tools: Five Encoding Advancements Coming in AV1

The post Bitmovin Improves Support AV1 Video Encoding for VoD appeared first on Bitmovin.

Table of Contents

Introduction

The post will explain how Per-Title Encoding works and the advantages of using Per-Title Encoding compared to using the same bitrate ladder for all your content. Per-Title often requires fewer ABR ladder renditions and lower bitrates that translate into storage, egress and CDN cost savings. It also improves QoE with less buffering and quality drops for viewers, along with better visual quality. On top of that, it can make 4K streaming viable, turning it from a loss leader and financial burden into a revenue generator. Keep reading to learn more. 

Per-Title Encoding is key for cutting streaming costs

For the past couple of years, “controlling costs” has been among the top challenges for video streaming, according to the results of Bitmovin’s annual video developer survey. While the pandemic years created a boom for streaming services and content creation, things have now shifted toward cost-cutting in a few different ways. Several platforms have cut back their budgets for original content and are removing shows and films from their libraries to save on licensing and other operational costs. 

Another trend highlighted by industry analyst Dan Rayburn, has been the lowering of bitrates, including removal of 4K streaming in some cases. Services that do still offer 4K often restrict it to their highest-priced subscription tier. Back in 2014, Dan called out the cost and QoS challenges services would face when delivering 4K video, and many are still struggling with that reality, especially those using a fixed bitrate ladder for their encoding. 

Per-Title Encoding can have a huge impact on 4K content, something that can be seen in the recommended internet connection speeds for 4K streaming:

Netflix: 15 Mbps (they use their own version of per-title encoding)

Disney+: 25 Mbps

Paramount+: 25 Mbps

Max: 50+ Mbps 

For long form content that gets even in the tens of thousands views, the difference between 15 Mbps and 25 or 50 Mbps will add up quickly in the form of excess network egress and CDN costs. With non-optimized encoding at those high bitrates, a viral hit that ends up getting hundreds of thousands or millions of views can end up being a financial burden. Using Per-Title Encoding ensures each video uses only the bits needed for its content and complexity and when combined with more advanced codecs like HEVC and AV1, it can make a game-changing difference. When Bitmovin added support for using Per-Title Encoding with the AV1 codec, I was shocked to see just how low the bitrate could go (often under 2 Mbps). 

How does Per-Title Encoding work?

In 2012, Bitmovin’s co-founders published a research paper titled “Dynamic Adaptive Streaming over HTTP Dataset” that, among other things, provided data for per-genre encoding that would further evolve into Bitmovin’s Per-Title Encoding. Per-Title Encoding is an optimization of adaptive bitrate encoding that analyzes the complexity of a video file and determines the encoding settings needed to maintain the highest level of visual quality together with the most efficient adaptive bitrate ladder. 

In 2015, Netflix published a tech blog that detailed their research and development of their own per-title encoding. Through brute force encoding of content at different resolutions and quality levels, they found that the ideal adaptive bitrate ladder for each video would form a smooth convex hull when plotting quality vs bitrate. When the bitrate and resolution pairs in their ABR ladder fell along the convex hull, it maximized quality for the viewer and meant that data was being distributed efficiently. Bitmovin’s Per-Title complexity analysis spares you the excessive testing and experimentation and automatically determines the ideal ABR ladder and convex hull for each file. 

Per-Title Encoding ABR ladder vs fixed bitrate ladder

The graph below shows how Per-Title Encoding provides better QoE with lower bitrates than the competition’s static bitrate ladder for a 4K source. Per-Title Encoding matches the source at 3840x2160p, with a bitrate of 6050 kbps and a VMAF score of 95.5. The static ladder is capped at 1080p and requires 7830 kbps for a lower VMAF score of 90.9. That’s 22.7% bitrate savings with better quality by using Per-Title.

The next example uses the HEVC codec for the customer’s UHD ladder vs Bitmovin Per-Title Encoding. The highest rendition on the Per-Title ladder only needs 1.9 Mbps to hit a VMAF score of 94.9, while the static ladder uses 15 Mbps, an increase of 13.1 Mbps in bandwidth for an undetectable VMAF difference. This equates to 87% savings on the CDN bill for viewers of the top rendition, without sacrificing quality. 

With a duration of 44:39, the top rendition for Per-Title would mean 0.622 GB in data transfer while the top rendition for the fixed ladder would require 5.023 GB. For popular content tens of thousands of views (or more) those savings add up quickly. At a time when some services are removing 4K renditions, these optimizations make it feasible to provide UHD and improve margins on premium subscription tiers. 

Next we have some medium-complexity 1080p content where using Bitmovin Per-Title with a more advanced codec like HEVC can make a huge difference. Throughout the ladder, using Bitmovin Per-Title with H.264 provides some quality gains and bitrate savings compared to the customer’s static ladder with ffmpeg, but the results from Per-Title with HEVC highlight the impact of using a newer generation codec. HEVC delivers 1080p in the 90+ VMAF range with only 2 Mbps while ffmpeg with H.264 needs over 6.5 Mbps for the same quality. That’s around 70% bandwidth savings for viewers of the top rendition. At the lower end of the spectrum, a viewer with 1 Mbps available bandwidth would be limited to 432p with the static H.264 ladder, but would still receive 1080p with Per-Title HEVC.

Storage savings with Per-Title Encoding

Bitmovin’s Per-Title Encoding can deliver massive storage savings when compared to fixed bitrate ladders, by removing unnecessary renditions from the ABR ladder and ensuring the most efficient bitrate is used for each piece of content. The chart below shows the potential savings on your storage bill from using Per-Title Encoding over a fixed ladder with AVC encoding. 

Improve quality without increasing bitrates 

Per-Title Encoding can also improve quality without needing to use additional data. The chart below references our customer’s fixed ABR ladder using the AVC codec and shows the quality improvements (% VMAF score) that Bitmovin’s Per-Title provided with different codecs at the same bitrate. 

Bitmovin Smart Chunking prevents lower quality outlier frames

The graphs below plot the VMAF quality scores of every frame in our customer’s sample content. Bitmovin’s smart chunking virtually eliminates all of the lower quality outlier frames that are present in our competitor’s encoding and would be noticeable by viewers. Smart Chunking is now active for all Bitmovin VOD Encoding without any additional configuration or cost to the user.

Conclusion

In the past, balancing cost and quality has always been a tradeoff, but using Per-Title Encoding may be the single most effective way for streaming services to reduce their total cost of ownership without sacrificing their viewers’ quality of experience. With consumers having an abundance of options, the QoE improvements Per-Title provides can mean the difference between renewal and churn and its cost savings can tip the scales toward profitability. With streaming firmly in a cost conscious era, using Per-Title Encoding makes more sense than ever before.   

Ready to see what difference Per-Title Encoding can make with your content? Anyone can test it out for free with no coding required using our VOD encoding wizard. We also have a comparison tool in our dashboard where you can input your own content or use common test videos. Try it out today!

Related Links

Choosing the Best Per-Title Encoding Technology

What is Per-Title Encoding and how does it work 

How to Create a Per-Title Encoding

Advanced Per-Title configuration 

The post Game-Changing Savings with Per-Title Encoding appeared first on Bitmovin.

This post will provide some background information on video codecs and the pros and cons of the most common codecs used by streaming services. We’ll discuss why you should be taking advantage of newer codecs and the many benefits they can provide. Finally, we’ll provide a step-by-step guide through the factors and questions you should consider when evaluating or adding AV1 encoding to your workflow.

What are video codecs?

The word codec is a combination of the words coder and decoder. Video codecs are used to reduce or compress the size of video files for storage and transmission, because raw source files produced by professional studios or cameras use too much data to be delivered smoothly over the internet. For video streaming purposes, the master file or live source is encoded for transmission, then decoded for playback on the end-user device. This encoding typically involves lossy compression, meaning the overall file size is reduced, often significantly, with a tradeoff of slightly lower visual quality. How noticeable that quality reduction is and the amount of data compression possible depends on the codec and settings that were used.

What is the best video codec?

Like with many technical questions, the frustrating, but most accurate answer is “It depends.” Each codec has its own pros and cons in terms of cost, performance and support, so the best codec (or codecs) can differ based on your company’s goals, applications and business model. Let’s take a closer look at some of the high level pros and cons of the most common codecs used by streaming platforms.

H.264/AVC

–  Almost universal playback compatibility with nearly 20 years in the field. 

–  It’s almost 20 years old, and several newer codecs offer better compression and can produce higher quality video while using less data, providing a better viewing experience and reducing your long-term storage and delivery costs. 

H.265/HEVC

–  MPEG’s successor to H.264/AVC can provide higher quality per bit and makes it possible to deliver HD,  4K and even 8K video to a wider audience, while using less bandwidth. H.265/HEVC is usually encoded with a higher bit depth than H.264/AVC, making high dynamic range (HDR) support possible, so most Dolby Vision streaming workflows use H.265/HEVC.

– Royalty fees and complex licensing terms for H.265/HEVC have led some content creators and streaming services to avoid it. There are multiple patent pools, along with some individual companies that claim to own patents covering essential parts of the standard, creating uncertainty for end users around their liability and obligations.

VP9 

– Google’s open source VP9 is a royalty-free alternative to H.265/HEVC with similar performance in terms of compression efficiency. Platforms like YouTube and Facebook use VP9 to offer higher resolution video than H.264/AVC and improve video quality over low bandwidth, which is especially important for viewing on mobile devices. 

– VP9 is over 10 years old and while it is still widely used and supported, most of the current innovation and new development is focused on its successor, AV1.

AV1

– AV1 offers the best compression performance of any of the codecs discussed here. On average it can achieve the same visual quality using ~50% less data than H.264/AVC and ~30% less data than H.265/HEVC and VP9. Those are significant savings that really add up over time and millions of views. YouTube and Netflix both use AV1 for their most popular HD and 4K content as a way to enable high-res viewing for people on slower connections, improving quality of experience and drastically reducing data usage, which lowers their egress and CDN costs.  

– Even though it’s a few years old, adoption of AV1 has been lagging behind initial expectations, mostly because of slower than expected rollout of playback and decoding support. Because of that, anyone encoding with AV1 will still need to provide backup H.264/AVC streams in the near future. However, recently launched products like the iPhone 15 Pro, Apple’s M3 desktop processor and the Meta Quest 3 headset all include AV1 decoding support, so hopefully the tide is turning and AV1 adoption will ramp up in the coming months.

Why should I use newer codecs?

In general, newer codecs provide better compression efficiency than their predecessors, using less bits to deliver the same level of quality. They can also support higher resolutions and wider color and contrast ranges than earlier generations. This has multiple benefits for streaming services and their viewers.

Better visual quality at lower bitrates

When delivering video to mobile phones, to viewers on shared, congested wifi or in areas with limited bandwidth, the tradeoff between quality and bitrate becomes especially apparent. This is where the compression performance of newer codecs can really stand out. Using a newer codec like AV1 can mean the difference between acceptable quality and a pixelated mess as you can see in the screenshot below from Facebook.

Improved quality of experience and less buffering for all viewers

It’s not just the low bitrate streams where newer codecs can improve quality of experience. The ability to deliver higher quality with less bits improves QoE across all adaptive bitrate renditions. At Demuxed 2023, Netflix reported that using AV1 not only reduced network bandwidth, but also lowered rebuffer rates and play start delay, while providing up to 38% reduction in quality drops compared to other codecs. They were also able to improve VMAF quality scores by up to 10 points with AV1. 

Higher resolutions and more advanced features

H.264/AVC can technically be used for HD and 4K encoding, but you typically need extremely high bitrates to achieve the quality viewers expect from HD and UHD content. These higher bitrates can lead to more rebuffering and QoE issues, not to mention higher prices for storage and delivery. Most H.264/AVC workflows are limited to 8-bit encoding and decoding, which means it can’t really be used for HDR. H.265/HEVC, VP9 and AV1 can all handle 4K and 8K resolutions and support 10-bit and higher encoding, making them a better choice for premium content tiers. Netflix reported that using AV1 led to a 5% increase in 4K streaming time, so it can make a meaningful difference for the viewing experience and how your platform is perceived by users. 

Long term cost savings

In addition to improving QoE, the better compression performance and lower bitrates required by newer codecs will also save you money over the lifetime of your video content. Even though the encoding is more complex and may cost a little more up front, the smaller file sizes can reduce your storage footprint and drastically cut your bandwidth on network egress and CDN delivery charges. For popular content receiving tens of thousands or millions of views, the overall savings can be substantial.

Should I use more than one codec?

This could potentially be another frustrating “It depends” question, but in most cases, the answer here is “Yes, you probably should”. Using H.264/AVC together with at least one newer codec ensures you have video that is backward compatible and available for everyone, while providing a higher quality viewing experience for those on newer devices with more advanced codec support. YouTube for example, encodes content in H.264/AVC, VP9 and AV1. Netflix uses H.264/AVC and AV1 for standard dynamic range content and H.265/HEVC for Dolby Vision. The right combination for your business will depend on your target viewing platforms, output quality, and business model. The step-by-step guide below will focus on AV1, but the same questions and steps could also be applied to other codecs.    

Step-by-step guide to adding AV1 encoding:

1. Determine how AV1 can help your business goals and viewer experience

Adding a new codec can be beneficial in many ways as discussed above, but it’s a big decision and should be made with deliberate intentions and goals in mind. What is it that you want to achieve by adding a new codec to your encoding stack? 

• Quality improvement on mobile devices or areas with limited bandwidth? 
• Long term reduction of storage, egress, and CDN delivery costs? 
• Making premium 4K and HDR tiers available to a broader audience? 
• Improving QoE and reducing buffering across your platform? 

AV1 can help in all of those areas, and you can fine tune your implementation to achieve your specific objectives.

2. Project Total Cost of Ownership (TCO), savings and revenue potential

AV1 is more complex to encode and can cost a little more in terms of computing power and encoding time. But the significant bandwidth savings it provides over H.264/AVC mean that in most cases, this initial cost will be more than offset by the longer term egress and CDN savings, giving you a lower TCO over the lifetime of each video. The actual break-even point will depend on your own contracts and delivery costs, so we created this calculator to help forecast where you might begin to see overall savings by adding AV1.

Besides lowering TCO for your video, AV1 can actually help generate revenue and retain subscribers. Its lower bitrate requirements for HD and 4K content mean that it can grow your total addressable market for higher quality upsell tiers. It can also help reduce subscriber churn by lowering rebuffering rates and providing a generally better quality of experience for your viewers. 

3. Utilize cloud-native encoding and analytics for fast, risk-free evaluation

Some on-premises encoders may require a pricey license upgrade to support AV1 encoding and in other cases, you’ll need to purchase a new appliance or hardware. You can avoid those high up-front costs that may stifle innovation by using a cloud encoding service that lets you pay on a usage basis. Even better, with Bitmovin’s cloud-native encoding, you can try it for free with our 30-day trial. With Bitmovin, you can also use your cloud credits on AWS, Azure or Google Cloud or use some of your contracted commitment to help fund your initial evaluation.

One of the knocks on AV1 has been that there is less playback support compared to older codecs, but there’s been steady improvement and some big announcements over the last year, so momentum is building. Our AV1 playback support guide details which SmartTVs, streaming sticks and consoles already support AV1, but you can actually go beyond rough projections and use Bitmovin’s Video Analytics to see how much of your own audience could already be taking advantage of AV1. This is also available to you in our free 30-day trial and is easy to add to most players, so you should definitely take full advantage of that as part of your evaluation. The numbers may surprise you!

4. Accelerate proof-of-concepts with content-aware, Per-Title Encoding

AV1 is more complex to configure than previous generations of codecs and many technicians and engineers have spent years developing expertise with H.264, so there may be some reluctance or lack of time to fully dive into the details of a new codec configuration. You can overcome that obstacle by taking advantage of a content-aware encoding service, like Bitmovin’s Per-Title Encoding. Each file is automatically analyzed for complexity and the ideal adaptive bit-rate encoding ladder is generated for each piece of content, with no advanced knowledge or configuration of AV1 required. This not only simplifies the process, it will save you on storage costs by eliminating unnecessary renditions from your adaptive bitrate ladder and ensure you’re making the most efficient use of data on the delivery and playback side. 

5. Deploy with most popular content for early ROI 

One thing to keep in mind for AV1 deployment is that you don’t need to take an all-or-nothing approach. Some of the earliest adopters like YouTube and Netflix initially only encoded their most popular content with AV1, since that is where the delivery costs were highest and they could maximize their returns on the initial encoding investment. 

In this study released by Netflix, analysts found that 21% of content streamed in one month in early 2022 benefited from the most recent improvements in codec efficiency (Per-Title HEVC and AV1). Without those improvements, they estimated that total Netflix traffic globally would have been around 24% higher. That’s a pretty solid proof point that using Per-Title AV1 with even a portion of your most popular content can make a big difference for overall data usage and cost. As of October 2023, Netflix reported that over 30% their catalog has now been encoded with AV1, with more being backfilled daily, and AV1 encoding is enabled for all new videos ingested to their platform. That shows they’ve now seen the value and potential benefits of AV1 make it worth adopting even beyond the most viewed content. 

Conclusion

In a crowded marketplace where viewers have many options, taking advantage of newer codecs can help streaming services stand out with a better quality of experience for their viewers, while saving themselves money in the background. Which codecs provide the most benefit will depend on each service provider’s goals, business model and target devices, but it’s clear that evolving beyond H.264 alone is necessary to provide best-in-class streaming experiences. In addition to better quality, adding newer codecs like AV1 can help relieve the financial burden of higher egress and CDN costs that come with increased popularity. Making an investment in upgrading codecs now will pay dividends for years to come. 

Related Material

For more information about video encoding in general, our comprehensive streaming technology guide goes into greater detail about codecs, compression, and available encoding products and services. 

Check out our AV1 hub and data sheet for more information on its development, benchmarking and playback support. 

This Bitmovin community post explains how to configure multi-codec playback on the web.

Our github repo has code examples for implementing multi-codec streaming.

The post Guide to Adopting AV1 Encoding appeared first on Bitmovin.

Everything from premium shows and motion pictures to user-generated content (UGC) is delivered via the internet today. Online video consumption has never been higher, with the average viewer spending a whopping 19 hours a week watching digital videos.

The move from analog to digital has fueled this trend, as well as advancements in data compression. Video encoding accomplishes both of these needs, making the distribution of streaming content both efficient for publishers and abundantly available to end users.

Whether it’s a Netflix series, an interactive fitness class, or a Zoom call with coworkers, streaming video is everywhere. But plenty goes on in the background to prepare digital content for multi-device delivery. Encoders play a pivotal role in shrinking down the data of today’s high-definition videos without sacrificing quality.

In this guide, we dive into what video encoding is and how it works. We also explore the differences between hardware vs. software encoding, lossy vs. lossless compression, and encoding vs. transcoding (two terms that are often used interchangeably).

Use our interactive table of contents to navigate to the section of your liking or start reading for the whole picture.

What is video encoding?

Video encoding is the process of converting RAW video into a compressed digital format that can be stored, distributed, and decoded. This type of processing is what made it possible to compress video data for storage on DVDs and Blu-ray discs back in the day. And today, it powers online video delivery of every form. 

Encoding is essential for streaming. Without it, the video and audio data contained in a single episode of Ted Lasso would be far too bulky for efficient delivery across the internet. Encoding is also what makes today’s videos digital: It transforms analog signals into digital data that can be played back on common viewing devices. These devices — computers, tablets, smartphones, and connected TVs — have built-in decoders that then decompress the data for playback. 

That said, you’ve likely streamed before without using a standalone encoding appliance. How could this be? Well, for user-generated content (UGC) and video conferencing workflows, the encoder is similarly built into the mobile app or camera. Embedded encoding solutions like these work just fine for simple broadcasts where the priority is transporting the video from point A to point B. 

For more professional broadcasts, though, hardware encoders and computer software like Open Broadcaster Studio (OBS) come into play. Content distributors use these professional encoders to fine-tune their settings, specify which codecs they’d like to use, and take advantage of additional features like video mixing and watermarking. 

Encoding always occurs early in the streaming workflow — sometimes as the content is captured. When it comes to live streaming, broadcasters generally encode the stream for transmission via the Real-Time Messaging Protocol (RTMP), Secure Reliable Transport (SRT), or another ingest protocol. The content is then converted into another video format like HTTP Live Streaming (HLS) or Dynamic Adaptive Streaming over HTTP (DASH) using a video transcoding service like Bitmovin.

What is video compression?

Video compression is the piece of video encoding that enables publishers to fit more data into less space. By squeezing as much information as possible into a limited number of bits, compression makes digital videos manageable enough for online distribution and storage. 

Imagine you have a hot air balloon that needs to be transported to a different location by ground. The balloon would be too unwieldy to fit anywhere without deflating it. But by removing the air and folding the balloon up into a compact size, it would become significantly smaller and easier to handle.

Video compression works the same way. Specifically, it removes redundant information and unnecessary details for compact transmission. Just as the deflated hot balloon becomes easier to handle, a compressed video file is more suitable for storage, global transmission, and end-user delivery. 

Lossless vs. lossy compression

Video compression technology falls into two camps: lossless and lossy encoding. These opposing approaches work just as they sound: 

• Lossless compression describes an approach to encoding that shrinks down the file size while maintaining data integrity. With lossless compression, 100% of the original file returns when it’s decoded. ZIP files are a great example of this. They allow you to cram a variety of documents into a compressed format without discarding anything in the process. 
• Lossy compression, on the other hand, describes encoding technologies that remove any data deemed unnecessary by the compression algorithms at work. The goal of lossy compression is to throw out as much data as possible while maintaining video quality. This enables a much greater reduction in file size and is always used for streaming video. When you use image formats like JPEG, you’re also using lossy compression. That’s why JPEG image files are easier to share and download than RAW image files. 

How is video encoded?

Encoding works by using algorithms to find patterns, reduce redundancy and, in turn, eliminate unnecessary information. Video streaming workflows employ lossless compression to create an approximation of the original content that makes it easy to transmit the data across the internet while maintaining video quality for end users.

This involves three steps:

1. Identify patterns that can be leveraged for data reduction.
2. Drop all data that will go undetected by the human eye or ear.
3. Quickly compress all remaining data.

Accomplishing this requires the help of video and audio codecs. Literally ‘coder-decoder’ or ‘compressor-decompressor,’ codecs are the algorithms that make it all happen. They facilitate both the compression and the decompression that occurs once the video file reaches end-users.

Temporal compression

In order to capture the required visual data without going overboard on bitrate, video codecs break up the frames of a video into groupings of a single keyframe followed by several delta frames. The keyframe depicts the entire image of a video, whereas the subsequent delta frames only contain information that has changed. This is called temporal compression.

When a stagnant backdrop appears for the entirety of a talking-head news broadcast, for example, there’s no need to store all of that data in every single frame. Instead, the compression algorithm prunes down any visual data that goes unchanged and only records differences between frames. The stagnant backdrop offers a major opportunity to toss out unnecessary data, whereas the gestures and movements of the reporter standing before the backdrop are captured in the delta frames.

So, while the keyframe of a newscast will always show everything within the frame — including the reporter, their desk, the studio background, and any graphical elements — the delta frames only depict the newscaster’s moving lips and hand gestures. This is called temporal compression because it takes advantage of the fact that large portions of video images often stay similar for some time. In this way, video codecs can quickly remove excessive information rather than recreating the entire scene in each frame.

Spatial compression

Another strategy for tossing out superfluous information is spatial compression. This involves a process of compressing the keyframes themselves by eliminating duplicate pixels in the same image. Consider the previous example. If the newscaster is presenting in front of a solid green backdrop, it’s not necessary to encode all of the green pixels. Instead, the encoder would only transmit the differences between one group of pixels and the subsequent group. 

Spatial compression enables video encoders to effectively reduce the redundant information within each frame, resulting in smaller file sizes without significant loss in perceived video quality. It plays a vital role in modern video codecs like H.264 (AVC), H.265 (HEVC), AV1, and VP9, enabling efficient video encoding and transmission for various applications, including streaming, broadcasting, video conferencing, and storage.

Hardware vs. software encoding

As noted above, encoding can occur within a browser or mobile app, on an IP camera, using software, or via a stand-alone appliance. Dedicated software and hardware encoders make the encoding process more efficient — resulting in higher throughput, reduced processing time, and improved overall performance. They also offer more advanced configurations for precise control over the encoding parameters. This enables content creators and streaming providers to optimize the video quality, bitrate, resolution, and other aspects to meet their specific requirements and deliver the best possible viewing experience.

Dedicated hardware used to be the way to go for video encoding, but plenty of easy-to-use and cost-effective software options exist today. Popular software options for live streaming include vMix, Wirecast, and the free-to-use OBS Studio. On the hardware side of things, Videon, AJA, Matrox, Osprey, and countless other appliance vendors offer purpose-built solutions for professional live broadcasting. 

When it comes to VOD encoding, FFmpeg is a popular software option. Vendors like Harmonic and Telestream also offer hardware options. 

The decision between software and hardware encoding often comes down to existing resources, budget, and the need for any advanced configurations or features. Plenty of producers elect to use a mix of software and hardware encoding solutions for their unique workflows. The chart below highlights the pros and cons of each option.

Check out our comprehensive guide to The 20 Best Live Streaming Encoders for an in-depth comparison of the leading software and hardware encoders available.

What are the most common encoding formats?

Encoding formats is a vague term. That’s because a compressed video is ‘formatted’ in three ways:

1. By the video codec that acts upon it to condense the data. Examples of popular codecs include MPEG-2, H.264/AVC, H.265/HEVC, and AV1.
2. By the video container that packages it all up. Examples of popular video container formats include MP4, MOV, and MPEG-TS.
3. By the streaming protocol that facilitates delivery. Examples of popular protocols include HLS, RTMP, and DASH. 

Here’s a closer look at all three. 

What are video codecs?

A video codec is a software or hardware algorithm that compresses and decompresses digital video data. It determines how the video is encoded (compressed) and decoded (decompressed). Different video codecs employ various compression methods, such as removing information undetectable by the human eye, exploiting spatial and temporal redundancies, and applying transformation techniques. 

One popular codec you’re sure to know by name is MP3 (or MPEG-1 Audio Layer III for those who like their acronyms spelled out). As an audio codec, rather than a video codec, it plays a role in sound compression. I bring it up because it demonstrates how impactful codecs are on media consumption trends.

The MP3 codec revolutionized the music industry in the 1990s by making giant audio libraries portable for the first time. Music lovers swapped out stacks of CDs for hand-held MP3 players that stored the same amount of music without any noticeable change in audio quality. The MP3 codec did this by discarding all audio components beyond the limitations of human hearing for efficient transportation and storage.

Streaming requires the use of both audio and video codecs, which act upon the auditory and visual data independently. This is where video container formats enter the picture. 

What are video containers?

A video container format, also known as a multimedia container or file format, is a file structure that wraps audio codecs, video codecs, metadata, subtitles, and other multimedia components into a single package. The container format defines the structure and organization of the data within the file, including synchronization, timecodes, and metadata. It doesn’t directly impact the compression of the video itself, but rather provides a framework for storing and delivering the compressed video and associated audio and metadata.

MP4 is a common container format that most know by name due to its compatibility across devices, websites, and social media platforms. Chances are, you have several MP4 files saved to your computer that encapsulate audio and video codecs, preview images, and additional metadata. 

Here’s the contents of an MP4 container saved to my own computer:

As you can see, this specific MP4 file contains the H.264 video codec and the AAC audio codec, as well as metadata about the video duration. Protocols like HLS and DASH support the delivery of MP4 files for streaming, which brings me to our third category.

What are streaming protocols?

A video streaming protocol is a set of rules governing how video data is transmitted over a network. It defines the communication and data transfer protocols required for streaming video content to playback devices. 

Each time you watch an on-demand video or live stream, video streaming protocols are used to deliver the data from a server to your device. They handle tasks like data segmentation, error correction, buffering, and synchronization. Examples of video streaming protocols include HTTP-based protocols like HLS and DASH, as well as video contribution technologies like RTSP, RTMP, and SRT for live streaming.

Different protocols require different codecs, so you’ll want to consider your intended video delivery technology when encoding your video.

Codecs, containers, and protocols summarized: 

A video codec handles the compression and decompression of video data; a video container format organizes and packages the compressed video, audio, and metadata into a single file; and a video streaming protocol governs the transmission and delivery of video content over a network. Each component plays a distinct role in the overall process of encoding, packaging, and streaming video content.

But what if you need to package your video into multiple different formats to ensure broad compatibility and optimize distribution? No problem. Video publishers often process and repackage streaming content after it’s initially encoded using a transcoding service like Bitmovin. 

What is transcoding? 

Transcoding involves taking a compressed stream, decompressing and reprocessing the content, and then encoding it once more for delivery to end users. This step always occurs after video content has first been encoded, and sometimes doesn’t occur at all. Unlike encoding, it employs a digital-to-digital conversion process that’s more focused on altering the content than compressing it. 

A primary reason for transcoding live videos is to repackage RTMP-encoded streams for delivery via HTTP-based protocols like Apple’s HLS. This is vital because RTMP is no longer supported by end-user devices or players, making transcoding a critical step in the video delivery chain.

When it comes to VOD, transcoding is used to change mezzanine formats like XDCAM or ProRes into a streamable format. These formats are proprietary and not supported by end-user devices. By transcoding the mezzanine formats into streamable formats like MP4 or HLS, the content becomes broadly accessible.

Transcoding is also done to break videos up into multiple bitrate and resolution renditions for adaptive bitrate delivery. This ensures smooth playback in the highest quality possible across a wide range of devices. 

Here’s a look at the different processes that fall under the transcoding umbrella:

Standard transcoding 

In most video streaming workflows, the files must be converted into multiple versions to ensure smooth playback on any device. Broadcasters often elect to segment these streams for adaptive bitrate delivery using an HTTP-based protocol like HLS or DASH. That way, viewers can access the content across their mobile devices and smart TVs, without having to worry about whether the encoded video content is optimized for their screen size and internet speed.

Transrating

Transrating is a subcategory of transcoding that involves changing the bitrate to accommodate different connection speeds. With pure transcoding, the video content, format, and codec would remain unaltered. Only the bitrate would change. An example of this would be shrinking a 9Mbps stream down to 5Mbps.

Transsizing

Also under the transcoding umbrella, transsizing takes place when content distributors resize the video frame to accommodate different resolution requirements. For instance, taking a 4K stream and scaling it down to 1080p would be an example of transizing. This would also result in bitrate reductions, which is why overlap between all of these terms is common.

So why do we need the extra step of transcoding, when many of these processes could be accomplished during encoding? It comes down to efficiency and scalability. 

Most content distributors prefer to encode a master file (or mezzanine file) up front and then rework it as needed. The purpose of a mezzanine file is to provide a high-fidelity source for generating different versions of the content optimized for specific delivery platforms or devices. Mezzanine files serve as the basis for creating multiple versions with different bitrates, resolutions, or codecs for large-scale distribution. Transcoding also enables broadcasters to tackle the more computationally-intensive tasks in the cloud rather than doing all of their video processing on premises.

Transcoding can be done using an encoding solution like Bitmovin, a streaming platform like YouTube that has transcoding technology built into its infrastructure, or an on-premises streaming server. 

Video encoding vs. transcoding: What’s the difference?

The terms transcoding and encoding are often conflated. We’ve even used the two interchangeably here at Bitmovin. But the primary differences are as follows:

• Encoding is an analog-to-digital conversion; transcoding is a digital-to-digital conversion
• Encoding is necessary to stream video content; transcoding isn’t always required.
• Encoding occurs directly after video content is captured; transcoding doesn’t occur until later when the content has been transmitted to a streaming server or cloud-based service.

To reuse an analogy from a previous post, the difference between encoding and transcoding is similar to the way crude oil is processed in the gasoline supply chain:

1. First, crude oil is extracted from underground reservoirs. This crude oil can be thought of as the RAW video source itself.
2. Next, the crude oil is refined into gasoline for bulk transport via pipelines and barges. This is the encoding stage, where the video source is distilled to its essence for efficient transmission.
3. Finally, the gasoline is blended with ethanol and distributed to multiple destinations via tanker trucks. This represents the transcoding step, where the content is altered and packaged for end-user delivery.

Live transcoding vs. VOD transcoding

Another reason for the overlap in the usage of these two terms has to do with the nuance between live and video-on-demand (VOD) transcoding. 

VOD transcoding involves processing pre-existing video files — such as movies, TV shows, or recorded events — and transforming them into suitable formats and bitrates for efficient storage and delivery. This type of video processing can be performed at any time, independently of the actual playback, allowing for more extensive processing and optimization. 

Live transcoding, on the other hand, involves processing live data that’s in flight. It occurs immediately after the video is captured and moments before the video is viewed. Timing is everything in live streaming workflows, and all of the steps must take place in concert. For this reason, the nuance between ‘encoding’ and ‘transcoding’ is more pronounced when discussing live streaming workflows.

How is video transcoded?

Video transcoding is a multi-step process:

1. Decoding: The encoded stream is decoded using the video codec.
2. Processing: The uncompressed video file is edited and processed if needed. This can include resizing to different resolutions, changing the aspect ratio, adjusting the frame rate, or applying video effects.
3. Encoding: The altered video is re-encoded, potentially using different settings and/or codecs than those with which it was initially encoded.
4. Packaging: The transcoded stream is then packaged in a container format suitable for storage or delivery. The container format encapsulates the encoded video and audio streams along with necessary metadata, synchronization information, subtitles, and other supplemental data.

As mentioned above, this resource-intensive process requires significant computational power and time. For this reason, it’s important to consider your existing resources and where you want to tackle this part of the video streaming pipeline.

Where is video transcoding deployed?

Transcoders come in two flavors: on-premises transcoding servers or cloud-based services like Bitmovin. Here’s a look at the different deployment models we see developers using.

On-premises transcoding

Some video publishers choose to purchase transcoding servers or deploy transcoding software in their on-premises environments. This route puts the onus on them to set up and maintain equipment — yielding additional security but also requiring more legwork to architect the streaming workflow across multiple vendors. On-premises transcoding is only a viable option for organizations with enough resources to manage every aspect of their technology stack. 

When going with on-premises deployment, you’ll want to overprovision computing resources to prepare for any unpredictable spikes in viewership. Many companies that experienced surging demand during the pandemic switched to cloud-based transcoding solutions for this reason.

Lift-and-shift cloud transcoding

Some content distributors host their transcoding software in the cloud via a lift-and-shift model. This occurs when organizations rehost their streaming infrastructure in a public or private cloud platform without optimizing their applications for the new environment. Although lift-and-shift deployments ease the burden of equipment maintenance and improve scalability, they fail to fully deliver on the promise of the cloud. 

Cloud-native transcoding

“Cloud native” describes any applications that take full advantage of cloud computing. This can be delivered via software as a services (SaaS) offerings like the Bitmovin aVideo Encoder or it can be built in house.With cloud-native transcoding, developers benefit from the most flexible and scalable streaming infrastructure possible. This is the most cost- and energy-efficient of all three deployment models, making it a more sustainable approach to streaming.

According to Amazon Web Services (AWS):

“Cloud native is the software approach of building, deploying, and managing modern applications in cloud computing environments. Modern companies want to build highly scalable, flexible, and resilient applications that they can update quickly to meet customer demands. To do so, they use modern tools and techniques that inherently support application development on cloud infrastructure. These cloud-based technologies allow for quick and frequent changes to applications with no impact on service, giving companies an advantage.”

Source: Amazon Web Services (AWS)

Beyond offering lower capital expenditures for hardware, software, and operating costs, cloud-native transcoding makes it easy to scale. Video encoding expert Jan Ozer weighs in:

“Two types of companies should consider building their own encoding facilities. At the top end are companies like Netflix, YouTube, and others, for which the ability to encode at high quality, high capacity, or both delivers a clear, competitive advantage. These companies have and need to continue to innovate on the encoding front, and you can do that best if you control the entire pipeline.

At the other end are small companies with relatively straightforward needs, in which anyone with a little time on their hands can create a script for encoding and packaging files for distribution… Otherwise, for high-volume and/or complex needs, you’re almost always better off going with a commercial software program or cloud encoder.”

Source: Jan Ozer

It’s worth adding that with cloud-based deployment, you’ll never have to worry about peaks and valleys in usage or spinning up new servers. Instead, you can offload management duties and maintenance costs to your service provider while benefiting from the built-in redundancy and limitless flexibility of the cloud.

Bitmovin’s solution is based on Kubernetes and Docker to deliver on the cloud’s promise of infinite scalability and flexibility. It can be deployed in customer-owned accounts or as a managed SaaS solution using AWS, Azure, and/or Google Cloud Platform.

Considerations when architecting your encoding pipeline

When architecting the encoding pipeline of your digital video infrastructure, you’ll want to consider your requirements:

• Format and codec support: Verify that the products and services you select can support the input and output formats required.
• Output quality: Look for solutions that offer high-quality encoding with minimal loss or degradation. Consider factors such as bitrate control, support for advanced video codecs (e.g., H.264, HEVC), and the ability to handle various resolutions and frame rates.
• Scalability and performance: Confirm that your encoding and transcoding solution can efficiently handle the scale of your broadcasts. Cloud-based solutions offer an advantage here.
• Security and content protection: If you’re broadcasting sensitive or copyrighted content, you’ll want to look for digital rights management (DRM) support, watermarking, encryption, and the like.
• APIs and integration: Look for solutions that offer comprehensive API documentation, SDKs, and support for popular programming languages to ensure seamless integration with your existing workflows. 
• Per-title encoding: Make sure the encoding/transcoding solution you choose offers per-title capabilities — meaning that the encoding ladder is customized to the complexity of each video. With per-title encoding, you’re able to combine high-quality viewing experiences with efficient data usage by automatically analyzing and optimizing the adaptive bitrate ladder on a case-by-case basis.  After all, each video file is unique. So you risk wasting bandwidth or compromising quality without per-title encoding. 

Top encoding software and hardware

In our recent guide to the 20 Best Live Streaming Encoders, we compared the industry-leading software and hardware encoding solutions.

OBS, Wirecast, and vMix are the most popular options on the software front. These solutions range from free to upwards of a thousand dollars (for a lifetime license). 

A much broader selection of hardware encoders are available, with many being designed for specific use cases like enterprise collaboration or remote live video production. They can range from specialized component tools to out-of-the-box studio production kits. And while many hardware encoders help integrate all of your equipment into a full-functioning studio, you’ll want to ensure that the appliance you choose is compatible with your current gear. 

Software Encoders

1. OBS: Free, open-source encoding software for Windows, Mac, and Linux.
2. Wirecast: Highly customizable professional encoding software for Mac and Windows. 
3. VMix: Easy-to-use encoding software for Windows only. 

Hardware Encoders

4. Videon EdgeCaster EZ Encoder: Portable encoding appliance with cloud functionality, as well as both 4K and ultra-low-latency support. 
5. AJA HELO Plus: Compact live streaming encoder with support for SRT contribution.
6. Matrox Monarch HD: Rack-mountable encoding appliance that supports simultaneous recording.
7. Osprey Talon 4K: Purpose-built 4K encoding with broad protocol support. 
8. VCS NSCaster-X1: Encoding touchscreen tablet that acts as a complete live production system. 
9. Haivision Makito X and X4: Award-winning encoder that ensures reliable low-latency streaming with SRT video contribution.
10. TASCAM VS-R264: No-frills live streaming encoder designed for YouTube streaming.
11. Datavideo NVS-40: Multi-channel streaming encoder that can be used for end-user delivery via HLS.
12. Magwell Ultra Encode: Affordable and complete encoding appliance for video production, contribution, and monitoring.
13. Blackmagic ATEM Mini: Affordable and portable option for on-the-go encoding and multi-camera setups.
14. Black Box HDMI-over-IP H.264 Encoder: Straightforward H.264 encoder for delivering media over IP networks.
15. Orivision H.265 1080p HDMI Encoder: Low-cost option for remote video transmission with support for SRT, HLS, and more. 
16. Axis: M71 Video Encoder: IP-based video surveillance encoder with PTZ controls and built-in analytics.
17. LiveU Solo: Portable appliance built to deliver reliable 4K video over bonded 4G and 5G.
18. YoloLive: One-stop encoder, video switcher, recorder, and monitor that eliminates the need for additional equipment.
19. Pearl Nano: Live video production hardware designed for small-scale events.
20. Kiloview Encoders: Affordable H.264 encoder with support for SRT, HLS, and Onvif. 

Check out the full comparison here for a deep dive into each option. 

Top transcoding services

When it comes to transcoding, there are also a handful of open-source and free software options. These include:

1. FFmpeg: A free command tool for converting streaming video and audio.
2. HandBrake: Another transcoder originally designed for ripping DVDs.
3. VLC media player: A media player that supports video transcoding across various protocols.

For professional large-scale broadcasting, though, we’d recommend using a cloud-based streaming service. You’ll want to search for something that offers the security capabilities, APIs, per-title capabilities, and codec support required for large-scale video distribution.

Robust solutions like Bitmovin integrate powerful features at every workflow stage and can be used for both live and VOD streaming. This means you’re able to simplify your video infrastructure without compromising quality and efficiency.

We mention this because even the most simplistic streaming workflows include four distinct steps:

1. Video source or origin: Whether your source is a live camera or a cloud storage solution that houses your input files, the video origin is where it all begins.
2. Encoding pipeline: The encoding pipeline comprises all of your encoding and transcoding systems, hardware, and software. For complex live streaming workflows, this often includes a blend of software and hardware encoding technologies, as well as a cloud-based transcoding service like Bitmovin.
3. Content delivery network (CDN): These systems of geographically distributed servers are essential when delivering content to large global audiences, ensuring that the video is retrieved when your viewers push play.
4. HTML5 Player: Players refer to the media software application that allows viewers to watch streaming content online without additional plugins. These ensure video compatibility across various browsers, operating systems, and devices. They also provide standard playback control, captions, and dynamic switching between ABR renditions.

Security, analytics, and other needs often further complicate these pipelines. For that reason, you’ll want to approach the entire workflow holistically, and look for solutions that can be easily integrated with others or consolidate multiple steps into a single solution. 

We launched Streams in 2022 to help simplify streaming, which serves as a single platform for transcoding, CDN delivery, video playback, analytics, security, and more. This type of solution doesn’t compare apples-to-apples with standalone transcoders like FFmpeg — and that’s by design. As an all-in-one solution that’s built for the cloud, it eliminates the complexity of building your streaming infrastructure in-house. 

Best video codecs

Once you’ve landed on your encoding and transcoding solutions, you’ll want to consider which video codecs are best suited for your use case. Most video developers use a variety of codecs to ensure compatibility across devices while also benefiting from the improvements in compression efficiency and quality offered by next-generation technologies.

In our annual Video Developer Report, we consistently see six codecs playing a role in both live and VOD streaming workflows.

H.264/AVC

The majority of digital video takes the form of H.264/AVC (Advanced Video Coding) because it’s unparalleled in terms of device reach. As an efficient and well-supported compression technology, it lends especially well to low-latency workflows.

H.265/HEVC

HEVC encoding has been on the rise — a trend that we expect will continue since Google added support in Chrome late last year. It’s poised to become a common technology for browser-based video streaming as well as premium OTT content delivery to living room devices.

H.266/VVC

As one of the newest video codecs out there, VVC (Versatile Video Coding) usage has been lagging due to limited playback implementations from device makers. That’s all changing in 2023 (with LG’s 8K smart TVs already adding support), making VVC a good candidate for 8K and immersive 360° video content.

AV1

This open-source, royalty-free alternative to HEVC was created by the Alliance for Open Media, made up of Amazon, Netflix, Google, Microsoft, Cisco, and — of course — Bitmovin. It’s 30% more efficient than HEVC and VP9, which drastically cuts bandwidth and delivery costs. 

VP8

The VP8 codec is another open and royalty-free compression format that’s used primarily for WebRTC streaming. Many developers architecting WebRTC workflows are shifting their focus to VP8, so we anticipate a gradual decline in the coming years.

VP9

VP9 is a well-supported video codec that’s suitable for both low-latency streaming and 4K. More than 90% of Chrome-encoded WebRTC videos take the form of VP9 or its predecessor VP8, and top TV brands like Samsung, Sony, LG, and Roku also support it. 

We’ve been monitoring the adoption of these codecs for six years running by surveying video developers across the globe. Our CEO Stefan Lederer summarized this year’s findings as follows:

“H.264 remains the most popular codec among video developers, which is likely due to its more widespread browser and device support. Yet, when we look at the codecs developers plan to use in the short-term future, H.265/HEVC and AV1 are the two most popular codecs for live and VOD encoding. Personally, I am particularly excited to see the growing popularity of AV1, which has been boosted by more companies introducing support for it.”

– Stefan Lederer (CEO, Bitmovin)

Source: Video Developer Report

The good news is that Bitmovin’s transcoding service supports all of these codecs — giving you the flexibility to pick and choose based on your needs. We’re also committed to driving cutting-edge encoding technologies forward, so that our customers can adapt as the industry evolves.

Video quality vs. video resolution

High-resolution streams are often high-quality streams, but it’s not a guarantee. That’s because video quality is determined by several other factors such as frame rate, buffering, and pixelation. 

Here’s how the two compare:

Resolution

Resolution describes the number of pixels displayed on a screen. The more pixels there are, the more stunning and crisp the picture. Nearly 4,000 pixels go across the width of a screen displaying 4K video, whereas 1080 pixels fit horizontally across a screen displaying 1080p content. Think of it as the difference between a paint-by-number kit and the original masterpiece you’re trying to recreate. Using more pixels allows the encoded file to maintain more detail in each individual frame. 

Video quality

Video quality is a broader and less scientific measurement. It’s impacted by the resolution, for sure, as well as the frame rate, keyframe interval, audio quality, color accuracy, and more. Remember that Game of Thrones episode that was too dark for fans to make out what was happening? The episode’s cinematographer Fabian Wagner said that HBO’s compression technology was to blame for the pixelation and muddy colors. In this case, even 8K streams of the episode wouldn’t have yielded improvements in video quality.

TL;DR: When it comes down to it, video quality is subjective and can be influenced by a multitude of factors; whereas video resolution is a cut-and-dry measurement of the number of pixels displayed.

What is adaptive bitrate streaming?

The majority of video traffic today is delivered via adaptive bitrate streaming. If you’ve ever noticed a digital video change from fuzzy to sharp in a matter of seconds, you’re familiar with how it works.

Called ABR for short, adaptive bitrate streaming provides the best video quality and experience possible — no matter the connection, software, or device. It does so by enabling video streams to dynamically adapt to the screen and internet speed of each individual viewer.

Broadcasters distributing content via ABR use transcoding solutions like Bitmovin to create multiple renditions of each stream. These renditions fall on an encoding ladder, with high-bitrate, high-resolution streams at the top for viewers with high-tech setups, and low-quality, low-resolution encodings at the bottom for viewers with small screens and poor service.

The transcoder breaks each of these renditions into chunks that are approximately 4 seconds in length, which allows the player to dynamically shift between the different chunks depending on available resources.

The video player can then use whichever rendition is best suited for its display, processing power, and connectivity. Even better, if the viewer’s power and connectivity change mid-stream, the video automatically adjusts to another step on the ladder.

How to do multi-bitrate video encoding with Bitmovin

Encoding (or transcoding) your streams into a multi-bitrate ladder is required for ABR delivery. In some cases, this can be done as the RAW file is being encoded, but many broadcasters and video engineers opt to transcode the content into multiple bitrate options later in the video workflow. For this, you’ll need a live or VOD transcoding solution like Bitmovin.

Multi-bitrate video encoding, a.k.a. adaptive bitrate streaming, comes standard when processing videos with Bitmovin. Our platform can also easily be configured for per-title encoding using the Per-Title Template. 

Here’s a look at the steps involved:

1. Create an encoding in our API using version v1.53.0 or higher.
2. Add a stream or codec configuration by providing information about which video stream of your input file will be used. 
3. Add muxings by defining the desired output format (whether fragmented MP4, MPEG-TS, etc.), as well as the segment length, streams to be used, and outputs. 
4. Start the per-title encoding using the start Encoding API call. This can be configured for standard, two-pass, or three-pass encoding to achieve desired quality and bandwidth savings.

Get the full tutorial here.

Per-Title automatically prepares content for adaptive streaming in the most optimal way. Use our Per-Title Encoding Tool (Bitmovin login required) with your own content to get a good overview of what Per-Title can do for you.

Common video encoding challenges

Ensuring playback support for the codecs you’re using

As shown in Jan Ozer’s Codec Compatibility chart below, different codecs are compatible with different devices. If you’re only encoding content for playback on iOS devices and Smart TVs, for instance, AV1 wouldn’t be the right fit. This is why it’s useful to leverage a video infrastructure solution like Bitmovin that can provide insight into which codecs are compatible with your viewers’ devices using analytics.

Solving for limited user bandwidth

Today’s viewers expect the same video experience on their mobile devices as they do on their Ethernet-connected Smart TVs. Adaptive bitrate delivery is crucial, with technologies like per-title encoding yielding additional opportunities to reduce bandwidth while still exceeding your audience’s expectations. 

Justifying the costs of next-gen codecs

Emerging codecs are often computationally intensive (and expensive) to encode in real time. For this reason, you’ll want to make sure that a given video asset is worth this investment. While viral content will benefit from more advanced and efficient codecs, standard assets don’t always warrant this level of tech. A great way to weigh the benefits is by using analytics to determine what devices your audience is using, how many people are tuning in, and how your video performance compares to industry benchmarks. Another way, in respect to the AV1 codec is to use our break even calculator to estimate the number of views it takes to justify the cost of using AV1 in addition to H.264 or H.265.

Ensuring low latency for live broadcasts

Many live and interactive broadcasts like live sports, e-commerce, online learning, and esports require sub-ten second delivery. For these, you’ll want to use a video codec like H.264 that’s optimized for low-latency streaming. We’d also recommend finding a live transcoding solution that accepts emerging low-latency protocols like SRT and RIST.

Streaming video encoding glossary

Application-Specific Integrated Circuits (ASICs)

ASICS for video encoding are purpose-built video processing circuits designed to optimize performance, power consumption, and costs. Because they are manufactured for the specific application of video encoding and transcoding, ASICs can achieve high throughput and superior performance compared to general-purpose processors like CPUs and GPUs.

Bitrate

Bitrate refers to the amount of data transmitted in a given amount of time, measured in bits per second (bps). Higher bitrate streams include more data in the form of pixels, frames, and the like — resulting in higher quality. That said, they require more bandwidth for transmission and storage. On the other end of the spectrum, low-bitrate streams can be more easily viewed by users with poor internet connections, but quality suffers as a result of greater file compression.

Frame rate

Frame rate refers to the number of individual frames displayed in a video, measured in frames per second (fps). This determines the temporal smoothness and fluidity of motion in a video. Higher frame rate results in smoother motion, while lower frame rates introduce perceived choppiness or jerkiness. Common frame rates in video production and streaming include 24 fps (film standard), 30 fps (broadcast standard), and 60 fps (smooth motion and gaming). The appropriate frame rate selection depends on the content type, target platform, and desired viewing experience.

Graphic Processing Units (GPUs)

GPUs are hardware components designed to handle complex graphics computations and parallel processing tasks. Originally developed for accelerating graphics rendering in gaming and multimedia applications, GPUs have evolved into powerful processors capable of performing general-purpose computing tasks. They consist of thousands of cores that can execute multiple instructions simultaneously, making them highly efficient for parallelizable workloads. GPU-based video encoding solutions are both flexible and widely available, but they aren’t as specialized as application-specific integrated circuits (ASICs), defined above.

Group of pictures (GOP)

Also called the keyframe interval, a GOP is the distance between two keyframes measured by the total number of frames it contains. A shorter GOP means more frequent keyframes, which can enhance quality for fast-paced scenes but will result in a larger file size. A longer GOP means there are fewer keyframes in the encoding, which leads to encoding efficiency and decreased file size but could degrade video quality. Another consideration in determining the keyframe interval is the tendency for users to skip ahead or back to random points in a video. Shorter GOPs place more keyframes throughout the video to support viewing from these random access points.

Metadata

Metadata refers to descriptive or structural information that provides additional context and details about a piece of data. In the context of video streaming, metadata refers to the supplementary information about the content, such as title, description, duration, resolution, language, genre, release date, and more. Metadata can also include technical information like encoding settings, aspect ratio, and audio format. Metadata is essential for content organization, searchability, and providing an enhanced user experience. It’s typically embedded within the video file or delivered alongside the streaming data in a standardized format, allowing for easy retrieval and interpretation by video players, search engines, and content management systems.

MKV

Matroska Video, or MKV, is a popular open-source multimedia format that can store an unlimited amount of video, audio, picture, and subtitle tracks in a single file It’s also flexible in its support for codecs, accepting H.264, VP9, AV1, and more. As such, it’s a popular format for video distribution and digital archiving.

MPEG-4

Short for Moving Picture Experts Group-4, MPEG-4 is a group of video compression standards developed by the International Organization for Standardization (ISO). Several codecs implement this standard, including H.264/AVC and AAC.

Video file

A video file, also called a video container, is a self-contained unit that holds the compressed video and audio content, as well as supporting metadata. These come in different formats, including MP4, MOV, AVI, FLV, and more. Different file formats accept different codecs and play back on different devices.

WebM

WebM is an open, royalty-free multimedia container format developed to provide a high-quality alternative to proprietary video formats. Google, Mozilla, and other leading companies developed this format, which utilizes VP8 or VP9 video codecs.

Video encoding FAQs

Does video encoding affect quality?

Yes and no. While an encoded video will always be of lower quality than the RAW file, today’s codecs are advanced enough to reduce the amount of data included without degrading quality in a way that viewers would notice. 

Why do I need to encode a video?

No matter the industry or use case, encoding is a key step in the video delivery chain. It prepares the video for digital distribution and compresses the data into a more manageable size. It’s always taking place in the background for UGC applications like streaming to Twitch. And by using a professional hardware or software solution, broadcasters can tap into additional functionality. 

What’s the difference between encoding with GPUs vs ASICs?

Encoding with GPUs involves different hardware architectures and approaches than encoding with ASiCs. While GPUs are programmable and versatile across applications, ASICs are purpose-built for specific tasks. This makes them expensive and less accessible. Currently, only companies like Facebook and Google as using ASICs for encoding, which provides fast turnaround times and better encoding capacity.

Why do I need to transcode a video?

If you’re serious about scaling your broadcasts to reach viewers on any device and connection speed, transcoding is vital. Most encoders support contribution protocols like RTMP and RTSP. While these protocols work great for video transmission, they’re ill-suited for end-user delivery and aren’t even supported on most viewer devices. You’ll want to transcode your videos into adaptive bitrate HLS or DASH to ensure that your stream reaches viewers and delivers the best experience possible. 

How much does video encoding and transcoding cost?

True video encoding (meaning contribution encoding software and hardware) can range from cheap to expensive. And while software options like OBS are technically free, they have hidden costs associated with the computing equipment on which they’re deployed. Hardware encoders are more costly, but affordable options like the Blackmagic ATEM Mini Pro (a $295 piece of equipment) are also available. 

When it comes to transcoding, you’ll want to consider both storage and egress costs to calculate the total cost of ownership (TCO). When using Bitmovin’s all-in-one platform, pricing is based on a simple fee per minute of video in each output, with rates depending on the features used. These include the chosen resolution, codecs, and use of multi-pass encoding. Our pricing can be offered as both pay-as-you-go or as a custom plan. Learn more here.

How is video encoding different from video compression?

Video encoding always involves compression, but video compression doesn’t always involve encoding. Rather, video encoding is one version of compression that involves converting RAW video data into a condensed format using video and audio codecs. Video compression can also take place during transcoding or even by compressing an MP4 on your hard drive into a lossless ZIP file.

What are the best encoding settings for quality?

The right encoding settings will always depend on your specific use case. For instance, action-packed streams of sporting events require a shorter keyframe interval than videos depicting static scenes such as talk shows. Similarly, a low frame rate works fine for surveillance footage, but wouldn’t be the right fit for sporting events. 

You’ll want to strike a balance between quality and efficiency by avoiding unnecessarily high bitrates and overly complex codecs. Encoding expert Jan Ozer recommends the following:

“When you upload a file to an online video platform (OVP) or user-generated content (UGC) site, the mezzanine file you create will be transcoded into multiple ABR rungs. Given that video is a garbage-in/worse-garbage-out medium, the inclination is to encode at as high a data rate as possible. However, higher data rates increase upload time and the risk of upload failure.

It turns out that encoding a 1080p30 file above 10 Mbps delivers very little additional quality and that ProRes output may actually reduce quality as compared to a 100 Mbps H.264-encoded file.”

Source: Jan Ozer

What is the most effective way to encode/transcode a large volume of videos?

Cloud-based platforms like Bitmovin make it simple to transcode a large volume of videos with scalable, on-demand resources that can process multiple videos simultaneously. Streaming APIs and tools for integration also make it easy to automate your encoding workflow and streamline efficiencies.

What is per-title encoding?

Per-title encoding customizes the bitrate ladder of each encoded video based on complexity. This allows content distributors to find the sweet spot by dynamically identifying a bitrate that captures all of the information required to deliver a perfect viewing experience, without wasting bandwidth with unnecessary data. Rather than relying on predetermined bitrate ladders that aren’t the right fit for every type of content, per-title encoding ensures that resources are used efficiently by tailoring the encoding settings on a per-video basis.

What is H.264 video encoding?

Also referred to as Advanced Video Coding, H.264/AVC is a widely supported codec with significant penetration into streaming, cable broadcasting, and even Blu-ray disks. It plays on virtually any device and delivers quality video streams, but is gradually declining in usage due to more advanced alternatives like H.265/HEVC and AV1. We cover all of the popular encoding formats in more detail in our annual Video Developer Report.

Conclusion

Thanks to video encoding and transcoding technologies, today’s viewers have access to anywhere, anytime content delivered in a digital format. The ability to compress streaming data for efficient delivery  (without sacrificing quality) is key to staying competitive in the online video market. And for that, you need to architect your video workflow using the right technologies.

Once you’ve mastered everything that goes into encoding, the next step is finding a video processing platform for multi-device delivery and transcoding. At Bitmovin, we deliver video infrastructure to broadcasters building world-class video platforms. Our live and VOD platforms can ingest streams from any of the encoders detailed above and output HLS and DASH for delivery to streaming services and end users.  

Find out how you can achieve the highest quality of experience on the market and deliver unbreakable streams. Get started with a free trial today or reach out to our team of experts. 

As always, we’re here to help you navigate the complex world of streaming and simplify your workflow.

The post Video Encoding: The Big Streaming Technology Guide [2023] appeared first on Bitmovin.

SD: 700 kbps to 1.1 Mbps 

1080p HD: 5 Mbps 

4K UHD: 15 Mbps to 25 Mbps

Those are largely based on what’s needed to view SD/HD content encoded with AVC and 4K content encoded with VP9 or HEVC. Apple’s HLS spec has examples for AVC and HEVC that fall in line with those numbers and they are familiar ranges to anyone who has done any live streaming recently. So I’ll ask again, would you be interested in 1080p at 286 kbps or 4K at 684 kbps? It really is possible, right now, with Bitmovin Per-Title and 3-pass encoding together with the industry standard AV1 codec. Keep reading to see a few real examples and learn how to get started with your own AV1 encoding.

Leading the Way with AV1

Bitmovin has a long history with AV1, first making headlines in 2017 for debuting the world’s first AV1 live stream at that year’s NAB show, winning a Best of NAB award in the process. Later that fall, we partnered with Mozilla to create the first AV1 HTML5 playback demo in the Firefox browser. You can read more details about our history and progress with AV1 in this post, but recent highlights include adding smart presets and support for 3-pass and Per-Title encoding optimization. We’ve also developed 2 new patent-pending technologies around AV1 that we hope to share more details about soon!
While early adoption was limited by slower performance and minimal playback support, the data from our most recent Video Developer Report shows that AV1 may be at an inflection point, with 15% of respondents saying they were already using AV1 in 2021 and another 22% saying they planned on adding it to the mix in 2022.

Getting started with a new codec usually means beginning with some educated guesswork followed by a lot of experimentation that can end up being costly in terms of both time and resources. With AV1, we’re trying to help eliminate that phase altogether by pairing our Per-Title ABR ladder optimization together with 3-pass quality optimization in a SINGLE API CALL!!! Our Simple Encoding API combines Bitmovin’s market-leading quality together with DASH and HLS packaging, delivering the ideal bitrates to maximize QoE without any unnecessary overhead data. You choose the codec, provide the source file and destination and that’s it, no other configuration required. We also have a Postman Repository with some example templates, which is actually what I used to create the AV1 samples below. For workflows requiring additional configuration, AV1 and Per-Title are fully available and supported within our standard APIs and SDks.

4K Bitrates: YouTube VP9 vs Bitmovin AV1

YouTube’s use of AV1 and other codecs was covered thoroughly in an enlightening series of posts by Jan Ozer. The quick recap is that YouTube is using AV1 for the 4K versions of their popular videos, but VP9 for less-viewed 4K content, as I confirmed with my own uploads below. So while it would be nice and a more direct comparison if we had the results of YouTube’s AV1 transcoding, it’s still interesting to see the potential bitrate savings when jumping from VP9 to AV1. For the analysis, I used youtube-dl to download the 4K VP9 versions and MediaInfo to identify the bitrate.  

Shot on iPhone 13 – “marina”

First let’s look at a short 4K (3840 x 2160) clip I shot on iPhone 13, “marina”. It’s a mostly static shot, with a few high motion/high detail regions. Bitmovin’s Per-Title optimization set the top 4K ladder rung at 1.37 Mbps. At the low end, the 240 kbps rendition was 1440p, meaning even on the slowest connections, viewers would still see better than full HD.
[bitmovin_player id=’225410′]

Youtube’s 4K version – VP9 @ ~22 Mbit/s

Big Buck Bunny

Next, let’s look at a more familiar example, everyone’s favorite animated short,  Big Buck Bunny. On the top end, Bitmovin’s Per-Title AV1 delivers 4K under 2 Mbps. The bottom 240kbps rung is 1600 x 900, so even on over-shared wifi, your users would still be happily watching even better than 720p video. You can see the result here. 

Youtube’s 4K version – VP9 @ ~12 Mbit/s

Tears of Steel

Last but not least, we have Tears of Steel, the cinematic short from the Blender Foundation that combines real footage and human actors with computer generated animation and effects. Again we see the top quality 4K variant under 2 Mbps and  still have a full HD frame for the bottom 240 kbps version. Imagine being able to deliver at least HD video to all of your customers, regardless of connection speed. You can see the result here.

Youtube’s 4K version – VP9 @ ~12 Mbit/s

Get Started Now and Ride the AV1 Momentum

YouTube and Netflix have both been delivering AV1 to compatible Android devices since 2020 and last November, Netflix published a detailed post about using AV1 for 4K content, so while support for AV1 still lags a bit behind older codecs, there’s clearly already enough of a device footprint to make it worthwhile. Roku and Amazon’s newest “sticks” and the upcoming generation of Chromecasts are all <$100 options that give older TVs an AV1 upgrade and Intel, Qualcomm, and multiple SoC manufacturers have recently announced upcoming AV1 support, so the wave of capable devices will continue growing. While the actual numbers will vary based on your CDN and storage pricing, we’re seeing that for content with as few as 10K views, there is potential ROI in adding AV1 to your multi-codec strategy (AVC +HEVC). 

Updating to AV1 means being able to deliver at least HD, in many cases 4K, over network connections that had been limited to Standard Def or compromised, blocky versions of your content. Improving and maintaining consistently higher quality with AV1 means happier customers and a larger potential market for HD and UHD upsell tiers, not to mention lower CDN and storage costs. For most streaming services, adding AV1 to their multi-codec strategy will be inevitable, the only question is how soon will they start reaping the benefits?
You can start seeing the benefits of AV1 + Per-Title and 3-Pass encoding today with our free trial, so sign up now and give it a shot! !
AV1 is the next generation video codec and it’s on track to deliver a 30% improvement over VP9 & HEVC – Learn More

More AV1 Resources:

• Bitmovin Improves Support AV1 Video Encoding for VoD
• Bitmovin’s AV1 Encoding Gift Guide
• Cool New Video Tools: Five Encoding Advancements Coming in AV1

The post 4K Video at SD Bitrates with AV1 appeared first on Bitmovin.

AV1 Codec – Ready for Production

The Alliance for Open Media is a non-profit industry consortium that was formed in 2015 with the mission of developing open, royalty-free technologies for multimedia delivery. Its creation was largely a response to the complicated and potentially expensive licensing and patent protection around the dominant MPEG standard codecs (mainly HEVC) and would be especially important for creators of UHD content requiring more advanced compression techniques. The efforts of the AOM became a reality when the AV1 codec was finalized in 2018, but even before that, Bitmovin had added experimental support and debuted the world’s first AV1 live stream in April 2017. 
As is the case with most codecs, it would still be some time before the rest of the OTT ecosystem would fully develop to make AV1 a viable production option. Hardware manufacturers had to wait for the final spec to be locked before designing their decoders and only then could their production cycles begin. This meant that it would be at least another year if not longer before AV1 decoding would be available on mobile and OTT devices. The data from Bitmovin’s annual video developer report illustrates this effect, with AV1 adoption rates still in the single digits for the first 2 years, followed by momentum building more recently toward broader usage.

In addition, 22% of those same developers indicated that they plan to implement AV1 encoding into their workflows through 2022 – surpassing predictions for h.265/HEVC (20% planned adoption) for the first time.
This is further supported by the rapid adoption of the codec by AOMedia members and industry giants. YouTube’s recent use of AV1 encoding for their most popular videos has been well documented and is an implicit proof point that AV1’s lower bitrates enable savings on storage and CDN that outweigh its slightly higher encoding costs. Netflix began delivering AV1 video to their Android app in 2020 and recently announced they are expanding their AV1 usage to deliver 4K videos to compatible TVs and connected devices. Given all of that, you could say that AV1 encoding is having a moment. 

Give Yourself the Gift of AV1 Encoding with Bitmovin

As mentioned, Bitmovin first implemented AV1 encoding in 2017 and has been updating and optimizing along the way, but we recently focused our efforts and have made some substantial improvements. With the release of Encoder version 2.100.0, AV1 encodings are 5 times faster than before, show approximately 30% higher quality than HEVC and VP9, and are significantly more cost-effective.

In the image above we have a comparison of the bitrate ladders and VMAF objective quality scores for H.264/AVC, VP9, H.265/HEVC, and AV1 encoding (2-pass) of the same video. For the 1080p variants at the top end of the ladder, we see roughly equivalent VMAF scores, all in the very good, 90+ range, but the bitrates required to reach that quality level:

• 6.0 Mbps for H.264/AVC
• 4.1 Mbps for VP9
• 3.9 Mbps for H.265/HEVC
• 3.1 Mbps for AV1

show AV1 as the clear winner. Even though it needed significantly less data than the other codecs, AV1 also technically had the highest VMAF score of all at 92.7. On average across all renditions, the AV1 had a 32% increase in compression efficiency over HEVC, as measured with BD-Rate.  
With that in mind, how can you give a tangible gift of a high-quality and efficient AV1-supported device to your friends and family?  By grabbing one of these great devices:

AV1 Compatible Holiday Gifts

Google Pixel 6 Pro

Google announced their latest flagship Android phone lineup, the Pixel 6 and Pixel 6 Pro in October ‘21, which feature hardware-accelerated AV1 decoding powered by Google’s custom Tensor chip. The Pixel 6 Pro has a 3120 x 1440 OLED screen with a 120Hz Smooth Display and HDR support, providing best-in-class viewing capabilities for $899 and with the lower bitrates of AV1, more people than ever before can enjoy its full quality potential.

Amazon Fire TV Stick 4K Max

The most affordable item in our guide (on sale for $34.99 at time of writing), Amazon’s new Fire TV Stick 4K Max is an instant upgrade for older 4K TVs or even newer ones without AV1 support. Wi-fi 6 combined with lower bitrate AV1 streams make congestion issues and quality drops a thing of the past, ensuring you get the full 4K experience you expect. Plug and play with integrated support for your favorite platforms that are already encoding AV1 video…Netflix, YouTube, Amazon Prime Video, and definitely more to come.

Roku Ultra

The Roku Ultra also offers a quick and easy AV1 upgrade for your 4K TV at $99.99. With support for 4K, Dolby Vision, and Dolby Atmos, it provides the premium viewing experience with clever new features like a built-in lost remote finder, and once you find the remote, it has a headphone jack so you can pump up the volume without disturbing the neighbors. The free Roku Channel has been adding new content and gaining popularity recently, so this could be the best budget-friendly option.  

Samsung Neo QLED 8K QN900A TVs

At the top end of our guide and Samsung’s 2021 lineup are the Neo QLED 8K TVs. With 4 times the pixels of 4K, AI-based 8K upscaling, and support for all of the HDR formats, the QN900A models are among the best of the best on the market. AV1 encodings have never looked better than on its edge-to-edge Infinity Screen. Pick up the 65” for a cool $3,497.99 or treat yourself to the 85” for $6,497.99.

What’s next in AV1?

*UPDATE – Feb 8, 2022* With the release of Encoder version 2.109.0, the Bitmovin Simple Encoding API now supports AV1 encoding! Click below to try it out in Postman and check the documentation tab there for more details.

As production deployments of the AV1 codec become more widespread, Bitmovin is continuing to innovate, optimizing for both quality and performance, in addition to creating an automated testing suite with real-world scenarios and DRM-protected streams on multiple SmartTVs and connected devices. One specific development we can tease is that our Innovation team recently developed a new technique that is cutting the processing time of 3-pass encoding by an impressive 40%. If you’ve been curious about AV1, there’s never been a better time to check it out. It’s available for free in our trial with no obligation or credit card required, so sign up today and see for yourself!
AV1 is the next generation video codec and is on track to deliver a 30% improvement over VP9 & HEVC – Learn about Bitmovin and AV1

More AV1 Resources:

• • 4K Video at SD Bitrates with AV1
  • Bitmovin Improves Support AV1 Video Encoding for VoD

• • Cool New Video Tools: Five Encoding Advancements Coming in AV1

The post Bitmovin’s AV1 Encoding Gift Guide appeared first on Bitmovin.

This scientific evaluation puts AV1 to the test against industry standard codecs and shows that AV1 is able to outperform VP9 and even HEVC by up to 40%

Introduction

For practical Over-the-top (OTT) streaming applications it is mostly necessary to supply streams using multiple different video codec standards in order to stream to a wide range of devices and platforms.

The most commonly used video codes in this scenario are AVC, VP9 and HEVC. With the standardization of AV1, another modern video coding standard is joining in.

While AVC offers the best compatibility across devices and platforms, the newer standards such as HEVC and AV1 offer a much higher compression efficiency and thereby also a better user experience.

Another key difference between the codecs is that VP9 and AV1 were developed with the goal of being open source and freely available for anybody to implement and use without any royalties while AVC and HEVC require a royalty to be paid.

The multi-codec dataset presented here adopts the aforementioned standards in a practical OTT adaptive streaming scenario. The full dataset is freely available online (http://www.itec.aau.at/ftp/datasets/mmsys18/). For an in-depth description of the dataset, please reference (https://arxiv.org/abs/1803.06874).

The Dataset

Since the main focus is on an HTTP Adaptive Streaming (HAS) dataset, we adopted a set of bitrate/resolution pairs – referred to as the bitrate ladder – with a range from very low bitrates/resolutions of 100 kbits at 256×144 pixels up to 4k resolutions at 20 megabits.

This is a well-established approach for OTT streaming applications.

For the video sequences, we tried to cover a range of video sequences with different properties. For this, we calculated the spatial and temporal information so that the sequences contain different amounts of motion and texture.

For the adaptive streaming encoding, a size per segment of 2, as well as 4 seconds, was used.

For AV1 encoding a snapshot of the reference software was used (v0.1.0-7691-g84dc6e9). For the encoding, the cpu_used preset was set to 2.

The encoding for AVC, HEVC, and VP9 was performed utilizing ffmpeg and, thus, libx264, libx265, and libvpx-vp9 are used. For these codecs, encoding performed with the slow preset. For all codecs, a two-pass scheme is employed.

Encoding of the AV1 bitstreams according to these specifications was performed by the Institute of Information Technology at the Alpen-Adria Universität Klagenfurt. Encodings using the other codecs AVC, HEVC, and VP9 was carried out by Bitmovin using the Bitmovin Video Encoding cloud infrastructure.

All bitstreams were then collected and jointly evaluated.

The Evaluation

For evaluation, the reconstruction at lower resolutions was upscaled to the original resolution and the weighted PSNR relative to the original source was calculated ((6*Y+U+V)/8).

From these values we calculated the corresponding Bjøntegaard-Delta bit-rate (BD-rate) values.

When calculated over the entire bitrate ladder, we were able to observe an average bitrate reduction of AV1 compared to VP9 of 13% and compared to HEVC of 17%.

When we focus on the higher part of the bitrate ladder, the BD-rate reduction compared to VP9 increases to 22%-27% while compared to HEVC, the reduction increases to 30%-43%.

It should be noted that because of the fixed bitrate ladder, the overlap becomes rather small for the highest resolutions in some sequences and the results should therefore be interpreted with some caution.

This could definitely be improved by adapting the bitrate ladder to the properties of the different sequences.

Conclusion

The dataset is meant to offer a first HLS set environment for the emerging video coding standard AV1 and the other in OTT applications most frequently used codecs AVC, VP9 and HEVC.

The coding performance results for this test set indicate, that AV1 is able to outperform VP9 and even HEVC by up to 40%.

Please note that this evaluation primarily targets HAS services and has a very specific setup.

While it can give an indication on the coding performance of AV1, the results should be interpreted with caution.

Video technology guides and articles

• Back to Basics: Guide to the HTML5 Video Tag 
• What is a VoD Platform?A comprehensive guide to Video on Demand (VOD)
• Video Technology [2023]: Top 5 video technology trends
• HEVC vs VP9: Modern codecs comparison
• What is the AV1 Codec?
• Video Compression: Encoding Definition and Adaptive Bitrate
• What is adaptive bitrate streaming
• MP4 vs MKV: Battle of the Video Formats
• AVOD vs SVOD; the “fall” of SVOD and Rise of AVOD & TVOD (Video Tech Trends)
• MPEG-DASH (Dynamic Adaptive Streaming over HTTP)
• Container Formats: The 4 most common container formats and why they matter to you.
• Quality of Experience (QoE) in Video Technology [2023 Guide]

The post Best Video Codec: An Evaluation of AV1, AVC, HEVC and VP9 appeared first on Bitmovin.