Apple’s Worldwide Developer Conference is an annual event used to showcase new software and technologies in the Apple ecosystem. It was created with developers in mind, but sometimes new hardware and devices are announced and its keynote presentations have become must-see events for a much wider audience. There is also usually news about changes and additions to the HTTP Live Streaming (HLS) spec and associated video playback APIs. These HLS updates are often necessary to support new features and capabilities of the announced OS and hardware updates. This post will expand on Apple’s “What’s new in HTTP Live Streaming” document, with additional context for the latest developments that content creators, developers, and streaming services should be aware of.

The lastest HLS updates for 2024

The first draft of the HLS spec (draft-pantos-http-live-streaming) was posted in 2009, then superseded by RFC 8216 in 2017. There are usually draft updates published once or twice per year with significant updates and enhancements. A draft proposal was shared on June 7, that details proposed changes to the spec to be added later this year. Let’s look at some of the highlights below. 

Updated Interstitial attributes

In May 2021, Apple introduced HLS Interstitials to make it easier to create and deliver interstitial content like branding bumpers and mid-roll ads. Now, new attributes have been introduced for Interstitial EXT-X-DATERANGE tags, aimed at enhancing viewer experience and operational flexibility. 

1. X-CONTENT-MAY-VARY: This attribute provides a hint regarding coordinated playback across multiple players. It can be set to “YES” or “NO”, indicating whether all players receive the same interstitial content or not. If X-CONTENT-MAY-VARY is missing, it will be considered to have a value of “YES”.

2. X-TIMELINE-OCCUPIES: Determines if the interstitial should appear as a single point “POINT” or a range “RANGE” on the playback timeline. If X-TIMELINE-OCCUPIES is missing, it will be considered to have a value of “POINT”. “RANGE” is expected to be used for ads in live content.

3. X-TIMELINE-STYLE: Specifies the presentation style of the interstitial—either as a “HIGHLIGHT” separate from the content or as “PRIMARY”, integrated with the main media. If X-TIMELINE-STYLE is missing, it is considered to have a value of “HIGHLIGHT”. The “PRIMARY” value is expected to be used for content like ratings bumpers and post-roll dub cards. 

More detail is available in the WWDC Session “Enhance ad experiences with HLS interstitials“.

Signal enhancements for High Dynamic Range (HDR) and timed metadata

HDR10+

Previously, the specification had not defined how to signal HDR10+ content in a multi-variant HLS playlist. Now you can use the SUPPLEMENTAL-CODECS attribute with the appropriate format, followed by a slash and then the brand (‘cdm4’ for HDR10+). The example Apple provided shows the expected syntax: SUPPLEMENTAL-CODECS=”hvc1.2.20000000.L123.B0/cdm4″. For a long time, HDR10+ was only supported on Samsung and some Panasonic TVs, but in recent years it has been added by other TV brands and dedicated streaming devices like Apple TV 4K and a few Roku models.

Dolby Vision with AV1

Dolby Vision has been the more popular and widespread dynamic HDR format (compared to HDR10+) and now with Apple adding AV1 decoders in their latest generation of processors, they’ve defined how to signal that content within HLS playlists. They are using Dolby Vision Profile 10, which is Dolby’s 10-bit AV1 aware profile. HLS will now support 3 different Dolby Vision profiles: 10, 10.1 and 10.4. Profile 10 is “true” Dolby Vision, 10.1 is their backward compatible version of HDR10 and 10.4 their backward compatible version of Hybrid Log Gamma (HLG). For profiles 10.1 and 10.4, you need to use a SUPPLEMENTAL-CODECS brand attribute and the correct VIDEO-RANGE. For these, 10.1 should use ‘db1p’ and PQ, and 10.4 should use ‘db4h’ and HLG. The full example codec string they provided is: CODECS=”av01.0.13M.10.0.112″,SUPPLEMENTAL-CODECS=”dav1.10.09/db4h”,VIDEO-RANGE=HLG.

If you’re interested in Apple’s overall AV1 Support, you can find more details in this blog post.

Enhanced timed metadata support

HLS now supports multiple concurrent metadata tracks within Fragmented MP4 files, enabling richer media experiences with timed metadata (‘mebx’) tracks. This will enable new opportunities for integrating interactive elements and dynamic content within HLS streams. .

Metrics and logging advancements

The introduction of the AVMetrics API to AVFoundation will allow developers to monitor performance and playback events. This opt-in interface lets you select which subsets of events to monitor and provides detailed insights into media playback, allowing you to optimize streaming experiences further.

More details are available in the AVFoundation documentation and the WWDC 2024 session “Discover media performance metrics in AVFoundation”.

Common Media Client Data (CMCD) standard integration

HLS now supports the CMCD standard, enhancing Quality of Service (QoS) monitoring and delivery optimization through player and CDN interactions. AVPlayer only implemented the preferred mode of transmitting data via HTTP request headers. They have not included support for all of the defined keys and for now is only supported in iOS and tvOS v18 and above. There was no mention of support in Safari. 

Bitmovin and Akamai debuted our joint CMCD solution at NAB 2023. You can learn more in our blog post or check out our demo.

FairPlay content decryption key management

As part of ongoing improvements, HLS is deprecating AVAssetResourceLoader for key loading in favor of AVContentKeySession. AVContentKeySession was first introduced at WWDC 2018 and until now, Apple had been supporting both methods of key loading for content protection in parallel. Using AVContentKeySession promises more flexibility and reliability in content key management, aligning with evolving security and operational requirements. This move means any existing use of AVAssetResourceLoader must be transitioned to AVContentKeySession. 

Conclusion

The recent HLS updates show Apple’s commitment to enhancing media streaming capabilities across diverse platforms and scenarios. For developers and content providers, staying updated with these advancements not only ensures compliance with the latest standards but also unlocks new opportunities to deliver compelling streaming experiences to audiences worldwide. 

If you’re interested in being notified about all of the latest HLS updates or you want to request features or provide feedback, you can subscribe to the IETF hls-interest group.

The post WWDC 2024 HLS Updates for Video Developers appeared first on Bitmovin.

This post was originally published in Nov. 2023. It was updated on Feb 19, 2024, with information about current applications that are available on the Apple Vision Pro.

Apple has always been great at creating hype around its new devices or features and getting millions of people to talk about and use them. This was especially true in September at this past IBC, when many conversations we had at our booth and on other stands included mention of Apple’s big announcement on supporting AV1 for the next version of their iOS devices. With their latest push into the AR/VR space, it’s no different. In early 2024, Apple is expected to release the Apple Vision Pro, which is set to be the next big device streaming services are getting ready to support. However, what does it look like to stream video on these devices, and how will it ensure a high-quality experience for viewers? 

In this blog, we will look at what Apple Vision Pro and visionOS are bringing to the table, the features and industries that can take advantage of it, and how Bitmovin’s Player is focused on supporting the new device.

What is Apple Vision Pro and visionOS?

Apple Vision Pro is an augmented reality (AR) headset announced in June 2023. It features a new three-dimensional interface that users can control with their eyes, hands, and voice. The headset is also powered by their latest operating system, visionOS, explicitly designed for spatial computing. Creating applications for visionOS is pretty straightforward, as Apple has streamlined the development process for all Apple platforms through Xcode. 

Xcode is Apple’s integrated developer environment application for those unfamiliar, enabling you to develop, test, and distribute apps for all Apple platforms. Because of this, our iOS Player team found it to be a very straightforward process to add the new platform to existing projects already deployed on iOS and tvOS. 

Does Apple Vision Pro and visionOS support video streaming?

The short answer is yes. The Apple Vision Pro was made with video as the focus and outputs superior picture and sound quality from its 23 million pixel resolution and dual-driver audio pods that enable an immersive experience for users. It supports up to 4K video, allowing streaming platforms to engage viewers with the highest quality live and on-demand content across browsers and dedicated applications.

Are there active applications streaming on Apple Vision Pro?

As the Apple Vision Pro just launched in the US (scheduled for Europe and other areas later in the year), there is currently a growing number of apps that are available or working to go live on the device. With its immersive augmented reality, every platform streaming video has the ability to provide its users with a way to “step into the screen” for a truly unique viewing experience. Some of the active applications that are offering a great viewer experience on the Apple Vision Pro are:

• NBA (check out a video of their app) and MLB and other sports leagues and broadcasters are bringing viewers onto the field with live games, replays, and stats.
• OTT platforms like Paramount+, Peacock, and others are giving their viewers a way to engage with content in any environment
• Social Media platforms like TikTok have users scrolling their feeds while using other apps at the same time

Which industries and use cases work on the Apple Vision Pro?

Depending on the industry and use case, Apple Vision Pro will be a great way for viewers to stream content. Some of these industries include: 

• Sports
  • Viewers will be able to watch their favorite sporting teams like never before with mixed reality capabilities.
    
• Entertainment and OTT platforms 
  • With 4K video and surround sound capabilities, streaming TV and movies will be a more immersive and personalized experience.

• Esports and gaming
  • Users will be able to follow the latest gaming events closer and stream their favorite gamers.

• Online events
  • With virtual conferences, concerts, and more, users will be able to engage and interact more with content they are interested in.

• Health and Fitness
  • Apple Vision Pro will be used to provide remote workouts, physical therapy, and rehabilitation sessions, making it easier for people to access care from the comfort of their own homes.

• Edtech and eLearning
  • Video learning will be more interactive and immersive, making learning and retaining information more accessible for students.

Does Bitmovin support video playback on Apple Vision Pro?

Yes, we do! As Apple Vision Pro will give users a new viewing experience, it’s vital for streaming platforms to integrate and keep a consistent video quality across all devices and browsers. With Bitmovin’s support for visionOS enabling video playback on Apple Vision Pro, integration is possible today. This makes Bitmovin’s Player the only video player on the market that officially supports visionOS, providing dev teams with a unified development experience across all Apple platforms with our iOS, tvOS, and visionOS SDK.

 Some of the features that are available with our iOS, tvOS, and visionOS SDK. See the whole feature by downloading our one sheet.

To further our development, we recently attended an Apple Vision Pro developer lab in Munich, where we got to test out the Bitmovin Player on physical Apple Vision Pro devices. It was essential to get this hands-on experience as we were able to see firsthand how our Player performed when streaming video up to 4K. Today, Playback capabilities on Vision Pro devices are similar to others in Apple’s ranges. However, Bitmovin will continue to support any additions to functionality or integrations that Apple may introduce to this new device type. You can see our latest SDK updates in our release updates.

The wrap-up

As the industry gets ready for the release of the Apple Vision Pro in 2024, it’s clear there are a lot of streaming platforms working to get to market and reach the new viewer base on the device. Ensuring playback is consistent and in the highest quality across every device will be key for streaming platforms in maintaining viewer retention and engagement. That is why Bitmovin continues to focus on supporting the broadest range of devices in the market and providing a robust integration and feature set to enable our customers to reach their viewers with the best quality of experience possible. 

If you want to know more about Bitmovin’s support for visionOS and playback on Apple Vision Pro, join the conversation in our Bitmovin Community, and if you want to test out our solution suite, sign up for our 30-day free trial.

The post Streaming Video on the Apple Vision Pro: Supporting visionOS with the Bitmovin Player appeared first on Bitmovin.