Event-based tracking

A good starting point is a concept that’s applicable across all variations of analytics, event-based tracking. However, GA is architected around collecting so-called “hits”. While a hit, as a generic event, is a good proxy to measure various behaviors, the GA version is far too general for online video analytics. In video, GA is able to simply measure frequencies via the first click (e.g. as a proxy for a play), but that’s not enough to measure the quality of experience (QoE) of a user – in this case, it tells you not whether the video successfully started after a user clicked play.
For video analytics, the play is only where the story begins, as video is a time-based medium. 
To get an accurate representation of video engagement it’s important for the metrics to reflect the medium. To do so, you need more than a frequency measurement – every metric that’s duration-based has to be normalized by total session/segment length. For example, a real use comparison is around video quality via bitrate expenditure, consider the following two measurable events:

1. A two-minute segment captured at 1 MBit quality
2. A 30-second segment captured at 5 Mbit quality

Google Analytics will not weigh the Mbits based on the time segments and will measure the average bitrate at 3Mbit. However, when you normalize the values according to time, the average bitrate is 1.8Mbit. If you’re looking at GA, your perception is that you have a pretty good overall quality video stream, when the reality is a completely different story. This is pretty standard for Google Analytics averages, it even applies for time spent on site.

The instrumentation challenge

Now that we’ve established the importance of tracking events correctly, we can address the next issue – types of trackable video events and their origin. Tracking video events pose an additional challenge that one shouldn’t underestimate, platform consistency. Using GA or any general Analytics tool will come with the challenge of identifying and unifying events across different platforms. Since there are thousands of different devices, applications, and players, you need to be able to expose different events to hook your metrics up to. These differences need to be accounted for at any time when instrumenting and maintaining a video analytics deployment. An industry best practice is to establish a collector approach that provides data consistency across all platforms (ex: iOS, Android, web, Roku, …).

Aggregate vs. the individual

At the next level of online video analytics, it’s absolutely critical that you have the ability to monitor individual events of specific users. This granularity enables you to derive insights and hypotheses into service improvements (without even considering error or piracy monitoring). The shortfall of GA is its inherent design towards data aggregation, instead of identifying stand-out events (like errors), GA will aggregate the data to provide a holistic view of your service.
To really improve aggregate metrics you need to be able to review the behavior of a player, session-level monitoring is essential to making smart decisions around your video service.
Bitmovin’s Video Analytics applies the session-level methodology to ensure that any video service has a clear understanding of their player’s performance, providing data around Startup Time, Video Bitrate Expenditure, Device Type, Location, Stream Type, and more!

Real-time data for real-time decisions

The final shortcoming of GA is its ability to deliver data in real-time, an especially critical feature when you’re monitoring the performance of a live stream. Depending on the trackable event, GA will deliver data up to 24 hrs after its occurrence, which isn’t at all helpful when you start hearing about platform streaming issues on your Twitter feed. GA is capable of delivering information such as impressions and clicks in near real-time, but it doesn’t compare to analytics that measures performance in real-time and provide in-depth alerts that enable quick and timely reactions to any potential issues.

What you need for online video analytics

To reiterate, Google Analytics is a great base layer of analytics, especially for web and marketing metrics, but it has its limitations outside of that, especially if you’re running an OTT platform or service. To ensure that you’re delivering a top-of-the-line QoE, I’d highly recommend launching an easy-to-implement set of video analytics that measure detailed event-based data sets, can be applied for nearly every consumer device, and does it in real-time. To find out how Bitmovin’s Online Video Analytics can supplant the shortcomings of GA check out our page here, or get in touch with one of our experts.

Video technology guides and articles

• Back to Basics: Guide to the HTML5 Video Tag 
• What is a VoD Platform?A comprehensive guide to Video on Demand (VOD)
• Video Technology [2022]: Top 5 video technology trends
• HEVC vs VP9: Modern codecs comparison
• What is the AV1 Codec?
• Video Compression: Encoding Definition and Adaptive Bitrate
• What is adaptive bitrate streaming
• MP4 vs MKV: Battle of the Video Formats
• AVOD vs SVOD; the “fall” of SVOD and Rise of AVOD & TVOD (Video Tech Trends)
• MPEG-DASH (Dynamic Adaptive Streaming over HTTP)
• Container Formats: The 4 most common container formats and why they matter to you.
• Quality of Experience (QoE) in Video Technology [2022 Guide]

The post Why Google Analytics are not suitable for Online Video Analytics appeared first on Bitmovin.

Actionable insights for reactive users

Across all walks of life and varieties of business, teams, or individuals, humans can be categorized into two types of people – proactive or reactive. I found that these categories are just as applicable for Analytics vendors through discussions with my product peers for Analytics-oriented tools. Much like their general worker counter-parts proactive analytics users actively dig into their data sets to find actionable insights and improvement potential to their workflows or performance. Whereas reactive users only react to specific triggers to investigate issues, such as customer tickets, reports, and/or inquiries from other sources. And in the case of Analytics tools, from specific alerts or notifications within a given analytics tool.
After a quick review of Bitmovin Analytics, I came to the realization that our dashboard was biased towards proactive users and thus a large group of users was missing out on the benefits of the insights we provide to their service. Each metric that we provide has its own dedicated screen with a variety of filters and breakdowns that would encourage proactive users to dig into the data that best suits their needs. However, we recognized that these insights should be applied at a much broader scale and that an alarm or notification should occur whenever an issue or action to improve a user’s video platform is an essential feature to our Analytics tool. In the end, regardless of your proactive or reactive nature, we want you to sleep well at night knowing that any issues will automatically be flagged so that you don’t need to stay up all night monitoring your platform to catching any issues before they hit Twitter.

Threshold-based alerts

After some additional considerations, I also realized that a single alert-type may not be the best solution for all, as the whole proactive vs reactive user-type is more of a spectrum than a binary category. To address the spectrum of video analytics users, I had our engineering team craft threshold-based alerts. Because, let’s be honest, even the most proactive users don’t mind having an alert or two to let them know that there’s an addressable issue, instead of compulsively seeking them out. So, what do threshold-based alerts even mean?

• Sample Size –  Following the proactive vs reactive spectrum “tend to the fires that matter” approach, we added a minimum sample size which depending on the importance of the issues (or a users’ need to proactively check performance). For those who want to be notified of every possible fire, you’ll likely want to set a subjectively low sample size. Contrarily, you don’t need to stress out knowing that an issue is only affecting a small fraction of your viewer-base
• Persistence Time – The “let the baby cry it out” approach, allows users to set a minimum persistence time so that an alert fires if the problem persists for a certain period…or not if you think it’s just crying for attention
• Recovery time – The “nursing a serious injury” approach, allows users to users to set a minimum recovery time threshold, defining the duration for which the affected metric value has to remain below a specific threshold for the incident to be considered resolved

All of these configurations will ensure that regardless of your position on the proactive vs reactive spectrum, you can rest easy knowing that there won’t be any fires to put out that you missed during the day or couldn’t find during your data deep-dive. All these configurations can be made either by using the Bitmovin Analytics dashboard or the with API.
Although this all sounds very straightforward, the true art of setting up your perfect notification cycle lies in refining the alerts to match your technical and business definitions so that you don’t miss critical issues on your platform, all while not overwhelming your technical teams. 

Visibility is key – a webhook based approach

Obviously, alerts have zero value if not one sees them, even if they are set with the perfect thresholds. Hence after consulting our customer base, we decided to take these alerts one step further and integrated them into those tools where most users can see them, in messaging platforms such as MS Teams or Slack. In addition to the custom alerts, we also offer a dedicated webhook for both market-leading chats.

The post Video Analytics Alerts: Impactful business notifications for reactive individuals appeared first on Bitmovin.

• Many devices require unique support for different streaming formats
• Each new set of devices comes with unique capabilities in terms of CPU power or memory. Hence your video platform might not be as well tuned to those capabilities: e.g. some devices may not have the power to decode a video in 4K

New regions

• Different global regions have different bandwidth conditions and bitrate expectations – in the case of COVID-19, streaming services in Europe were forced to reduce their bitrate expenditures to optimize for heavier work-from-home bandwidth usage.

Even streaming giants like Netflix, Disney+, and Youtube were forced to shift their delivery strategies on the fly, reducing the quality of their content from 4k and UHD, down to the seemingly archaic SD – resulting in widespread consumer disdain in a time where video streaming and consumption were at an all-time high.
In the end, as an OTT or streaming organization, you will have to accept that there will be errors, especially under uncertain conditions. However, it is possible to mitigate some of the worst problems with proper error reporting metrics, which can indicate a wide range of different issues, some affecting viewers’ playback or some simply being warnings thrown by the player. With these errors, it’s even more important to tackle them the right way to avoid losing subscribers and/or viewers.

Choose which fire to extinguish

So, how do you properly determine and monitor for critical errors in times when everything seems to be flashing red? The main video metric you should be reviewing constantly is error percentage – which works well as the main indicator to assess whether the numbers of errors have increased vs. the overall number of sessions. So the next question is: Where should I look when the error percentage increases?
Bitmovin’s Analytics dashboard helps identify the most critical and impactful error codes which may affect the majority of your user base (view below). It’s imperative to keep an eye on such metrics to ensure that the majority of your user base will be able to view your content with minimal issues.

As a best practice: you go from top to bottom to determine which errors affect the most viewers. Alternatively, you look at errors for a specific user set, e.g. premium users or others using custom data filters. Despite being an excellent indicator of high volume issues, catch-all error codes can be ambiguous – therefore a root cause analysis is often unavoidable. However, there are additional assessments that can be done to help determine the root cause and how to solve/prevent the error from reoccurring again in the future.

First assessment of Error Reporting: Check the timeline

To determine the root cause of an error, it’s imperative to begin by looking at errors on a timeline and correlating error spikes with events or specific streams. Next, you’ll want to examine the error occurrence broken out by specific platforms, browsers, CDNs, specific domains, or any other (custom) breakdown. In some cases, this early assessment will be all that you need to identify the root cause. Consider Disney+ at launch, the root cause was easily identified as a higher than anticipated demand paired with a flawed initial architecture (which was quickly resolved). However, in circumstances where the first assessment doesn’t provide a clear answer – you’ll need to dig deeper into specific sessions to get more context around the error.

Second assessment of Error Reporting: Check Error Detail to see Error Context

Bitmovin’s Analytics provides sample sessions for each of the errors selected in an Error Table that’s broken out by different columns. These Error tables are often surprisingly helpful to identify potential root causes as they’ll indicate at a quick glance if the error originates from a specific videoTitle, or browser, domain, or operating system.

Third (and final) assessment of Error Reporting: Additional Error Data

In addition, looking at an individual error session will provide more clarity, especially for in-stream errors, e.g. the available bandwidth for the user.  
Looking at specific error sessions provides context to error and can help answer questions like Was there already extensive buffering before the event? Did the user’s available bandwidth significantly decrease before the error? How many quality switches did occur before the error?
The combination of these factors will give you a clearer understanding of why, how, and where each error occurred. However, there is always a chance that the error will still be unclear, so what are the other factors that you should consider?

Errors can be complex, and some error codes/messages can potentially indicate many issues under one umbrella – especially in scenarios when error code and message display unknown. Therefore, it’s important to evaluate any additional error data that you may have on hand to speed up your root cause analysis. Bitmovin Analytics show additional error data that we draw from the stack traces of your app or website that will indicate where these unknown errors might originate from. This will ultimately help identify the different cases per error, taking the anonymity out of the Source Error and mapping it to an actual cause.

How to Identify the Root Causes of Errors

To summarize succinctly, identifying the root cause of an error is imperative to not only repairing your service but also to help your organization prevent these errors from coming up again. To identify the root cause you can three easy (but not so simple) steps:

1. Check the timing of the error – was it caused by an event? Content release? App update? Browser issues? CDN miscommunications?
2. The devil is in the details – where did the break happen in your app/service? Did it happen for everyone, or just for users in specific regions?
3. “Double-tap” – check the additional information that you might have missed on your first pass, errors can be tricky and hidden within multiple factors. Due diligence is key

Taking this top-down assessment approach will save you immensely on time, as you may not need to dig into the details for every error type. However, when the codes are ambiguous and can continue multiple factors, it’s important to dig deep and find a pattern. I hope this has been helpful to you! All-in-all there are five different metrics that matter to ensure that your streaming app and/or platform stay successful. Check out some of our other great content that identifies the other four:
[Blog Post] Why you should have a healthy obsession with startup time
[Blog Post] Encoding Excellence: Reducing Redundancy with Bitmovin’s Video Bitrate Heatmap

The post Metrics That Matter: Top-Down Error Reporting appeared first on Bitmovin.

Healthy obsessions for startup time in audio streaming

Much like its history in creation, audio streams came before video – as such audio streaming experts, much like Spotify founder, Daniel Ek, found himself (healthily) obsessed with startup time.
Before becoming the audio streaming giant that Spotify is today, Ek found his company competing against the traditional media market of the time, offline experiences, and recognized that he had to match that experience of listening to music offline. An integral part of that experience was the ability to double-click on a song (or press play) and the song would playback immediately without any perceived delay. Ek identified that the only way that music consumers would switch to an online stream would be if that experience would match offline playback.
Daniel found that there was one prominent key that helped Spotify achieve his goal of matching offline experiences: You guessed it, he obsessed about obtaining a startup time with 200 milliseconds startup delay.

“We never got it down to 200 milliseconds consistently. We got it down to about half a second but when you play then with things such as the throbber … and you actually, cognitively make it move even before there’s sound, the human brain perceives it to be instant even though technically, it wasn’t. There’s a lot of things that you could do, and I got really into the details thereof just creating an amazing experience that solved the end goal.”

Considering that much of this story played out in 2006, even though audio was a bit of spearhead in terms of streaming, it is amazing how this concept is still relevant for video in 2020. Especially looking at startup times today. According to Bitmovin’s Analytics, median video startup time value across all customers is 1.537 seconds, miles apart from Spotify’s goal for audio.

But why should we actually obsess about startup time? 

According to research from our 2019 Video Developer Report, as well as conversation with prospective customers, one of the biggest assumptions in the video streaming industry is that longer startup times lead to abandonment before the content starts playing. According to This was supported in Akamai’s 2012 Whitepaper, Maximizing Audience Engagement, where they found that “for every additional second of startup delay, an additional 5.8% of your viewership leaves”

Many indicators suggest that viewers have become more impatient in the 8 years since the Whitepaper, even though network conditions have improved tremendously, hence we’re still struggling with the same problem. This was displayed prominently at Demuxed 2019 by Ben Dodson from Snap Inc. He showed that if a “snap” took 2 seconds or more to load, the entire user base would have abandoned their individual sessions (see graph below).

It’s important to note, however, that Dodson mentioned (and correctly) that the specific context of Snapchat has to be taken into consideration:

• Very specific target audience (millennials and younger)
• Length of the video (short) 
• Consumption pattern
s – due to the length and duration of content (10-60 secs)

Although Snap’s data is very context-dependent, it still acts as compelling evidence that startup time is more important than ever for viewers, especially for younger audiences.

What affects startup time?

Startup time may be the metric that makes or breaks your business, but it ultimately comes down to a variety of factors that would result in slower startup times. This is especially relevant in video streaming as the workflow is often very complex as a result of elements coming from a variety of different systems and vendors (internal ones included). The industry-accepted metric for startup performance is video startup time or join time. The Bitmovin Analytics dashboard defines video startup time as the total delay between a viewer pushing “play” and the first frame.
Although startup time is the primary measurement that effects viewer bounce rates, it doesn’t tell the full story of a viewer’s experience. Indicators such as page and player load times are additional integral “startup” metrics that affect a user’s perception and willingness to stay on-page. 
Other factors that frequently affect your video’s startup time are Content Delivery Network (CDN) implementations, the bandwidth conditions, the bitrate ladder, bitrates, segment size, adaptation logic, and ad insertions. Despite how simple analyzing startup sounds, the reality is, it’s a complex measurement system that needs careful consideration and attention…one might even say that you need a healthy obsession to manage it appropriately.

Monetization Models – startup time friend or foe?

Advertising-based video on demand (AVOD)

The number one reason any organization might get into VOD services is to create content, and therefore monetize on that content. When it comes to startup time, the monetization faction that may result in the negative times would be Ad Insertions – streaming services that use ad insertions, better known as advertising-based video on demand (AVOD) services (ex: Youtube, Hulu, etc). AVOD services bring in guaranteed revenue, but their back-end implementations often cause major issues: Your video content might be well-tuned, encoded, and set up to perform seamlessly, but a media server might deliver an ad with additional delays that’ll instantly impact the first experience of every viewer by serving content (ads) with high buffer times. In a demo session, our analytics dashboard displayed these ad-based delays with alarming frequency – can you find the worst offender?

Subscription-based video on demand (SVOD)

On the other side of the monetization spectrum for VOD services are subscription-based models (Ex: Netflix, Amazon, Disney+, etc). Given that ad servers aren’t involved in these workflows, startup time is not directly affected by subscription models. However, SVOD services are often more at risk of content piracy from individuals who want to avoid paying at all costs. This is where Digital Rights Management (DRM) services come into play. DRM is almost essential to protect costly content licenses against piracy. As with Ad Insertions, DRM license servers will add seconds to startup time in the form of loading times – potentially influencing viewer experiences negatively. This can especially become an issue during live streams at a certain scale where load time is dramatically affected. The example below illustrates one of these delays – Y-axis: load time | X-axis: video timestamp

Final thoughts

Knowing what you know now, are you convinced and obsessed with startup time as much as we are yet? Given the severity of viewer bounce rate for every second of delay (5.8%/s) and how many factors go into properly managing your app’s startup time, I certainly hope so! So if you want to be as obsessive about startup time as Spotify, it is not only important to look at video startup time but to consider the overall startup experience and it’s contributing parts.
Bitmovin tracks all of the individual metrics mentioned in this post, as well as the combined metric of all parts mentioned before, otherwise known as Total Startup Time.
Did you like this post? Then check out some of our other great content:
Everything you need to know about DRM [Blog]
Bitmovin Analytics API [Datasheet]
Do you want to talk to a Bitmovin Analytics expert to learn more about any of our five key video metrics? Send us an email @ inside-sales@bitmovin.com and we’ll startup a chat!

The post Why you should have a healthy obsession with startup time appeared first on Bitmovin.

How Does it Work: Video Bitrate Heatmap Explained

The video bitrate heatmap (available exclusively via the Bitmovin Analytics dashboard) displays the encoding rendition consumption patterns of viewers for OTT streaming service providers, telcos, and publishers. Our dashboard even offers a heatmap for per-video titles, with additional filters for specific dimensions like device-type, browsers, and CDNs.

Example 1: Balanced Bitrate Heatmap

Heatmap Key:

Y – axis: video bitrate and resolution of an encoding profile for a specific video asset 
X – axis: shows the runtime of the video.
Y – axis2 (% bars): percentage of total view impressions per bitrate and resolution

How to identify redundant renditions within the video bitrate ladder

The example above indicates that the 1.21 Mbit/s rendition is rarely consumed by viewers (2%). Especially in comparison to the next highest rendition (2.4 Mbit/s at 19%). The recommended action would likely be to remove the rendition altogether.  We do, however, caution against removing renditions without testing the effects on your videostack first. Removing lower bitrate renditions can have implications on your startup-time performance. Lower bitrate renditions are typically initialized in a player’s ABR logic to enable fast startup times. This is further visualized in the heatmap above: renditions crucial for a video’s startup time are displayed in dark blue at the beginning of the video. In this case – the 800 kbit/s rendition is not only the most consumed format, but it’s also enables faster start times.

How to identify if the highest rendition you offer is optimized for the bandwidth conditions of your users

Given today’s bandwidth & resolution landscape (5G internet speeds and 4K video), video distributors expect that the highest rendition (4.8 Mbit/s – 1920×1060) is viewed the most.
Contrary to expectations, we’ve seen multiple cases where consumers view content at “average” renditions, with highest renditions seeing little-to-no usage. Example 3 above showcases this exact scenario where ~82% of consumers viewed content at maximum renditions of 795 kbit/s.
This is often linked to the bandwidth conditions under which your viewers are watching the video. Viewers might not have the bandwidth conditions available to stream at the highest quality video rendition. If your viewers’ bandwidth is limited, you should reduce both your resolution & your bitrate accordingly. In the case of example three, the video bitrate heatmap enabled the content provider to adapt the bitrate of their highest renditions, so all viewers are able to consume the video in the highest possible quality.
Obviously, this works the other way round as well. Full usage of higher renditions might indicate that a viewer’s bandwidth conditions allow higher renditions for better quality. In the end, you are not only saving costs by optimizing bitrates and reducing renditions based on your viewership, but you are also fulfilling your audience’s expectations to watch their favorite content in the best possible quality. 
Did you enjoy our post and want to learn more about quality metrics or video encoding? Then check out the following content:

• Encoding Excellence: Top Three (plus one) Methods to Objectively Measure Video Quality (Blog)
• Battle of the Video Format: MKV vs MP4 (Blog)
• Choosing the Best Per-Title Encoding Technology (E-Book)

The post Encoding Excellence: Reducing Redundancy with Bitmovin’s Video Bitrate Heatmap appeared first on Bitmovin.

Increase click-rates and make your content more attractive to your viewers

Today, the most common way of creating thumbnails and sprites for videos involves selecting frames based on fixed time segments: for example, creating sprites by selecting a frame every 10 seconds or creating a thumbnail by selecting a frame 30 seconds from the start of a video.
This often leads to thumbnails and sprites displaying misleading imagery, or in some cases even showing out-of-focus or black frames. Also, this often produces images that are unrepresentative of the content at hand. For example, a video stacked with images of human interaction might end up with a random building as a thumbnail image. This is an important issue because, by building an emotional connection with the viewers, especially thumbnails displaying humans tend to perform better.
Youtube recently underlined the importance of relevant thumbnails by publishing that 90% of their best performing videos have custom thumbnails.¹ Vevo reported a 12% average increase in views for the first 20 days after a thumbnail has been optimized – with one video, “Ghost” by singer Halsey, showing a whopping 4000% increase.²

The graph shows the average increase in views through thumbnail optimization across Vevo’s entire library. Source: Vevo – https://variety.com/2019/digital/news/vevo-video-thumbnails-youtube-optimization-120318690

Tests by Wistia and Vidyard showed that custom thumbnails can, on average, increase your click rate by 25%.³ For a site with around 5 million clicks per day, assuming $8 CPM⁴, we’re talking about $10k additional ad revenues daily just because of optimized thumbnails.
In short, videos with random thumbnails perform worse than videos with representative thumbnails. This leads to lower CTRs (click-through rates), and in consequence, to decreasing ad revenues for ad-supported video offerings and to higher churn rates for subscription-based services.
Nonetheless, manual thumbnail creation uses up lots of resources, especially when services offer videos at scale, for example through integrating user-generated content into their offering. But also the average publisher uploads around 40 videos daily, which is already a threshold at which manual thumbnail creation becomes quite resource intensive.
The answer? Train an algorithm to select the most suitable thumbnails and sprites.
Machine learning based thumbnail creation can increase the relevance of the thumbnails and sprites, without using any additional manual resources for this task.
This can happen in two ways – either the algorithm selects the most suitable sprites and thumbnails from a video based on a description text. Alternatively, the algorithm can select sprites and thumbnails by looking at the frequency of occurrence of certain objects and faces.

But applying an object detection algorithm to your video workflow is not limited to a thumbnail creation use-case. Information about detected objects and content can be transformed into metadata, which can be provided to advertisers, enabling more targeted advertising, warranting higher CPMs from advertisers.
By providing additional data points, ML learning based indexation of videos also has the potential to help content creators to quantitatively assess the performance of different types of content better. Plus, by applying tags about detected objects to videos, the method can save valuable resources when archiving footage and makes it much easier to find relevant content again after it has been archived.
Additional Readings:

• Per-Title Encoding Whitepaper
• Encoding Datasheet
• DRM Whitepaper

¹ https://creatoracademy.youtube.com/page/lesson/thumbnails
² https://variety.com/2019/digital/news/vevo-video-thumbnails-youtube-optimization-1203186909/
³ Average for Wistia (34%) and Vidyard (15%). Source: https://wistia.com/learn/marketing/optimizing-play-rates; https://www.vidyard.com/blog/video-roi-how-vidyard-increased-the-ctr-homepage-video-15
⁴ https://monetizepros.com/cpm-rate-guide/video/

The post Machine Learning-based Object Detection appeared first on Bitmovin.