Introduction
The main challenge that your adaptive streaming player faces is to keep the users engaged by choosing the ‘optimal’ representation (quality level) of the video. This always leads to a trade-off between providing the best possible quality, minimizing the startup time and avoiding buffering. If the user’s bandwidth conditions are not perfect, optimizing one of those three values will have a negative impact on one of the other two.
Unlike traditional video downloads, that need to be completed before playback, users expect streamed videos to start playback instantly, or within a fraction of a second. In fact research shows that if the startup time is longer than just 2 seconds, around 50% of users will switched to other content or services. With this in mind, it is fair to assume that any noticeable delay in startup represents a huge risk for content providers in potential audience loss and therefore revenue loss. As well as long startup times, re-buffering also has a negative impact to the user experience and therefore carries the same potential risks in terms of audience and revenue loss.
General concepts
Quality / bitrate
The quality of a stream is the result of several parameters such as the codec, the resolution, the frame-rate and the average video bitrate. Basically, Adaptive Streaming standards like MPEG-DASH and Apple HLS are designed to offer several representations (qualities) at the same time in order to fit every single viewing condition with only one stream. In typical use-cases, we consider that codec and frame-rates are constants from one representation to another and the difference in terms of quality depends mostly on bitrate and resolution variations. We consider the bitrate as the main quality factor, assuming that the resolution is closely connected to the representations bitrate. (Test your own stream with the Bitmovin Player)
Why does quality matter?
For a user, it is important to have the quality that fits best for his or her use case. Thanks to the technologies improvements in terms of screens, compression and delivery users have become accustomed to experiencing a very high quality. Most users have at least one Full HD screen in their household, be it a laptop, monitor or TV, and this sets the benchmark of expectation. Trying to display a lower quality than the expected benchmark puts your service at risk of being seen as inferior, or “less smooth to watch” than your competitors.
Startup delay
As already mentioned, the startup delay (the time separating the playback trigger from the moment when the video starts playing) has to be as small as possible to keep the users engaged.
The easiest solution to minimizing the startup delay is to minimize the download time of the very first segment. This can be optimized by requesting the lowest quality representation available. Although this strategy obviously affects the startup delay positively it impacts the first perceived quality. The goal is to find the best compromise between quality and startup delay. To perform such an optimization, the decision might be based on data of a bandwidth estimation based on the very first elements that the player downloads (manifest, player component etc…). Thanks to this compromise inside our Adaptive Bitrate (ABR), the player will find a balance and download the highest quality that allows the lowest possible startup delay.
Buffering
The buffer is a memory space where the player stores some video segments before playing them. It allows the player to anticipate a potential bandwidth drop of short or medium duration. An empty buffer can mean the player didn’t anticipate a long connection failure (30s – 1min duration for instance), or didn’t had enough time to fill the buffer.
The above graphic illustrates the functionality of a buffer, used by adaptive streaming players. If the requested bitrate is higher than the user’s bandwidth, the input rate will be lower than “1” (playback rate) and the buffer will decrease until it is empty. If the requested bitrate is lower than the user bandwidth, the input rate will be larger than “1”, and the buffer will fill up.
A good adaptation logic will minimize the buffering probability, by maximizing the buffer occupancy. To do this it needs to pick the most appropriate quality for the current bandwidth conditions, by taking the viewers expectations on quality (as discussed before) into account. So choosing the lowest quality every time, would minimize buffering but the video quality and therefore the viewer’s Quality of Experience (QoE) will be bad. On the other hand, a too high quality might lead to frequent buffering – and once again a trade-off.
The maintained buffer size is also an aspect which has to be considered by the ABR. With a long and well-filled buffer, the player would be able to face also long connection failures. But if the user drops off, or switches to a different video, buffering too much in advance would lead to unnecessary traffic.
How ABR Algorithms work
Evaluations show that the most efficient ABR algorithms based their estimations on the buffer occupancy as well as the instantaneous connection bandwidth. We can assume that the instantaneous user’s bandwidth is a representation of the immediate viewing conditions while the average viewing conditions lies more in the buffer occupancy. An algorithm which only takes the available bandwidth into account, would directly ‘map’ the bandwidth on the representation that it should require by using the function f(Bwh) described below.
In the following figure, we consider a use-case with 3 available bitrates (representations) called low, mid and high. “f” function represents the ABR behavior – the bitrate chosen depending on the bandwidth.
Such an algorithm provides a good and appropriate quality in stable conditions but is inefficient in varying bandwidth scenarios, due to the fact that the buffer’s state is not taken into account. Any connection failure would potentially generate a buffering situation. In regards to startup, such an algorithm would immediately require the highest bitrate and therefore increase the startup delay (significantly).
In the following figure, we still consider a 3 bitrates use-case (low, mid, high). The function “f()” represents the ABR behavior. This time, it chooses the bitrate according to the buffer occupancy.
An algorithm which would only consider the buffer occupancy would minimize buffering, but not necessarily maximize the quality. While the buffer is not fully filled, this kind of algorithm will be much too restrictive in regard to the bitrate. In terms of startup, this behavior would lead to a minimal startup delay, as the player’s buffer is completely void and the aim is to fill it as fast as possible. In summary, in order to provide the user the optimal quality as well as the optimal buffering rate and startup time based on his viewing conditions, the best results are obtained by taking into account the connection bandwidth and the buffer occupancy.
Even if the resulting algorithm is efficient in terms of bitrate adaptation, it does not necessarily deliver a high QoE to the viewer. A good ABR also takes the switching rate (how often the playback quality changes in a certain amount of time) into account. It is admitted that frequent quality switches are perceived worse than watching a lower, but constant quality. This is due to the high comparative sensibility of human eyes.
In cases, where the user’s bandwidth gently moves around the border that separates two representations (as illustrated in the graph below), the algorithm may switch between those two frequently.
A common solution is to create two switching thresholds around the boundaries and/or disable the quality switching, if the algorithm detects that the quality improvement would be not sufficient enough to compensate the induced switching rate increase. The below figure exemplifies the first approach.
Unstable Bandwidth Scenarios
Today, a single video clip or movie will most probably be watched on many heterogeneous devices from a 4K TV directly plugged to a fast and stable internet connection to a smartphone watched in a fast-moving car on a highway. In the latter case, users might face a less stable connection and the player has to preserve a certain buffer margin to compensate the instabilities.
For a “mobile device on a highway” use-case, the user’s connection’s bandwidth variation versus time will probably change frequently and randomly as shown in the graph below:
The above graph illustrates an unstable bandwidth scenario: during the first minute, the bandwidth is quite constant and just sufficient to display a medium quality stream. The second minute of playback is characterized by a small decrease in bandwidth before an important increase. During this period, a performant ABR could probably switch to a higher quality and take the advantage of this increment to fill the buffer to its maximal size. During the last minute presented in this graph, the bandwidth decrease and the user finally loses his connection for a few seconds. If the buffer is decently filled, the small gap of bandwidth will not generate any buffering and the user will probably not realize that is connection has been lost for a moment.
Custom Adaptation with Bitmovin’s Adaptive Streaming Player
Choosing the ‘Right’ Default Logic
By default, our player comes with two types of ABR algorithms and it is possible to choose one of them in the player configuration, via the parameter “logic” in the “adaptation” set of the configuration object.
(...)
adaptation: {
desktop: {
logic: 'v2'
}
},
The algorithm implemented by default (v2) is based on buffer as well as on bandwidth estimation, whereas v1 is only buffer based.
Apply Bitrate Boundaries
In some cases, the provided algorithms may not completely fit with your requirements. For instance, if you want to use the highest quality in your pay option only, and restrict the quality in your free offerings, you may need to artificially limit the available bitrate. This can easily be achieved by the according settings in the player configuration:
(...)
adaptation : {
desktop: {
bitrates: {
maxSelectableVideoBitrate: '10000000bps'
}
}
}
On the other hand, if you consider that providing a low quality stream is worse than having some buffering, you may define a minimal used bitrate:
(...)
adaptation : {
desktop: {
bitrates: {
minSelectableVideoBitrate: '3000000bps'
}
}
}
Having this setting present, the player would only choose quality levels, within the specified boundaries. Furthermore, a differentiation between viewers, using a mobile device and those using a desktop or notebook is possible:
(...)
adaptation : {
desktop: {
bitrates: {
minSelectableVideoBitrate: '3mbps',
maxSelectableVideoBitrate: '6mbps'
}
mobile: {
bitrates: {
minSelectableVideoBitrate: '0.5mbps',
maxSelectableVideoBitrate: '2.5mbps'
}
}
}
The same parameters are also available for audio:
(...)
adaptation : {
desktop: {
bitrates: {
minSelectableAudioBitrate: '128kbps',
maxSelectableAudioBitrate: '320kbps',
minSelectableVideoBitrate: '900kbps',
maxSelectableVideoBitrate: Infinity
}
},
It is also possible to make your player able to choose the quality according to the player size. Even if the available bandwidth is maximal, the player will not require a resolution, which exceeds the player’s size. Indeed, displaying Full HD or 4K content only make sense if the user’s screen is able to display such resolutions and if the player size is large enough.
(...)
adaptation : {
desktop: {
limitToPlayerSize: true
}
}
Download Cancelling
In case the player’s ABR algorithm requests a high quality segment while the bandwidth turns suddenly much worse, the logic should cancel the current segment download and start downloading a smaller one. This approach avoids a potential buffering and ensures an input rate equal or larger than the playback rate. Download cancelling is enabled by default in our player. However, disabling this functionality for dedicated use-cases is also possible:
(...)
adaptation : {
desktop: {
disableDownloadCancelling: true
}
}
Define a Startup Bitrate
As explained earlier, the startup phase is typically a tradeoff between allowing the final users to start at the best possible quality (according to the bandwidth estimation) and minimize as much as possible the time which separates the playback trigger from the beginning of the video. The optimal startup quality will be estimated via a bandwidth based approach and means to fill the buffer as quick as possible.
Although our default ABR logic is also optimized for a short and good quality startup phase, in some scenarios, delivering a dedicated quality right at startup might be necessary. This can also be achieved in our player config:
(...)
adaptation : {
desktop : {
startupBitrate : "2000kbps"
}
}
A Complete Customization
Thanks to our API based approach, it is also possible completely customize the ABR algorithm. This empowers our users to choose the optimal quality based on the current bandwidth conditions, the buffer fill state, external data, etc. or a mix of them. For instance, it is possible to design an algorithm based on the CPU usage. Even if the bandwidth is optimal, such an algorithm could switch to a lower bitrate in order to continue displaying smoothly even if the user’s CPU is already busy or not powerful enough.
Such customization is possible, by making use of the on onVideoAdaptation and onAudioAdaptation callback function. Each time the player is requesting a new segment, a dedicated event is triggered. The onVideoAdaptation callback passes an object containing the type of event, representation ID and the timestamp of the segment that has been requested as parameter:
{representationID:"1080_4800000",
type:"onVideoAdaptation",
timestamp:1480423373421}
How this can be used in order to completely customize the adaptation logic is shown in the following examples.
Example: Setup a Buffer Based Adaptation Logic
In the callback function, the first step is to order all the available qualities and bitrates into two different objects.
After that, we can identify 3 scenarios:
1 – Conditions are met to switch the quality up
The buffer contains more than 20s and the same representation has been chosen at least 5 other times.
2 – Conditions are met to switch the quality down
The buffer contains less than 10s and the same representation has been chosen at least 1 other time
3- Else
In every other cases, the ABR will ask for the same bitrate in order to keep a low switching rate as well as a sufficient buffer level.
The variable called “sameRepCount” increases each time the 3rd case will happens and is reset in both other cases
Some other examples are provided in our GitHub demo repository and for instance, it is possible to implement a rate based switching algorithm.
var currentQuality = null;
var sameRepCount = 0;
/* ... */
adaptation : {
desktop: {
onVideoAdaptation: function(param) {
var qualities = player.getAvailableVideoQualities();
currentQuality = currentQuality || qualities[0];
var nextBitrate = currentQuality.bitrate;
var bitrates = qualities.map(function(quality) {
return quality.bitrate;
});
// get next better quality if buffer is full (> 20 s) and played the same representation 5 times
if(player.getVideoBufferLength() > 20 && sameRepCount >= 5) {
nextBitrate = bitrates[Math.min(bitrates.indexOf(currentQuality.bitrate) + 1, bitrates.length - 1)];
sameRepCount = 0;
// get next lower quality if buffer is less than 10s and keep that for at least another segment
} else if(player.getVideoBufferLength() < 10 && sameRepCount >= 1) {
nextBitrate = bitrates[Math.max(bitrates.indexOf(currentQuality.bitrate) - 1 , 0)];
sameRepCount = 0;
// keep the same quality
} else {
sameRepCount++;
return currentQuality.id;
}
currentQuality = qualities.find(function(quality) {
return quality.bitrate === nextBitrate;
});
return currentQuality.id;
}
}
}
Conclusion
To cope with the challenge of choosing the most suitable representation, depending on the user’s viewing conditions, the player uses one or several ABR algorithms. The ABR is one of the key elements that make a player more efficient than another.
See the Bitmovin player’s adaptation logic in action in our demo page.
By combining buffer and bandwidth estimations, historical data, as well as information about the player size, Bitmovin’s Adaptive Streaming Player picks the most appropriate quality level. With its advanced custom adaptation features, it is also possible to fit the player behavior to every use-case and think innovative ways to optimize the quality. This includes user based approaches, which make a decision according to the viewing behavior of certain users. This concept, used by Amazon Prime (according to rumors), will be discussed in one of our next posts – stay tuned!
