Quality Metrics

A user is not interested in bit error rates, frame erasure rates, or packet loss. He or she
is interested in voice quality and image quality. We can define quality in terms of audible
and visible properties, which can be directly experienced and judged by the user.
Audio quality metrics are well established and already widely measured, but now we
are adding value by simultaneously multiplexing images, video, and application data Just as we judge audio in terms of fidelity; we can judge image and video streaming
in terms of color depth, frame rate, resolution, and contrast ratio; and application quality
in terms of application integrity. Figure 7.5 shows the main quality metrics of video.
The quality metrics for an image are the same—without the frame rate.
As we increase the complexity of the media multiplex, value increases (we hope), but
so does the cost of delivery. Hopefully, the value increases faster than the cost of delivery;
otherwise, the whole exercise is rather pointless from a business point of view. Figure
7.6 shows value increasing as delay (and delay variability) increases. We cover this
in much more detail in Chapter 11, in a discussion on network bandwidth quality.
Effectively, as we increase delay and delay variability, our cost of delivery reduces
and our margin per user should increase, provided the delay and delay variability
have not destroyed the value of the user’s content, which may or may not be timesensitive.
Remember, we are replacing a user experience (PSTN) where end-to-end
delay is typically 35 ms with no end-to-end delay variability. It is always dangerous to
assume a user will not notice a reduction in service quality.
Finally, there is the consistency metric (see Figure 7.7). In 1992 when GSM was introduced,
the voice quality from the codec was (a) not very good and (b) not very consistent.
This was due to a number of factors—codec design, marginal sensitivity in the
handset and base station, and insufficient network density (a marginal link budget). It
was not until 1995 that voice quality both improved and became consistent.
Interestingly, though anecdotally, we are often very forgiving of poor quality
provided the quality is consistent. If something is inconsistent, we remember the bad
bits. The same applies to video quality in 3G networks. It will take at least 5 years for
video quality to be acceptable both in terms of quality (frame rate, color depth, resolution)
and consistency. Consistency requires good control of radio bandwidth impairments
and irregularities (that is, slow and fast fading) and network bandwidth
impairments and irregularities (delay, delay variability, and packet loss).

Hopefully, we are beginning to make clear the intimate relationship between radio
and bandwidth quality and an acceptable (i.e., billable) user experience.

Summary
We highlighted the transition from constant-rate source coding to variable-rate source
coding, both for audio and video capture, and the related significance of MPEG standards
evolution, particularly in the longer-term object-based coding technique and
rendering engines. We showed how processing in the handset can create the illusion of
bandwidth and interactivity, and how preprocessing and post-processing can reduce
the amount of radio and network bandwidth needed (including RF power) for an
apparently wide-bandwidth application.
The argument was put forward that we should use tangible (easily evident to the
user) quality metrics to judge radio and network bandwidth performance and to provide
the mechanism for implementing quality-based rather than quantity-based
billing. MPEG-4 is generally regarded as a compression standard, but in reality, MPEG
also helps us define what network quality requirements are needed to preserve rich
media value.
Application layer software needs to evolve within this context. The job of application
layer software is to increase session persistency and session complexity, as shown
in Figure 7.8.
User value (and user billability) increases as session persistency increases. As session
persistency increases, session complexity generally should also increase. Asimple
data exchange is developed into a data plus voice and video exchange, or a simple
voice exchange is developed into a voice and data and video exchange, or a user-touser
exchange is developed into a multiuser-to-multiuser exchange. As session persistency
increases, consistency also has to increase. The longer the session, the more
obvious it becomes when radio or network bandwidth constraints cause discontinuities
in the duplex transfer of real-time rich media information.

Consistency is often underrated as a quality metric. Consistently poor quality is
often perceived as being better than inconsistently good quality. We adjust (and learn
to live with) consistent quality, even if the standard is relatively poor. Consistency is a
product of protocol performance. Don’t send “same again” differentially encoded
image and video streams over “send again” channels.
Additionally, software performance is a key element in our overall user quality metric
(user Q), which brings us to our next chapter. 185