IPTV rollouts are being announced, and there are a number of successful IPTV (News - Alert) deployments to date. However service providers are still experiencing issues and problems, and trying to develop an understanding of how to monitor IP video services. Successful deployment of IPTV services requires performance to be monitored at the customer premise (e.g. set top box), key aggregation points such as DSLAM or CMTS and at interconnect points between disparate networks. This requires the development of an objective set of standardized metrics, which is the short term goal of a number of standards committees such as ATIS, DSL Forum, IETF and ITU-T.

 

Today, IP Videoconferencing is widely used, and large enterprises are starting to use HD Telepresence, which can use up to 20 Mbits/s of bandwidth. Enterprise network managers are concerned about both the effect that significant levels of IP videoconferencing traffic will have on their networks and the attainable video performance levels, hence need performance monitoring tools for pre-deployment test and ongoing performance monitoring.

 

Adoption rates for IPTV and IP Videoconferencing depends to some extent on the ability of service provider and major enterprise to get services up and running, with minimal impact on existing networked services. This requires specialized management infrastructure, metrics and protocols. This article provides a brief update on some of the key industry activities related to IP Video performance management.

 

Some of the key video quality problems and some of their symptoms are outlined below – a more comprehensive range of quality issues are described on the IPTV Troubleshooter Web site.

 

Encoded video content can have a variety of impairments related to poor camera work, the use of low quality cameras, degradation during storage of recorded media and encoder quantization level. This range of problems will often lead to image blurring, motion blur and noise around the edges of objects.

 

Video content is often transferred between media or transcoded from one format to another. Errors can occur in this process, which can lead to image breakup and color problems within decoded macroblocks.

 

Transport stream problems, such as packet loss, result in the decoder having to attempt to reconstruct video frames with missing macroblocks. Depending on the strategy used by the decoder, this may result in frame freezes, image breakup, blockiness and similar artifacts.

 

Packet loss is one of the key impairments that affects IP video quality however a common misconception is that packet loss is the only important metric. A lost packet in an I frame can propagate through the other frames in a Group of Pictures which may cause 0.5–2 seconds of image degradation, conversely a lost packet in a B frame may be barely noticeable; It is fairly obvious that a simple packet loss metric will not distinguish between these cases whereas a MOS estimation algorithm typically will. 

 

Packet loss rates in IP networks are not uniform, losses often occur in either consecutive or sparse bursts. Therefore it is important to look at the time distribution of lost packets and the impact of high loss periods on the video decoding process, and simple loss metrics cannot do this.

 

There are four types of video quality assessment algorithm, and there is standardization work underway with regard to all of these in ATIS, ITU-T Study Groups 9 and 12 as well as the Video Quality Experts Group (VQEG). 

 

(i) Full Reference algorithms compare the input video stream from a system under test to the output and perform a pixel by pixel comparison. The most established and well known method is PSNR (Peak Signal to Noise Ratio) which is a literal measure of the distortion between input and output samples. Newer algorithms, such as those defined in ITU-T J.144 and Opticom’s (News - Alert) PEVQ algorithm, use perceptual models to incorporate psychovisual effects such as temporal and spatial masking. This approach is often used during lab testing of equipment or during pre-deployment testing.

 

(ii) Decoded Video Zero Reference (No Reference) algorithms operate on the video stream from the decoder and identify visual artifacts by pixel level analysis. While this is very computationally complex, it is able to detect artifacts from both the original source and transmission problems. The drawback of this approach is that in an operational environment it can only be deployed in a set top box, as access to the decoded video stream is required, adding significant complexity and therefore cost.

 

(iii) IP Transport Zero Reference (No Reference) algorithms operate on the bit or packet stream going into the decoder. They identify problems related to the IP or video transport protocol and infer the impact of these on video quality. This approach has the merit that it can be deployed at multiple locations along the video path, however will not detect problems in the original video content. Complexity is very low and hence this class of algorithm can be economically deployed in set top boxes, residential gateways, routers and test equipment.

 

A similar approach is sometimes used by video decoders to estimate quality based on the degree of frame loss concealment that was required when reconstructing a video frame.

 

(iv) Partial Reference algorithms use information extracted from the video source and compare this to the equivalent information extracted at the receiving end.

 

VQEG is starting testing of Zero Reference and Partial Reference algorithms based on video output under its RRNR activity, which should have results in late 2007. Testing of IP based (bitstream) Zero Reference algorithms will follow.

 

There is considerable work within the industry related to IP Video performance metrics. One of the most active groups is the ATIS IPTV Interoperability Forum (IIF), which is developing a comprehensive set of metrics for IPTV services (that are targeted for release in mid-2007).

 

The ATIS IPTV metrics include:-

 

 

ITU-T Study Group 9 is developing standards for video quality estimation based on the results of VQEG testing. SG9 is primarily focused on standards for broadcast services.

 

ITU-T Study Group 12 works extensively in the area of QoS/ QoE, and has been the main source of performance measurement standards and algorithms for telephony applications, however has recently developed some standards related to IP video performance. SG12 also cooperates with the IETF IPPM group in the area of IP metrics.

 

ITU-T IPTV Focus Group is tasked with defining requirements for IPTV from an ITU-T perspective, including requirements related to QoS/ QoE.

 

Performance analysis technology vendors such as Telchemy (News - Alert) are actively participating in these committees to ensure that commercial implementations (e.g., VQmon/SA-VM and TVQM) closely track these standards activities.

 

Management protocols are needed to support the communication of metrics from endpoints such as IPTV set top boxes and IP Videoconferencing units. Two key protocol development activities are underway:

 

A lightweight protocol that supports the exchange of metrics between IP video endpoints, which could be IPTV, IP Video streaming or IP Videoconferencing systems. This exchange can be used for quality feedback (to support sending end adaptation) and performance reporting. The RTCP XR Video Metrics draft is currently a Working Group draft within the IETF.

 

The DSL Forum is developing extensions to TR69 to support the extraction of data from IP set top boxes, under their WT-135 activity.   The WT-135 draft includes some IPTV performance metrics as well as a wide range of parameters related to configuration.

 

There are a number of standardization activities related to IP Video performance that are expected to reach a stable state during mid-late 2007 and early 2008. This means that some of the essential protocols and performance metrics will be standardized during this period. The industry is still learning about the performance characteristics of IP video systems and how they interact with legacy networks and other networked applications. Video systems are by nature more complex, involving specialized encoder/decoder technology, reliable (and unreliable) transport protocols and forward error correction systems, all of which are rapidly evolving. If the industry’s last decade of experience in speech quality analysis and management are a good indicator it is clear that video performance measurement techniques are still in their infancy, however the steps being taken by the standards community is establishing a sound framework from which to build.