Human Systems Integration Division @ NASA Ames

"A standard method for estimating subjective visual quality was developed at Ames. NASA missions rely extensively on visual communications, from space imagery and remote monitoring of space and aircraft to advanced video communications. In all of these applications, maximizing visual quality, while minimizing weight, power, bandwidth and cost, is a primary concern. Yet, there have been no standard methods for measuring visual quality. Such a method was developed at Ames that is based on the just noticeable difference (JND) unit. In psychophysics (the study of the relation between physical stimuli and sensory experience), two physical stimulus intensities that can just be discriminated are said to be one JND apart. Our method provides a way of measuring the perceptual distance between two stimuli in units of JND.

This method uses a pair-comparison procedure in which a pair of samples are presented to an observer on each trial. The observer selects the one perceived to be greater on the relevant scale (e.g. heavier, brighter, louder). While simple for an observer to perform, this procedure is inefficient because many comparisons will be uninformative if all samples are compared. In general, samples separated by about 1 JND are the most informative. Thus, a more efficient adaptive method for selection of sample pairs was developed that is based on Bayesian estimation of the sensory scale after each trial. It is called Efficient Adaptive Scale Estimation, or EASE (""to make less painful"").

The EASE method was used to measure impairment scales for digital video. Each video was derived from an original source (SRC) with the addition of an artifact produced by a particular form of video compression at a specific bit rate, called a hypothetical reference circuit (HRC). Different amounts of artifact were produced by linear combination of the source and compressed videos. On each pair-comparison trial, the observer selected which of the two sequences appeared to be more impaired. The scale is estimated from the pair comparison data using a maximum likelihood method. The value at the top of the scale (with all of the artifact present) is the total number of JNDs separating the original and the compressed video.

Impairment scales for 25 video sequences derived from five SRCs combined with each of five HRCs were measured. EASE was found to be a reliable method for measuring impairment scales and JNDs for processed video sequences. Figure 1 shows the growth of the perceived artifact, in units of JND, as the proportion of artifact is increased from 0 to 1 for one particular video sequence. The black points are a sampled estimate of the impairment scale, while the red curve is a continuous functional estimate. The advantage of JND measurements is that they result in absolute units that are meaningful and unlikely to be subject to context effects. JND measurements offer a means of creating calibrated artifact samples and of testing and calibrating video quality models.

This work was performed in close collaboration with the IEEE G-2.1. Subcommittee on Video Compression Measurements of the IEEE Broadcast Technology Society so it could be readily adopted as the US standard for measurement of perceived impairment of digital video. The resulting