Research Report Prepared for Pixel Instruments

Byron Reeves
David Voelker
Stanford University

October 1993

Effects of Audio-Video Asynchrony on Viewer’s Memory, Evaluation of Content and Detection Ability

Research Questions

1. Does audio-video asynchrony change viewers' evaluation of television content, and memory for television content?

2. Can viewers detect audio-visual asynchrony in television programs?

Experimental Methods

Laboratory experiment - conducted at Stanford University.

Procedure - People viewed three different versions (no asynch, 2.5 fields asynch, 5 fields asynch) of six television segments. After each segment, viewers evaluated the speakers in the segments. After viewing all segments, viewers were tested for memory of the segments, and they were asked if they could detect synchronization problems in the segments.

Subjects - 18 adults between the ages of 19 and 45; half men, half women.

Equipment - All video material was shown to single viewers on a rear-projection screen using a 3/4 in Sony U-Matic with an Eiki video projector. Subjects were seated 7 feet from the screen and viewed a 24 inch high picture.

Manipulation of audio-video synchronization

Version #1 -- Perfect audio-video synchronization
Version #2 -- Audio preceded video by 2.5 video fields
Version #3 -- Audio preceded video by 5 fields

Description of stimuli

Message #1 -- Fitness Center ad (male and female announcers)
Message #2 -- Jewelry store ad (male announcer)
Message #3 -- News story about computers (female announcer)
Message #4 -- Ad for radio station (female announcer)
Message #5 -- News story with woman senator as featured speaker
Message #6 -- News story about federal budget (female newscaster)

Measures of evaluation

Eight 8-point scales evaluating main speaker (see Questionnaire):

Effective --Ineffective
Understandable -- Confusing
Pleasant -- Unpleasant
Influential -- Uninfluential
Bungling -- Skillful
Aggravating -- Soothing
Intelligible -- Unintelligible
Agitated -- Calm

Eight 8-point scales evaluating the overall presentation:

Well produced -- Poorly produced
Unsuccessful -- Successful
Fast -- Slow
Likable -- Unlikable
Unpersuasive -- Persuasive
Rough -- Smooth
Cast well -- Cast poorly
Good -- Bad

All evaluation scales (with the exception of slow/fast) summed for overall positive-negative index.

Measures of memory

Three multiple-choice memory questions were asked for each of the six message segments. All memory questions (18) were then summed into an overall memory index for all information.

Measures of detection ability

Two different types of questions asked viewers whether they noticed problems with audio-video synchronization:

Question 1: "Do you have any comments about the segments that you just viewed?" (No mention of possible synchronization problems.)

Question 2: "Please indicate in which of the segments you thought that the timing between audio and video may have been off."

Viewers who mentioned synchronization problems voluntarily constituted a third level of detection.

Summary of Results

1. When audio precedes video by 5 video fields, viewers evaluate people on television more negatively (e.g. less interesting, more unpleasant, less influential, more agitated, less successful). This difference is not large, but it is statistically significant.
(Figures 1 and 2)

2. Audio-video asynchrony has no effect on viewer's memory for audio information.
(Figure 3)

3. Viewers can accurately tell when a television segment is in perfect synch, and when it is 5 fields out of synch. Viewers cannot accurately tell the same segments are 2.5 fields out of synch. (Figure 4)

4. Even though detection is low when asynchrony is moderate (2.5 fields), viewer evaluations are still affected.

Implication Results

1. Audio - visual asynchrony does not inhibit memory for television material, its changes evaluations of television content.

There are two competing theories about how audio-visual asynchrony could influence the psychological processing of television. One view is that asynchrony is confusing, and therefore interferes with thoughtful processing of information. This interference could then preclude the mental rehearsal and elaboration necessary to remember material; hence, asynchrony causes poor memory.

A second view is that asynchrony is unconsciously disturbing. The experience of this disturbance (which is experienced as bad feelings) is then transferred to the primary content of the messages; hence, asynchrony causes negative evaluations of speakers and material with little or no conscious awareness of the effect.

The data in this experiment support the second theory. There were no effects of asynchrony on memory, and if anything memory was slightly better when the messages were maximally out of sync (perhaps because asynchrony attracts greater attention). But significant negative effects were found for the influence of asynchrony on evaluation of television segments and people in them.

Based on this experiment, the following could be said about the necessity of correcting synchronization errors: The presence of audio-video asynchrony will cause material to be negatively evaluated and remembered well - perhaps the worst commercial outcome of a persuasive message.

2. Awareness of audio-video asynchrony is not necessary for synchronization problems to have psychological effect.

Half of the subjects could identify the synch problems and half could not, yet both groups were affected by the problem. Therefore, an appropriate criterion for judging the need for asynch correction is not the subjective awareness of average viewers or even technicians. Rather, the correction is justified because of this experiment which was able to demonstrate effects that viewers were unable to define or discuss.

3. The magnitude of the effect of audio-video asynchrony is statistically small but quite large to the cost of correction.

The size of differences in the experiment were small. Even though differences were not due to chance, the differences only accounted for small amount of the variation in evaluation of television content. So it’s only fair to say that many other characteristics of television content, and methods of presenting content are more important than the degree of synchronization studied in this experiment.

However, the size of the synchronization effect is certainly large to the dollar cost of correcting the problem. There are not many points in the post-production of television where significant negative effects can be precluded with an easy and relatively cheap intervention.

The Effect of Audio-Video Asynchrony on Evaluation of Segments

Figure 1


The Effect of Audio-Video Asynchrony on Evaluation of Individual Segments

Figure 2



The Effect of Audio-Video Asynchrony on Memory

Figure 3



Number of Correct Identifications of A/V Asynchrony by Individual Segments

Figure 4



Number of Subjects who Noticed That Some Spots Were Out of Sync

Figure 5