timing requirements for sequential frame/field stereo

[T3D John W Roberts <roberts@xxxxxxxxxxxxxxxxx>]

There was a recent discussion on photo-3d regarding the number of frames per
second needed in a movie or video display in order to produce a perception
of continuous motion. I saw a television program a week or two back in which
it was stated that most people see pretty much continuous motion at 11 or 12
frames per second, with 24 frames per second (the standard for movies) being
judged sufficient for most scenes. (However, the *refresh rate* needs to be
48-72 Hz (or higher for more exacting applications) to avoid undue perception
of flicker - this is accomplished in movies by flashing each frame more than
once.

In sequential frame stereo, an additional problem shows up - if there is any
motion in the field of view, the left and right images presented to the
eyes differ not only from parallax, but also as a result of this motion -
for instance, if a person is rapidly raising or lowering his arm, the arm 
will be at different heights from frame to frame. It's true that these images 
are presented to the eyes at different times, but experience viewing the
field-sequential 3D from a Toshiba camcorder has convinced me that the part
of the brain that is responsible for stereo perception uses some very high-
speed circuitry to establish the relative timing of the two views. In other
words, rather than running two simulations of the outside world (one for
each eye), the brain at any given time will be trying to compare the view
from one eye at time T and the view from the other eye at time T-1 (since
that's the latest image available from the frame sequential stereo display).
The result is that the more rapid the motion on the display, the more visible
artifacts will turn up when trying to view it in stereo.

The Toshiba camcorder uses NTSC, with interlaced fields updated at 60 Hz
(or 59.94), so the total period for display of a stereo pair is 1/30 second.
This is fine for many applications, but not really satisfactory for fast
action such as a tennis game. This is especially noticeable if you switch
to the high speed shutter setting (~1/1000 second exposure, as compared to
the ~1/60 second exposure for normal mode, which tends to blur motion).

So - in the same way that useful frame rate and refresh rate had to be
determined for 2D video, there is a need to determine what timing would be
most useful for sequential frame stereo. Thus far the rate used has been 
determined by the capabilities of the display method used (i.e. NTSC,
Solido Imax). These rates produce good results for many types of scenes,
but for rapid motion there are significant problems. Does anybody have a
feel for the kind of timing that might produce acceptable results for
videos of tennis games, car races, birds in flight, etc.?

In 3D video where both images are presented in a single frame, the placement
of the images and the scanning format become matters of concern. In a
progressive-scan display where the top half of the screen is used for one
view and the bottom half for the other view, the timing issue is essentially
the same as that of an interlaced sequential-field display - the right and
left views will be separated in time by about half of a frame period. If the
right and left views are presented side by side, the homologous points will
be separated in time by about half of a horizontal scan period (1/30000
second for NTSC). (The statements in this paragraph assume a display where 
the horizontal scan is faster than the vertical scan - reverse them for a
vertical-scan display). It may be possible to tighten the timing in a
film presentation by rapidly flashing the two views and using very high
speed shutter glasses.

As a possible reference, the human eye moves at rates up to 200-1000 degrees
per second, but it's not clear that the visual information is actually being
processed during the periods of most rapid motion (or perhaps only certain
parts of the information are processed).

John Roberts     NIST


------------------------------