Saburo Teshigawara developed with Volker Kuchelmeister. Conceived for ReACTOR by Jeffrey Shaw and Sarah Kenderdine.
Multiperspektive image capture constitutes a novel method of documenting the entireness of an event, by recoding it from multiple points of view. In combination with stereo/3D representation, it captures the two modalities of three-dimensional representation, perspective and depth. These modalities are then applied for display in a immersive multi screen installation, the ReActor environment.
The six-channel stereo video dance installation is configured in the ReActor system, an hexagonal projection environment offering the audience a mobile and versatile platform for sophisticated artistic and cultural manifestation and a physically immersive three-dimensional space of representation that constitutes an augmentation and amalgamation of real and virtual realities. It consists of six back projection, passive stereo (linear polarization) screens. The audience can choose to move freely around the hexagon to view individual screens or step back and watch the screens simultaneously. Each screen displays the same scene from the dance performance, specifically choreographed and recorded for this installation, in time synchronicity but from a different perspective, analogous to the architecture of the space within which it is projected.
The modality in which the dance performance was captured, mirrors the physical configuration of the installation. Six evenly distributed stereo camera pairs encircle a stage, which has similar proportions to the virtual space. Precise horizontal and vertical positioning and orientation of the camera heads is essential to recreate a believable illusion of the physical space on screens. To strengthen the imitation of real world perception on screen, the focal length of camera lenses where chosen to reflect the natural field of view of the human eye.
The properties of a stereo image capture system are critical for the overall quality, depth perception and the sense of reality a viewer observes. The relationship between inter-ocular distance, near and far plane, the range of subject movement, focal length and position of the zero parallax plane, all had to be defined. These parameters were generated in a theoretical mathematical model first and its values confirmed in an experimental set up. The subjective qualities of the experiment results lead to a minor adjustment of some of the parameters.
Specifically for this installation, Saburo Teshigawara and Rihoko Sato choreographed and performed multiple solos and duet scenes lasting between 30 seconds and 4 minutes long. The multiple view point capture configuration challenged Teshigawara to rethink the traditional notion of stage performance, where the audience is usually restricted to the seating area and the dancers direct their performances accordingly. In this case, the virtual observer is omni-present, and the choreographer does not have knowledge or control over the position of the audience members at any time during the performance. In addition, the relatively restricted usable space on stage and the properties of the stereo capture system influenced the choreography.
In the four day long rehearsal period, the artists established a framework by defining the aesthetics, composition, lighting design and structure of the piece. The intention was to place the performers in a black void, giving them the notion of floating in space without spatial reference for the viewer. This process required a complex lighting design, due to the modalities of the set up. In this situation, with cameras encircling the stage, it was not possible to place lights anywhere in the scene, only suspended from the ceiling and beneath the stage plane. The final lighting design comprised of a large softbox from the top centre of the stage, giving it the necessary diffuse ambient light without casting strong shadows. Additionally, two concentric rings of six directional lights from the ceiling and two rings of (diffuse and directional) from under the stage plane were added. The intensity of all the lights could be controlled allowing us to make dynamic lighting changes during a take. The linear composition of the piece was defined as a series of solos and duets in which each began and ended in total blackness. From the beginning of each take, the performers were in place on the dark stage, then the lights faded in, and the take ended with fading lights. This notion simplified the final arrangement of the segments and gave the piece its structure (final total length 17:30 min).
State of the art remote-head compact single chip CCD cameras were used to achieve archive cinematic quality and fidelity of imagery. The cameras are able to capture images in high pixel and color resolution (1600×1200 pixel, 10bit) and high frame rate (30 fps progressive). Figure 7: Video still from six points of view
First-rate lenses allowed adequate aperture control, minimal color aberration and good focus. Image sensing, capturing, recording and previewing are all in the digital domain, ensuring the best possible image quality. The raw uncompressed data from the twelve cameras is streamed through capture boards directly onto hard disk arrays within the recording computers.
The post-processing workflow comprised of image extraction from the raw stream, bayer filtering, color grading and background noise removal. Stereo pairs are then aligned with the help of a calibration image and the zero parallax plane is defined. After cropping the images, a side-by-side stereo frame is generated (2800×1050 pixel resolution) and the sequence compressed in a lossless format. These sequences are then arranged to the final six clips and compressed in MPEG4 for playback. For synchronized playback across the six ReActor stereo projections screens, a custom playback, distributed in a cluster of six computers, was developed.
The choices of cameras, lenses, lighting design, the post-processing workflow and the high standard in stereo system design and calibration resulted in extraordinary image quality and fidelity which are in line with big Hollywood budget stereo feature film production, yet at a fraction of the cost.