Multiperspektive image capture constitutes a novel method of documenting the entireness of an event, by recoding it from multiple points of view. In combination with stereo/3D representation, it captures the two modalities of three-dimensional representation, perspective and depth. These modalities are then applied for display in a immersive multi screen installation, the ReActor environment.
Double District is a virtual human-scale duet choreography in collaboration between the acclaimed Japanese choreographer /dancer Saburo Teshigawara and Volker Kuchelmeister. Performer: Saburo Teshigawra, Rihoko Sato
Model of Installation
The six-channel stereo video dance installation is configured in the ReActor system, an hexagonal projection environment offering the audience a mobile and versatile platform for sophisticated artistic and cultural manifestation and a physically immersive three-dimensional space of representation that constitutes an augmentation and amalgamation of real and virtual realities. It consists of six back projection, passive stereo (linear polarization) screens. The audience can choose to move freely around the hexagon to view individual screens or step back and watch the screens simultaneously. Each screen displays the same scene from the dance performance, specifically choreographed and recorded for this installation, in time synchronicity but from a different perspective, analogous to the architecture of the space within which it is projected.
Installation at the eArts Festival Shanghai, 2008
This configuration of the system arrangement allows members of the audience to experience an immersive three-dimensional virtual space (the architectural container of reActor) in the physical world. The outcome is a very natural interface that does not require introduction or explanation. In common with the material world, a transformation in location and orientation in the exhibition space leads to an equivalent perspective change in the virtual world or the viewpoint the dance performance is observed from.
The modality in which the dance performance was captured, mirrors the physical configuration of the installation. Six evenly distributed stereo camera pairs encircle a stage, which has similar proportions to the virtual space. Precise horizontal and vertical positioning and orientation of the camera heads is essential to recreate a believable illusion of the physical space on screens. To strengthen the imitation of real world perception on screen, the focal length of camera lenses where chosen to reflect the natural field of view of the human eye.
Model of recording set-up
The properties of a stereo image capture system are critical for the overall quality, depth perception and the sense of reality a viewer observes. The relationship between inter-ocular distance, near and far plane, the range of subject movement, focal length and position of the zero parallax plane, all had to be defined. These parameters were generated in a theoretical mathematical model first and its values confirmed in an experimental set up. The subjective qualities of the experiment results lead to a minor adjustment of some of the parameters.
Video Stills in anaglyphic format
Specifically for this installation, Saburo Teshigawara and Rihoko Sato choreographed and performed multiple solos and duet scenes lasting between 30 seconds and 4 minutes long. The multiple view point capture configuration challenged Teshigawara to rethink the traditional notion of stage performance, where the audience is usually restricted to the seating area and the dancers direct their performances accordingly. In this case, the virtual observer is omni-present, and the choreographer does not have knowledge or control over the position of the audience members at any time during the performance. In addition, the relatively restricted usable space on stage and the properties of the stereo capture system influenced the choreography.
In the four day long rehearsal period, the artists established a framework by defining the aesthetics, composition, lighting design and structure of the piece. The intention was to place the performers in a black void, giving them the notion of floating in space without spatial reference for the viewer. This process required a complex lighting design, due to the modalities of the set up. In this situation, with cameras encircling the stage, it was not possible to place lights anywhere in the scene, only suspended from the ceiling and beneath the stage plane. The final lighting design comprised of a large softbox from the top centre of the stage, giving it the necessary diffuse ambient light without casting strong shadows. Additionally, two concentric rings of six directional lights from the ceiling and two rings of (diffuse and directional) from under the stage plane were added. The intensity of all the lights could be controlled allowing us to make dynamic lighting changes during a take. The linear composition of the piece was defined as a series of solos and duets in which each began and ended in total blackness. From the beginning of each take, the performers were in place on the dark stage, then the lights faded in, and the take ended with fading lights. This notion simplified the final arrangement of the segments and gave the piece its structure (final total length 17:30 min).
Stereo camera pair
State of the art remote-head compact single chip CCD cameras were used to achieve archive cinematic quality and fidelity of imagery. The cameras are able to capture images in high pixel and color resolution (1600x1200 pixel, 10bit) and high frame rate (30 fps progressive). Figure 7: Video still from six points of view
First-rate lenses allowed adequate aperture control, minimal color aberration and good focus. Image sensing, capturing, recording and previewing are all in the digital domain, ensuring the best possible image quality. The raw uncompressed data from the twelve cameras is streamed through capture boards directly onto hard disk arrays within the recording computers.
The post-processing workflow comprised of image extraction from the raw stream, bayer filtering, color grading and background noise removal. Stereo pairs are then aligned with the help of a calibration image and the zero parallax plane is defined. After cropping the images, a side-by-side stereo frame is generated (2800x1050 pixel resolution) and the sequence compressed in a lossless format. These sequences are then arranged to the final six clips and compressed in MPEG4 for playback. For synchronized playback across the six ReActor stereo projections screens, a custom playback, distributed in a cluster of six computers, was developed.
The choices of cameras, lenses, lighting design, the post-processing workflow and the high standard in stereo system design and calibration resulted in extraordinary image quality and fidelity which are in line with big Hollywood budget stereo feature film production, yet at a fraction of the cost.
Direction, choreography, lighting design and costumes: Saburo Teshigawara (Karas Tokyo [JP])
Developed with: Volker Kuchelmeister
Performed by: Saburo Teshigawara and Rihoko Sato
Conceived for: Sarah Kenderdine's and Jeffrey Shaw's Re-Actor environment
Production manager, technical director, stereoscopic cinematography, video and audio post-production: Volker Kuchelmeister (iCinema)
Lighting design: Paul Nichola
Lighting technician: Rob Kelly (NIDA)
Production assistant: Sue Midgely (iCinema)
Producer: Richard Castelli (Epidemic)
Co-produced by: Karas, Tokyo, Epidemic (Paris, Berlin), Le Volcan Scène nationale, Le Havre, UNSW iCinema Centre, Sydney, and kindly supported by Museum Victoria.
Premiere at Shanghai eArts Festival 2008, Urbanized Landscape. Shanghai Science and Technology Museum. Curator Richard Castelli (epidemic, France). 18 Oct - 10 Nov 2008
Volcan Scène nationale du Havre / Le Havre, France. 16-27, Jun 2009. Curator Richard Castelli (epidemic, France) .