| During the last decades moving images, video
and screens have expanded from on-the-wall projections to dynamic
and multi-modulated images in different spatial settings –
on multiple screens, in dynamic and interactive room environments
and in an immersive physical context. Narratives have evolved
from linear storytelling to open database structures, influenced
by computer game-like scenarios and simulated worlds. The dramaturgic
tools have evolved from the Aristotelian catharsis format to the
open-ended closure, expanding from strict linear forms to generative
codes using meta-tagging and computation. Pre-recorded video sequences
and live-feed real-time video are combined in artworks and on-line
worlds, commercial and open source software are integrating visual
elements on all media platforms – from computer to handheld
devices.
This expanding possibilities of media technologies
have opened up the screen into a dynamic room, an interactive,
real-time visual and cinematic stage. The screen is no longer
a static element; its scaleable and flexible, its immersive qualities
and interactive possibilities are challenging for visual artists
to explore and use in new artistic contexts. This short exposé
tries to pinpoint some contemporary tendencies and prolific artworks,
thus creating a reference and artistic perspective. It was originally
made as research for an interactive performance – the PRAMnet
project "Journey to Abadyl".
The exhibition and catalogue FUTURE CINEMA
/ The Cinematic Imaginary after Film at ZKM curated and
edited by Jeffrey Shaw and Peter Weibel
in 2003, was the first large-scale expose over experimental film
and digitally based video-art. The exhibition included historic
and contemporary media works, and made the first comprehensive
theoretical overview of the expanded narrative strategies of visual
moving imagery.
Using a setup of keywords, the authors formulated
dramatic tools for the future cinematic art and formulated a possible
future development of the art of the moving image. In their curatorial
statement, online at ZKM website, Shaw and Weibel wrote :
"The Future of Cinema can
be delineated from two sources. One way is the expansion
of existing cinematographic methods and codes into new
areas. The other way is the convergence of cinema, TV
and net. The classical cinema can be defined as collective
experience of one fixed stable projector projecting a
moving image on one screen in one room. Therefore each
change of one of these factors, for example multiple screens,
panorama screens, moving projections, different rooms,
is already an expansion of the contemporary practises
of cinema.
On the level of the material display
of the image, many new techniques are also in development.
Smart materials are developed which can be new sources
of light and colour. New lenses are developed which contain
information as a temporal code. In general, there is a
change from refractive optics to diffractive optics.
Especially new results
are to be expected from multi-local and multi-user virtual
environments, from massive parallel virtual worlds,
which can be developed based on the global net. These
new communication channels of the net in combination
with GPS-Systems, satellite transmission and WAP mobile
phones are allowing new forms of interactive personal
and collective cinema on a digital basis."
|
Since this was written Shaw and Wiebel in 2003,
we have seen several of their predictions come alive; online worlds
like Second Life, immersive web based multiplayer games like World
of Warcraft, to name but one, and numerous examples of the emerging
narrative worlds Shaw and Weibel pointed out as the future.
The following examples are primarily selected
as references to the interactive narrative installation PRAMnet
have investigated, they can be seen as examples of state-of-the-art
narrative works where interactive and digital dynamics have transformed
the visual screen during the last decade.
Keywords from FUTURE CINEMA / re-combinatory / immersive
/ Examples Re-combinatory /
Interactive cinema in its early versions meant
using different user interface schemes to re-edit and re-sequence
pre-recorded material. Lev Manovich developed
"Soft cinema", a software which shifted
pre-recorded and edited footage into pre-programmed sequences
on a multiple screen divided into a Mondrian grid. A sort of meta-collage
using algorithms for meta-tagging of a documentary content, a
database cinema presented as an installation. "Soft Cinema"
has no open interactive interface, you watch a stream of visuals,
a narrative collage using voice-over and soundsscapes from the
recorded footage.
The software behind "Soft Cinema" are
designed to combine keywords and tags to create different flows,
or streams, of experiences. The keywords are combined with different
tempo and time-capsules to make innumerable and multiple variations.
 |
| Illustration 1: Lev Manovich_"Soft
cinema |
The "Decay of Fiction",
an interactive drama documentary on DVD, are based on film sequences
from interiors in a Hollywood hotel, THE AMBASSADOR. Here seamlessly
narrated video and sound emerge from the corridors and rooms as
you enter them on the screen with movements from your mouse. The
visuals are interfoliated with still imagery, audio and footage
from film sequences shot in the hotel during its Hollywood glamour
years. These images are layered on top of the actual interiors,
creating an almost ghostlike and virtual presence.
This piece use a labyrinthine structure,
you explore a narrative world using screenbased interaction and
navigation controlled by the mouse on a DVD-platform. The material
in the piece are pre-edited and combined in an simple interface
setup.
Viewing and experiencing the content
of "Decay of Fiction" are more interactive than the
first glance makes you believe. Stay long enough on a spot in
a corridor or at a stage in the old hotel and randomly organized
footage and clips linked to this spot are screened. The programming
behind use a combination of metatagging and scipts that could
be transformed into a roomlike experience exchanging the mouse
with a body movement in an installation.
This multiple documentary are shot
by filmmaker Pat O‘Neill and this expanded
cinema setup are created in cooperation with The Labyrinth Project
at the Annenberg centre, USC.
 |
Illustration 2: Pat O'Neill_"Decay
of Fiction" |
Multiple screen /
Video artist have used multiple
screen format to construct narratives – Eija-Liisa
Ahtila with works like "Consolation Service"
use divided screen and multiple viewpoints to create a narrative,
which change viewpoint between the characters and move freely
in time and setting – this creates an associative form of
experience for the audience.
The three screens also create a
roomlike experience, you get close to either of the characters
in the fictional story.
Isac Julien, a
visual artist and filmmaker, have been using multiple screen movies
more than ten years, creating imagery that blend different point-of-views
and narrative elements. In one of his recent works "True
North", a three-channelled video screening, Julien
tells the story of the first black person to visit the North Pole.
The images mix landscape views with personal reflections and fictionalized
documentary scenes that create a narrative flow and mix.
This model of using the screen creates
a panoramic view and an combination of close-up and wide-angle
shots creates a open narrative form, the viewer is almost placed
in the editors chair. Moving from one screen to the other you
compose an personal experience.
 |
Illustration 3: Isaac Julien_"True
North" |
/ Examples Immersive / Video as texture / digital
material.
Many media/digital artists have
developed new means of combining live-feed with pre-recorded video.
Digitally synthesized graphic content are used in semi-autonome
processes, designed to interact with motion-movement setups or
controlled by a VJ-system or programming feature.
The emphasis of these immersive
videoworks lies more on texture and rhythm, using expressive colour
and graphics to create an emotion and feel rather than on story
and narrative. The moving images can be projected on a flat screen
as graphic illusions or in a stage setting on objects or performers,
creating texture and dynamic tension.
An early interactive example are
"Text Rain", by american media artist Camille Utterback,
where installation participants stand or move in front of a large
projection screen. On the screen they see a mirrored video projection
of themselves in black and white, combined with a color animation
of falling letters that can be "caught" on the contours
of your body and made into words and poems. A playful and engaging
work.
Another examples of this artistic strategy include Norwegian media
artist HC Gilje who worked with manipulated video
on stage performances, as video-sound-pieces with live musicians
as in the early piece "videonervous".
"The main focus of the project was to make use of the immediacy
which the digital technology offers, and develop video as an instrument,
but also to create fluid spaces through projections, and the ability
of video to function as both a set design element and a narrative
element".
Gilje investigated these possibilities
in a serie of stage performances in collaboration with dancers
and musicians in norwegian Kreutzerkompani, creating interactive,
live-feed videoloops as stage settings. On his blog "Conversation
with spaces" HC Gilje keeps track on his ongoing work with
programming visual space - a series of "relief projection installation"
which he refers to as "a dynamic audiovisual landscape,
a spatial light painting".
 |
Illustration 4: HC Gilje_"Videonervous
2 01" |
Digital noise and manipulation /
Another form of immersive performances
are made by he Austrian artist duo GRANULAR SYNTHESIS,
their aesthetic concept can be seen as an audiovisual abstraction,
using digitally generated images and sounds. "How does an
image fall into pixels as the resolution gets lower and what is
falling into pixels, and is falling into pixels the right equivalent
anyway? Any resulting algorithm or translation we see as a means
to create an abstract reality with emotional substance and depth,
a subjective artistic invention."
One of these works are "Model
5", an live-feed installation where five large screens
display portraits of people that slowly dematerializes into digital
noise and back into images again; here the audio and visuals are
manipulated live as a large scale VJ session where the process
are controlled by the artists. Real-time and hypnotic.
 |
| Illustration 5: Granular Synthesis_"Model
5" |
Other more playful and consumer oriented works include IOO
Interactive, an Italian company that create environments
and events using small-scale dynamic media – sensitive
wall / floor were imagery and surfaces change and reacts to
audience physical movements. These setups are used as demonstrators
at public fairs and exhibitions on trade shows.
Navigable /
Tracking devices and sensor tools can be
used to create an immersive room were physical movements are
interpreted and correlated with pre-recorded video / film
sequences. These dynamic set-ups are in a sense similar to
virtual reality techniques and game mechanics, you're movements
in the space create an interactive visual experience. You
put yourself in the players seat, you become the editor and
director of the experience.
In "Threatbox.us"
by artist Marie Sester, an art installation
with web surveillance interface, a movie frame "attacks"
visitors via a robotic video projector and computer vision
tracking system. On the screen you directly targeted by documentary
images, you are hold at gunpoint in extreme and violent situations.
Installed in a gallery space, this setup creates a claustrophobic
comment on how public space can be controlled and occupied,
and used almost as a terror device.
The classic piece "I Cinema"
are an immersive sound and video system inside a circular
dome constructed by artist Jeffrey Shaw.
His first installation piece using this dome was EVE (Extended
Virtual Environment) in 1993. Here the projector are controlled
by the audience, as you "shoot" the beam around
on the walls, you see a display of images, video clips and
sound emerging from the inside of the dome. The experience
is like watching through a peep-hole into a secret world,
you actually navigate in a constructed 3D environment with
video, graphics and sound.
At Future Cinema exhibition French artist
Jean Michel Bruyère used the dome
for creating a narrative interactive piece based on Ovidius:
Metamorphoses.
 |
Illustration 6: Jeffrey Shaw_"EVE
(Extended Virtual Environment)" |
Smaller scale projects using sensor and tracking technologies
are numerous, the one-to-one sized "Men that
fall" by artists Geska Helena Andersson
and Robert Brecevic collaborating at Performing
Pictures group in Stockholm are one simple, but striking example.
An interactive one-to-one video image of a person, who - if
you get to close to the image - starts to fall down. Back
away and the person rises again. Displayed in a public space,
a shopping mall, this piece created laughs and playful interaction.
Immersive and navigable /
Large-scale public interactive works using
video and projection techniques are often costly and therefore
relatively rare. In 2005 Mexican-Canadian artist, Rafael
Lozano-Hemmer developed the large-scale public art
installation Under Scan featuring 1,000 interactive
portraits. It was created specifically for the East
Midlands region of the United Kingdom.
Here people walked around on a brightly lit
square, turned into a gigantic dynamic screen. In their shadows
on the ground surface video-portraits emerged. Their bodies
moved and their heads turned to look straight at the passers
by. When a shadow moved away from a portrait, the portrait
likewise reacts by losing interest and looking away. This
large-scale work combine surveillance cam-technique and as
Lozano-Hemmer describes it "...a system that automatically
points 14 robotic-controlled projectors to the locations where
they will intercept the trajectory of pedestrians and cues
a sequence of video for them. "
 |
Illustration 7: Rafael Lozano-Hemmer_"Under
Scan" |
The sophisticated and advanced technology behind Lozano-Hemmer
public installations are made in collaborations with programmers
and software developers on large-scale sights around the world.
His piece Vectorial Elevation, celebrating
the arrival of the year 2000 in Mexico City's Zócalo
Square used eighteen powerful searchlight beams, illuminating
the night sky over the central square. The design of these
light sculptures were controlled by an online 3D simulation
program on a website open for everyone and transformed the
sky into a screen were geometrical shapes formed stunning
visuals.
In one of his latest large-scale public work,
Solar Equation, Lozano-Hemmer use the new
portable screens and interfaces on handheld devices and smartphones
as a dynamic dynamic tool. On a large inflated ballon, video
simulations of the suns turbulent sunspot acitvities are projected,
the forms and intensity of these shapes are controlled from
an app that can be downloaded from a website. This live-feed
interactive animation where staged at the Light in Winter
Festival in Melbourne 2010, uses actual data from NASA satellites
as base for the simulations created by mathematical, fractal
equations.
Dynamic screen / room in relation
to "Journey to Abadyl" /
A variety of these techniques using video
and on-screen narratives are possible to develop further.
Different combinations of narratives based on computer game
engines and scenarios can be investigated and tested. The
examples mentioned in this brief expose are but a few of the
the multiple area of possibilities made accessible with digital
tools and combinations of software/hardware techniques.
In our research we concluded that the main
limitations of using videostreams as content in interactive
artworks are the linearity of the images. True you can find
exemples of real-time editing and rendering of footage online,
art-pieces that enables you to create your own moving image
edition from a database on the fly. You can scroll back-forward
on image sequences and you can digitize videoimages in real-time
using pre-edited filters and effects from opensource software.
In PRAMnet we have worked with and devised
different setups, where physical movements and positions in
a sensor-tracked room can trigger images, sound, video and
sequences of narrative elements.
Jörgen Callesen's Falling man
was an early prototype using quick time loop and software,
so a performer with his/her body movements physically control
the actor on the screened video. Callesen's interactive video
sculpture Emotional Fish, performed at Warehouse9
in Copenhagen September 2007, elaborated on this technique
using computer generated images and sound set up with infrared
light and sensors in a public space.
 |
Illustration 8: Callesens interactive performance
"Emotional Fish"
|
The prototype Wunderkammer used a seemingly
easy setup – two opposite screens and a tracking device
using a webcam capturing movements on the floor. Audience
control video/sound in a setup with "good-bad" imagery
by moving around in the room – together and away from
one screen the opposite screen are active and vice versa.
In the prototype model we tested, the setup was made on a
table, using black and red miniature figures mounted on a
stick, almost as puppets, with two 19" inch computer
screens.
 |
Illustration 9: PRAMnet prototype
"Wunderkammer" |
The complexity of these setups can be scaled up to four screens
and database organized sequences. It can be used as a setup
for live-feed immersive video and sound environment as well
as more narrative based scenarios. Creating a dynamic screen
and room where the laws of time and space can be altered and
transformed.
In "Journey to Abadyl"
context /
During our workshops and discussions we found
it useful to percieve and organize the tools and technologies
according to two basic forms and rules.
Narrative moving images can be organized
in two major models – pre-recorded
and edited and live-feed set ups.
The pre-recorded can be
divided up in different narrative strategies like –
background story, eyewitness tales, multiple-choice scenarios
of events, arguments and discussions between characters.
Using these models we can establish a virtual
setting, create back-stories of characters, which the audience
encounter or/and recreate past or future events. Here a the
game mechanics and narrative dramaturgy rules and scenarios
developed by computer game designers form a backdrop of knowledge
and experience on how to set up a database of content.
The live-feed set up use
live manipulation and video feeds – as interaction it
can be controlled by a performer/guide or by the participants.
The viewing and experience can be orchestered by various scenarios
such as simultaneous events in a labyrinthine setup, by a
"VJ organ" controlled/triggered by physical movements,
by real-time stage-set design using projected colour and synthesized
images.
From these basic models we can choose to
develop media for different narrative setups, using specific
story tools for different city parts. Thus creating a variety
of possible interactive setups with video narration and moving
images.
( The JUKEBOX in TRAORA can be seen both
as a visual metaphor and a real tool for eyewitness tales
and ceremonies – music for the "masses" and
entertainment machine. Newscasts with debates between opposing
political leaders. Environments in the jungles of ARLEEANS.
Historic re-creations of events. The responsive digital guide
- controlled by audience movements. )
The short movie IN SEARCH OF THE
MILITANT CODE, a collaboration between Thore Soneson
and artist Michael Johansson, is a story set in the city part
NEEBI. An outtake from the fictive world of Abadyl. This work
explores the possibilities of combining narrative strategies
and tools from traditional filmmaking and a computer generated
universe. Video and digital sceneries in 3D are interfoliated
in a mockumentary style.
 |
Illustration 10: Thore Soneson_"In
search of the Militant Code" |
|
