Abstract
'Dot, a videogame with no winner' is an audiovisual performance with synchronized sounds and images, played by a 'game console' built and programmed by HOL, and controlled by retro videogame (Nintendo) joysticks. The instrument is completely autonomous and works without the need of a computer, using only a projector and a sound system to play its content.
This paper passes through the complete process of creation of the performance, showing the artistic concept, aesthetics and some generative techniques used by the artist to add some randomness to the process so that the show is different each time it is performed.
 |
| Example of image generated by the instrument |
1- About HOL
HOL, a project created in the beginning of 2008 by the transmedia artist Henrique Roscoe (aka 1mpar), intends to mix contemporary art, live audiovisual performances, digital art and the construction of custom-made interfaces and instruments.
The project plays out in the form of live audiovisual performances. Initially, a concept is created and, out of it, all the sound and image elements are developed in order to show emotions and sensations about the chosen theme. This narrative happens in a different way from traditional cinema and video, by the fact that it is performed live, taking a new form in each presentation, besides using abstract shapes to create metaphors of the real world. These analogies are made using the fundamental elements of the image that, through its colors, shapes and movements suggest sensations that will give the spectator ways to understand the poetics of each composition. The use of these elements is based on the studies of the Russian artists Malevich and Kandinsky. Both believed in the power of fundamental elements of image in the construction of personal narratives. Malevich, with his Suprematism, emphasized the intrinsic power of shapes and Kandinsky sought, through his work, extend painting through the suggestion of movement and the use of musical elements in his works (exemplified in titles borrowed from musical terms as Fugue, Improvisation, etc.).
 |
| HOL live at On_Off Festival. São Paulo, 2009 |
HOL is a high technology project, and at the same time, it uses traditional elements of modern and contemporary art. It has its influence from an art field called visual music where sound and image elements, synchronized, create a seamless narrative in which musical elements such as rhythm, harmony, etc, are translated into the visual field. The project is influenced by pioneering artists in this area like Oskar Fischinger, Norman McLaren and, more recently, Alva Noto, Ryoji Ikeda, among others. However, HOL adds to the purely aesthetic focus - common to this type of work - the conceptual question, inserting reflective and critical elements about each theme.
Another important feature of the project is the creation of all programming of each performance from modular and generative software like vvvv and max / msp, allowing complete customization of what will be played live, both in sound and image. It would be something like creating a new software for each part of each presentation. Moreover, the
random issue, widely used in the musical works by John Cage for example, plays an important role in determining paths that are not fully controlled by the artist. This procedure allows an expansion to infinity of possibilities to play each performance. Thus, each piece is unique and happens differently every presentation, though it always stays loyal to its theme.
Taking the concept of personalization to a higher level, the project goes further by also creating its own instruments, some of them allowing a complete separation from the computer and enabling the implementation of an audiovisual live performance using fully autonomous instruments created and programmed by the artist.
HOL has performed live at major festivals from this area in Brazil such as Sónar, FILE, Live Cinema, On_Off Itaú Cultural, Multiplicidade, FAD, and abroad as Robot and LPM (Italy) and Dialectos Digitales (Bolivia). With video documentation of performances it has participated in various festivals and exhibitions like Art Basel (Switzerland), Magmart (Italy), Kunstfilmtag (Germany), Generative Arts (Italy), Images Contre Nature (France), Espacio Enter (Spain) Videoart Athens (Greece), Computational Aesthetics (Canada) Sismógrafo (Belo Horizonte, Brazil), Hacklab (Bahia, Brazil), among others.
2- DOT, a videogame with no winner
2.1- Concept and first researches
I have been working with live audiovisual performances since 2004 and with creation of custom interfaces since 2006, when he started working with the microcontroller board Arduino. After producing some performances using standard software and hardware, I decided to start building my own interfaces and using custom software so that the results would be more creative and unique and specially to use every single element to say something about the theme that is being dealt.
In 2008 I created a new live audiovisual project called HOL that would differ from my VJ performances specially because of the freedom to build a show covering the whole process, from the definition of the concept and tools, passing through software programming, construction of hardware, creation of all sounds and images, until the presentation in the form of a live audiovisual performance.
In order to achieve this freedom to program exactly what I had in mind, I started working with the software vvvv. Differently from other VJ applications that have a standard interface, vvvv (and other software live max/msp. processing, etc) has initially a blank white page and all the programming has to be made by the artist, filling exactly his needs. In this way, instead of using a limited amount of features given by standard live images software like resolume, modul8, etc., the programmer has the freedom to build his own logic, features and interface. In vvvv, you can generate real time animations and control every single parameter live. The same happens in the audio side with Max/Msp.
All performances by HOL use this kind of software and I wanted to use this way of thinking also for the hardware side. So I began to use my recent experience with Arduino to build custom interfaces that would control parts of the performance. The first interface was built for the performance 'Aufhebung', in 2009. This interface was composed by 4 cylinders, each one with a LED and a IR distance sensor in their bottom. There was a lid on the top of each cylinder that, when pulled up, changed parameters in audio, images or both, according to the programming in vvvv.
 |
| Interface built for 'Aufhebung' performance (2009) |
After some time, I decided to go beyond standard hardware - a laptop - and started with the idea of building my own custom hardware, that would be autonomous and would not need a computer in order to play the performance.
The first attempt to build an autonomous audiovisual instrument was K-synth - http://1mpar.com/index1.php/portfolio/k-synth/. This small audiovisual instrument generated simple black and white animations and pure frequencies sounds. It was built using a cassette tape enclosure and had two RCA outputs, one for sound and other for images. The programming was made entirely in the Arduino software and, once finished and uploaded, the instrument wouldn't need a computer to work.
 |
| 'k-synth' - Autonomous audiovisual instrument (2011) |
In 2011 I was approved as resident at Marginalia+Lab to develop a research about building an audiovisual synthesizer using the Arduino board as a base. The idea was that the instrument had to be independent and would not need a computer to operate.
In the residency, it was first built a prototype where the animations were in black and white and had simple sounds, using the technique of circuit bending coupled with the Arduino programming. This instrument was called 'Glitchy Square'.
Starting from a basic circuit using Arduino, the artist made some circuit bending, added some components and tested many ways of connecting them. The result was an audiovisual instrument where audio and image feedback themselves, generating unusual sounds and animations. As the instrument is connected to an analog TV, sound output generates different types of noise according to the programmed animation and to the size of the shapes on screen. Sound interferes in the image as each audio pulse also goes to the video output, changing what is displayed on the screen.
The circuit was inserted in a wooden box and named 'Glitchy Square' – an homage to the famous painting 'Black Square', by Russian painter Malevich, who tried to reach the limits of (non)representation and this painting would be, according to him, the maximum of the non objectivity.
This instrument adds movement and sound to this idea and seeks the sensations of non objectivity. Images receive influence from sounds and the other way around, and the fundamental elements of the circuit (current, tension, etc) generate the content.
 |
| 'Glitchy Square' - Autonomous audiovisual instrument (2011) |
For exhibiting the work, the instrument is put on a desk and a video with the documentation and explanation about its operation is exhibited on a video projection. In this video, I also present an audiovisual composition created using exclusively sounds and images generated by the instrument.
Later in 2011, using Arduino and a shield called Gameduino, a new instrument was built. The circuit was inserted into a standalone console and programmed to have 5 different animations, each functioning as a level of a game. Following the aesthetics of the first video games, according to some of the limitations of the board, that later became a fundamental part of the concept of a performance called 'DOT, a videogame with no winner'.
2.2 - The Instrument
The instrument has an Arduino board, a Gameduino shield and two SNES controllers, enclosed into a plastic box. Arduino receives the information from the 2 joyticks and each button is assigned to a variable. The programming was made inside the Arduino enviroment, using the Gameduino library. Gameduino is a shield that can be linked to Arduino and has VGA (video) and a P2 (audio) stereo output.
 |
| Hardware used in the instrument |
Gameduino (http://excamera.com/sphinx/gameduino/) is the core of the circuit. It can generate color images and 64 synthesized independent voices. But this shield has many limitations. It can handle a 256 maximum number of sprites on the screen at the same time, each one with 16x16 pixels. It can display a small number of colors simultaneously, and also a limited processing amount. Video output has only 400x300 pixels and 512 colors.
A library (http://code.google.com/p/nespad/ ) was used to get the information from SNES controllers and convert them in variables that Arduino could understand. The joysticks control all the real time functions during the performance and each one controls different parameters according to the 'level' that is being played.
 |
| Table with SNES pinout |
After building the circuit, all the components were put into a plastic case with a 'retro feeling' adhesive on top. A 12V power source was used for it to work standalone. For the live performance, the VGA output should be sent directly to the projector and the P2 connected to a stereo sound system.
 |
| Instrument used in the performance 'DOT, a videogame with no winner' |
2.3 - Concept
After the instrument was built, the performance 'DOT, a videogame with no winner' was created with the idea of criticizing some aspects of game logics, but using its own aesthetics, sounds and characteristic graphic elements. The performance criticizes, through abstract images, themes linked to videogames and people's everyday life. All images and sounds were created and programmed by the artist and they are played in real time, in a 30 minutes performance. For the live performance, the audience is invited to play with the artist, and both produce together the soundtrack and the images.
The performance has 5 parts, like the levels of a game. People from the audience stay seated on cushions, on the stage, turned to the screen, controlling all the elements of the performance using 2 Nintendo joysticks – as if they were playing a videogame at home.
The title 'DOT, a videogame with no winner' is also a critic to the fact that only winning is valorized, while the process is given low priority. The aim of this "game" is not winning, but participating of the creation process of a spectacle.
The performance works as a game, and each part deals with a specific theme:
- Level 1 – Fragment
Violence: critic to the stimulus of violence in games. In a hole opened over a red background, the players movements draw veins that leave tracks of blood.
- Level 2 – Put you down
Ones value is measured by the diminution of the other. Two elements in the form of screws are stuck on the ground. The only possible action is to hit the 'opponent', sinking him more and more.
- Level 3 – Capital
Excess: each player controls the position of falling objects. These objects fill the whole screen until there's no more space for the player. This level deals with themes like consumerism and the necessity of filling all the empty spaces in people's life.
- Level 4 – Mimesis
Standardization: critic to fashion and the imitation behavior. Abstract shapes pass through the screen and the player should change his own shape in order to become similar or different from the others.
- Level 5 – To the Future
Decadence: Melancholic ending where both players goes down a 45 degrees ramp. The only possible movement is delaying the arrival of the bottom.
2.4- Aspects of Live performance
Each part of the performance has its own programming in which conceptual aspects can be seen in every single element. The instrument has no pre-recorded sounds or images and everything is created in real time, in a partnership between the artist and five invited players. There is no rigid score to be followed, only instructions for the players about what each button's function.
Another important concept is the randomness. This feature appears in the composition in the form of random parameters generated by the system and also by the public's participation. Each invited participant can interfere in a complete unpredictable way in the performance as they press each button in the joystick. As many parameters in sound and image are controlled by the guests, the host doesn't have the complete control of what is going on, although there
are some limitations that keep these randomness on an acceptable range. The role of the participants in fundamental to the success of the performance, since it depends on their sensibility and musical feeling in order to build a nice sounding soundtrack.
 |
| Instruction given to participant about Level 3 |
In the first level - a critic to the high level of violence in nowadays games - a red background is the scenery to the players actions, that consist of drawing red lines inside a blank red square. This square is an abstract shape that symbolizes a bullet hole, while the trickling lines resemble the blood coming from the injury. While the point where the drawing starts is defined by the players, generative animations take part of the scene, in the form of branches that randomly come out of the main line. In this way, only part of the image is generated by the players, while other random lines have their own particular behaviors. Each sound is composed by a single frequency that follows the X and Y current position of the end of each line. Other sounds complete this composition: a continuous pattern resembling a heartbeat, and some noisy sounds triggered each time a player presses a button symbolizing a painful cut in the flesh.
 |
| Photo of the first level - Fragment |
Level 2 is a metaphor of a human behavior that uses the degradation of the other as a way of self promotion. Each player controls an abstract shape that symbolizes a hammer that, once pressed, falls over the other participant's avatar, deepening him into the ground. The guest is able to make melodies pressing the joysticks buttons. Each button generates a synthesized sound. The host can play these melodies as well, and his joystick has extra functions used to add some patterns to the soundtrack and also change visual elements. The colors black and white were chosen to make more explicit the contrast between the players.
 |
| Level 2 - Put you down |
In the third part, called 'Capital', each player controls vertical and horizontal position of falling elements. The position where each new element appears is a random variable, and the players don't know where the next one will appear. The sound is composed by some generative elements and the horizontal position of each shape changes the pitch of the two main voices. The end of this part is controlled by the invited player, since it enters into a glitch aesthetic as all the gaps had been filled. As all the screen is filled with the falling shapes, the
instrument enters a glitch mode where non programmable images appear randomly on the screen. Now the players have no more control over their actions. This part is a metaphor for the chaos created by the ultimate level of capitalism. The sound gets messy and everything gets deteriorated until the initial scenery is no more recognizable. This happens because the counting variables enter into an overflow resulting in completely unexpected results in sound and image.
 |
Level 3 - Capital |
The next level - Mimesis - is made up of a black and white graphical background that changes each time the host press a specific button, and two red sprites that represent each player. Players can choose among different shapes in order to look equal (or not) to the background. If the player chooses a shape similar to the background he will almost disappear, whereas a different shape will distinguish him from the background. This programming criticizes the mass behavior of people that prefer to be lost in the crowd instead of assuming their own particularities. This level has a very rhythmic approach and the artist can turn on and off some audio patterns with random elements in their melodies.
The last level is called 'To the Future' as a metaphor for the decadence of culture in today's world. It resembles a continuous backward movement in the world's evolution. All the players can do is postpone their falling and disappearing in the bottom of the screen. The images refer to the idea is a ski diving down a mountain, with trees passing through. Here the position of each player builds the main melody of the soundtrack, added to the sound of background elements passing through the screen. Other patterns are turned on and off by the artist in order to create a dynamic soundtrack. The rhythm is composed by generative elements, with random notes composing the melody and rhythm.
 |
Level 5 - To the Future |
3- Conclusion
The performance 'DOT, a videogame with no winner' is a step further in the creation of live audiovisual performances, where all the elements in software and hardware were created specifically to this theme. A custom instrument was built from scratch and all the sounds and images generated live by the artist and the invited participants. The generative behavior of the whole process makes each presentation unique and the role of each guest fundamental to the complete performance.
Many random variables contribute to this, taking away from the composer the complete control of the performance. Here the participation of the audience really interferes in the final result. And despite all the technological effort put into the construction of the performance, what really matters is transmitting the concept, exposing the artist point of view about the theme.
 |
Live at roBOt Festival (Bologna, Italy - 2011)
|
|
