 |
Fig 1. The Book of Stamps Installation |
I
About
The « Book of Stamps » by the sounds artist Art Clay is a travel guide between sonic landscapes from cities to urban cultures. The sheets of the book provide a "recording surface" and the ink stamps with their various patterns provide the ability to place sounds into the book. Together they act as an interactive tangible interface for a variety of time based musical tasks that form a collaborative composition by its users. There are two sets of ink stamps: The stamps that look like natural things like trees, bushes or stone paths belong to the "Country Sounds" category; Those that look like buildings belong to the "City Sounds" category. By stamping a book page with a combination from both categories, a soundscape is created that will either tend to sound like a city, a country or an urban sonic mix of both. In this manner, sonic spaces are created for each of the pages and when the user turns the pages to other already stamped pages, it lends him or her the impression that they are actually "traveling" between places sonically.
To interact with the Book of Stamps, the visitor is asked to select one of the stamps from either of the two collections; ink it well on the pad by pressing it onto the inkpad several times. After inking the stamp, select an empty square on the page of the book and place the stamp on that square. Press the stamp down with much pressure and then return the stamp to where the other stamps are. Once the stamp is in place, you should hear a change in the sonic landscape. The more the page has been stamped the harder it might be to hear the sound you have placed into the mix by stamping the page.
How it Works:
Behind the artwork of the Book of Stamps lurks a computer vision application in the form of a visual markers recognition system. The distinctive feature of the application is that the markers can be visually designed to convey meaning to people and not only to machines. By allowing the creation of markers that support interaction both visually and functionally, one can enhance most applications normally supported by visual markers such as interactive guides, mobile service access and mobile game etc.
|
Fig 2. A "minimal" valid marker. The design contains one black region (outer multi-circle design) that contains three white regions. Two of the white regions contain one black region and one contains none. |
With this type of computer vision application, the markers are "seen" by the computer through a web-cam using a fiducial recognition algorithm. Each of the markers is labelled with a specially designed black and white symbol that is easily recognized by the computer. The algorithm used for the recognition is quite flexible in reading shapes, as long as a number of rules are respected. The essential rule to designing a valid visual marker is in the „nesting" of black and white areas or regions. A marker can be composed of one black region containing three or more white regions, and at least half of these white regions must contain one or more black regions. There must be no more than three and no less than three levels of nesting to have a valid marker, however, there is no limit in the number and shapes of the regions.
II
The Stamp Collection
The stamps divide into two collections, the "Country Collection" and the „City" Collections". Each collection flows design-wise from one stamp to the other by incorporating common elements into the design of each of the stamps within a collection. Further, the stamps within a collection can be used to combine into larger patterns (diverse garden and building types), which are recognizable by the application and make good visual sense to the viewer.
|
Fig 3. The "Country Collection" of stamps. The stamps combine into patterns such as "forests", "grasslands" and "walkways". |
The stamps in the City Collection are designed to combine into patterns that are reminiscent of an architectural plan and by using the stamps the visitor can „construct" building types in various forms and sizes. The stamps in the Country Collection modulate from "large tree" down to "flowered walkway" by incorporating common elements into the design of each. The controller collection stamps are designed to influence how the other stamps are being read by the computer. Their inclusion in the book directly changes the texture of the soundscape being played by the computer by changing the scanning parameters of the computer vision application.
|
Fig 4. The "City Collection" The stamps combine into patterns " reminiscent of architectural plans for the various types of buildings.
|
|
Fig 5. Controller stamps. These stamps are designed to influence how the other stamps are being read by the computer. Their inclusion in the book directly changes the texture of the soundscape being played by the computer.
|
IV
The Score Page
The Book of Stamps itself is a large spiral bound volume with many gridded pages in it. The book lies flat on within a Plexiglas frame. Each of the pages of it can be filled by the user simply by stamping the patterns into it and each page of the book acts as a parts of a type of "music score". The information contained in the stamped symbols is decoded by the computer visions application and then sent as coded instruction to a sound application, with which the actual soundscape is generated based on the type and number of symbols that were stamped onto the page.
To begin making a soundscape or to add to one that has already been started, the user chooses any one of the provided stamps from either of the collections and after inking it stamps into any one of the gridded squares not yet occupied. The system recognizes each of the stamps on the stamped page and will play the contents of the pages it back in real time so that any change made to the page is immediately heard in the soundscape produced by the application.
|
Fig. 6. A single page of the Book of Stamps. It acts as a score that sends instruction to computer application that generates a soundscape based on the type and number of symbols that were stamped onto the page.
|
As time goes passes and visitors come and go from the exhibition, the pages of the book begin to fill up with stamp patterns and the sheets of the book begin to take on the character of a multi-paged music score. Interested in what others have created on other pages of the book, the user can simply turn from one page to the next and hear the entire book’s contents and by doing so "travel" from one sonic landscape into the next with simply a turn of the page.
V
Creating the Soundscape
How the sound application actually generates a soundscape from the patterns that are on the page is a more involved procedure involving an algorithmic process. As can be imagined, there are two categories of sounds: „City sounds" & „Country Sounds" and for each of these categories a collection (a bank) of sounds has been provided for. The sound bank for each category contains eighteen sound samples which are grouped into threes. So, for each stamp, there are three sounds that are designated to it and where any one of the sounds could play back when the stamp is recognized by the computer. All of the sounds with in a group of three are related and are similar in terms of sonic content. As an example, for the stamp "walkway" from the Country Collection contains five circular stone like patterns. There are three different sound samples created for this stamp that because of their sonic content -the sounds of foot steps at various speeds and timbres. It is only in this way that it is possible to generate a convincing soundscape that resembles in manner one found in nature.
|
