Overview.

Messa di Voce (Ital., "placing the voice") is an audiovisual performance in which the speech, shouts and songs produced by two abstract vocalists are radically augmented in real-time by custom interactive visualization software. The performance touches on themes of abstract communication, synaesthetic relationships, cartoon language, and writing and scoring systems, within the context of a sophisticated, playful, and virtuosic audiovisual narrative.

Tmema's software transforms every vocal nuance into correspondingly complex, subtly differentiated and highly expressive graphics. These visuals not only depict the singers' voices, but also serve as controls for their acoustic playback. While the voice-generated graphics thus become an instrument which the singers can perform, body-based manipulations of these graphics additionally replay the sounds of the singers' voices — thus creating a cycle of interaction that fully integrates the performers into an ambience consisting of sound, virtual objects and real-time processing.

Messa di Voce lies at an intersection of human and technological performance extremes, melding the unpredictable spontaneity and extended vocal techniques of two master composer-improvisers with the latest in computer vision and speech analysis technologies. Utterly wordless, yet profoundly verbal, Messa di Voce is designed to provoke questions about the meaning and effects of speech sounds, speech acts, and the immersive environment of language.

An installation version of Messa di Voce makes select software from the performance available for public play.

Background.

Messa di Voce, like our previous augmented-reality artworks RE:MARK and Hidden Worlds of Noise and Voice, is concerned with the poetic implications of making the human voice visible. In Messa di Voce, the core technology which makes this possible is a custom software system which integrates real-time computer vision and speech analysis algorithms. Specifically, a computer uses a video camera in order to track the locations of the performers' heads. This computer also analyses the audio signals coming from the performers' microphones. In response, the computer displays various kinds of visualizations on a projection screen behind the performers; these visualizations are synthesized in ways which are tightly coupled to the sounds spoken and sung by the performers. Owing to the head-tracking system, moreover, these visualizations can be projected such that they appear to emerge directly from the performers' mouths. In some of the visualizations, projected graphical elements not only represent vocal sounds visually, but also serve as a playable interactive interface by which the sounds they depict can be re-triggered and manipulated by the performers.

Our group's interest in phonesthesia, or phonetic symbolism, is at the heart of the Messa di Voce project. According to this idea, the sounds of words tend to reflect, to some extent, associated connotations from other perceptual domains such as shape or texture. A classic illustration of the phonesthetic principle can be found in Wolfgang Köhler's pioneering psychology experiment from 1927, in which he asked subjects, "which of the figures below represents the sound maluma, and which one represents the sound takete?" Nearly all viewers respond with the same answer — suggesting rich research opportunities for artmaking and cognitive psychology alike.



Messa di Voce brings together findings from phonesthesia research with our group's shared interests in abstract language, extended vocal techniques, interactive systems, and live audiovisual performance. In our concert, these ideas take shape in a series of twelve brief vignettes which explore different symbolic, tactile and audiovisual aspects of phonesthetic relationships. The entire Messa di Voce performance generally runs 30 to 40 minutes in length.

Press and Reviews.


Messa di Voce has been awarded an Honorable Mention in Interactive Art in the 2004 Prix Ars Electronica competition.

A Logiciel Ouvert: En Autriche, Ars Electronica, baromètre annuel du numérique, marque un retour aux sources. Libération, p.31. Paris, 11 September 2003.

Bersten, Rosanne. Singing a stream of floating images. AustralianIT, 28 October, 2003.

Bliss, Abi. Ultrasound. Leeds Guide, 19 November 2003.

Feilmayr, Ingrid. Faszinierende 'Zaubereien' mit Stimme und Technik. Neues Volksblatt, Vienna, 9 September 2003.

Garulli, Lavinia. Ultrasound. FlashArt News, Fall 2003.

Judmayer, Irene. Magere Stimm-Bilder: Blonk/La Barbara im Brucknerhaus. OÖ Nachrichten, Graz, 9 September 2003.

Levin, Golan and Lieberman, Zachary. “In-Situ Speech Visualization in Real-Time Interactive Installation and Performance.” Proceedings of The 3rd International Symposium on Non-Photorealistic Animation and Rendering, June 7-9 2004, Annecy, France.

Messa di Voce. Press release, Ars Electronica Festival, Linz, Austria, 20 June 2003.

Noonan, Mickey. Q&A: Messa di Voce. Huddersfield Metro, 28 November 2003, p. 18.

Richards, Kate. Ars Electronica: Interface Futures. Realtime Arts #58, September 2003.

Somajni, Chiara. Dipingere con la voce. Il Sole 24 Ore, 14 September 2003.

Turco, Marina. Tutti nella rete digitale Ars Electronica a Linz, tante performance e un solo 'code'. Il Manifesto, 13 September 2003.

Artist Biographies.

Tmema is Golan Levin ( golan [at] flong [dot] com )
and Zachary Lieberman ( zlieb [at] parsons [dot] edu ).

	Golan Levin [CV] is an artist, engineer, and composer interested in developing artifacts and events which explore supple new modes of interactive expression. His work focuses on the design of systems for the creation, manipulation and performance of simultaneous image and sound, as part of a more general inquiry into non-verbal communications protocols in cybernetic systems. Levin is Assistant Professor of Electronic Time-Based Art at Carnegie Mellon University, Pittsburgh.
	Zach Lieberman [CV] is an artist, engineer, and educator whose work is focused on exploring the creative and human uses of technology. He produces installations, on-line works and concerts concerned with the themes of kinetic and gestural performance, interactive imaging and sound synthesis. Lieberman lives and works in New York City, where he teaches courses in audio-visual synthesis and creative image processing at Parsons School of Design.
	Jaap Blonk [CV] is a composer, voice performer and sound poet. In reciting poetry, especially the works of Antonin Artaud and Kurt Schwitters, he discovered the directness and flexibility of vocal utterance, whether using meaningful words or not. At present, he has developed into a prolific writer/composer and a specialist in the performance of sound poetry, supported by a powerful stage presence and an almost childlike freedom in improvisation.
	Joan La Barbara [CV] has been called one of the great vocal virtuosas of our time. A pioneer in the field of contemporary and sound art, she has developed a unique vocabulary of experimental and extended vocal techniques, including multiphonics, circular singing, ululation and glottal clicks. In addition to her own compositions, she has premiered landmark compositions written for her by noted American composers, including Robert Ashley, John Cage, Philip Glass, Steve Reich, Morton Subotnick and James Tenney.

Technical Requirements.

This section links to technical information about the staging requirements of the Messa di Voce performance and installation, including wiring diagrams, approximate breakdowns of fees and expenses, and necessary rental equipment.

Venues seeking further technical information about the Messa di Voce performance are encouraged to check this page:
TECHNICAL REQUIREMENTS: PERFORMANCE

Venues seeking further technical information about the Messa di Voce installation are encouraged to check this page:
TECHNICAL REQUIREMENTS: INSTALLATION

Contact & Inquiries.

The Messa di Voce performance and/or installation are available for touring dates in 2004 and 2005. Inquiries should be made at least four months prior to the anticipated date of presentation. Please direct inquiries to:

A print-ready, single-page description of Messa di Voce is available here.

A 20-page (5Mb PDF) report describing Messa di Voce, suitable for printing, is available here.

Links and References.

Several related projects form important conceptual and aesthetic precursors or sequels to this one. These include:

Michaela Pavlátová, "Reci Reci Reci" ("Words Words Words") [1991]. A delightful hand-drawn animation in which the conversations of café patrons are abstracted and visualized.

Grady Klein, "Afterbabble" [2002]. An animation and children's book which speculates about what happens to our words after we speak them.

Golan Levin and Zachary Lieberman, "RE:MARK" and "Hidden Worlds of Noise and Voice" [2002]. Our own prior work, a pair of interactive installations in which participants are able to "see" their voices.

Toshio Iwai and Ryuichi Sakamoto, "Music Plays Images X Images Play Music" [1997]. A masterful audiovisual piano/graphics performance, which explores a wide variety of possible relationships between physical action and virtual representation.

Toshio Iwai and Mayumi Miyata, "Sho of Light" [2003]. A real-time audiovisual performance which augments Ms. Miyata, a virtuosa performer of Japan's traditional wind instrument, with synthetic graphics.

Steven Blyth and Matty Salin, "CHIT CHAT" [2005]. A charming installation which projects a speech bubble above your head as you sit and talk in a café.

"Messa di Voce" is also a Dutch vocal ensemble which specializes in performing the choral music of the Middle Ages, Renaissance, and Baroque eras.

All texts, sounds, images and videos contained herein are © 2003-2005 Tmema/Blonk/La Barbara, and may not be reproduced without permission. Last updated: 11 March 2005.

Site keywords: Messa di Voce, Golan Levin, Zachary Lieberman, Tmema, Jaap Blonk, Joan La Barbara, Ars Electronica, new media performance, interactive performance, audiovisual performance, audiovisual concert, audiovisual substance, augmented reality, projection, speech visualization, sound visualization, music visualization, seeing speech, sound-image relationships, speech recognition, voice analysis, speech analysis, computer vision, head tracking, pitch tracking, video tracking, motion tracking, body tracking, computer graphics, interactive media, interactive art, software art, electronic art, generative art, sound art, performance media, extended vocal techniques, vocal improvisation, abstract speech, nonsense language, sound poetry, abstract animation, dynamic abstraction, real-time animation, live cinema, synesthesia, synaesthesia, phonesthesia, phonaesthesia, voice and computer, voice and electronics, voice interface, speech interface, whole-body interaction, shadow play, Flong, Joan Labarbara, Zach Lieberman, Systemis, Greg Shakar.