Beyond Speech

Coordinator -  Rob Mackay

 Participants – Speech Synthesis Conversion and Original Concept: Dr Robert Mackay

Speech Synthesis Engine: Prof Roger Moore

Interactive audio-visual design: Sam Eaton

Outline -

'Poem' is an interactive audiovisual installation that takes a creative look at speech synthesis and how we interact with, and interpret verbal communication. It uses interactive technology involving the Microsoft Kinect to allow users to control synthesised performances of poetry. Users are able to manipulate the vocals into something ethereal and unrecognisable, removing meaning and understanding from the voice. The voice becomes something more akin to an abstract instrument than a method of articulate communication. Controls are mapped to limbs upon your body, moving control of the voice from its natural internal spot to your external appendages. This disconnect creates a bewildering experience in which users attempt to marry creative exploration with legibility. They attempt to manipulate the sound presented to them into something understandable, something resembling human speech as snippets of the various poems intertwine and overlap.

The installation explores what communication is in its various forms, from literal storytelling to the emotive expression of abstract composition. It aims to allow users to explore how meaning is effected by context, how the users presence has repercussions and asks them to consider if a story can be told without relying on speech.

Visual feedback for the installation is provided in the form of interactive 3D landscapes which grow and change based upon input. This interface provides both feedback and tutelage, guiding the user through the controls they have available to them while fostering a sense of exploration.