I USED TO BE A TABLE- a multimedia performance with machine spoken word and robotic screens by Soundspiels

The English word secretary originally was a piece of furniture for writing and hiding sensitive documents, using the idea of the secret as a trigger we imagine what  Siri, Alexa, Cortana … all models of assistance dream of, speak of when unasked for support.

Inspired by the theories of  Kate Crawford about the entangled past of digital and physical secretaries we are developing our research into machine learning, using algorithms and data sets to produce text and voice and our working on a Croatian voice model using Nicole Hewitt’s voice to train the machine. The texts and sounds are generated using multiple processing methods, online translations, wordnet data bases, google translate and text to speech programmes, the texts are then processed as synthesised material and further processes to produce sound.  All the voices in the performance are sythesised, the images are a combination of found footage and machine learning image processing, screened on motorised and rotating screens, as well as text displays.

The project works with analogue and digital processes including dreams and the unconscious, and trying to imagine what artificial intelligence might dream of. Taking as its starting point the idea of the listening woman, the secretary both human and digital, the assistant, the digital assistant we try to give voices to the women who gave their voices to text to speech technologies. We encounter them and address them by their names, Goga, we also create our own voice models through training sets for text to speech. We produce narratives around their possible interactions.

The performance is based on blind composition, so we never know exactly what files are playing, what sounds are being created and what texts are being voiced. The sinsets we use are based on the most simple of differentiation and often result in ambiguous, frequently racist and sexist values. We work with a sentient lexicon that attaches simple positives and negatives to nouns. Newer algorithms are trying to not only produce polarised sentiments i.e. ‘panic – minus’, ‘ achieve- plus’ but are working to develop a prediction of sentiment intensity.

 In deep learning Wave net examples we can hear voice models producing sounds without making language, sounding as if in a trance of their own. Depending on how a model has been trained, the model will predict the correct enunciation, tone and pitch which allows us to train a model in the future on non-linguistic voicings for example crying, moaning, laughing, mumbling. In our visuals we therefore also work with GANs, and machine learning image processing techniques as well as found footage to present ecstatic women, women unconscious, or delirious, or those who hear and see and predict, for example Joan of Arc. 

What does ecstasy sound like in artificial intelligence, what does a machine hallucinate?