Immersive

.join(): An interactive installation confronting technological failure, surveillance and a fantasy of modernity.

Custom speech recognition software, sensor-embedded sculpture, and spatial design.

.join() is a multimedia installation in Wesleyan’s Zilkha gallery that employs custom software, sensor-based sculpture, and projection to promote interaction between users and speech recognition tools.

The intention of this project is to invite the user to consider the unexpected ways in which digital technologies fail and reveal subjectivity and uncertainty within technological systems we take for granted as being neutral and logical.

The fantasy of modernity and black boxes is present in computation, and .join() calls this into question by displaying the unexpected failures of cloud-based speech recognition and commercially used API databases for word interpretation, by exposing the tools of production, and by employing techniques of light and color synthesis.

The installation includes three Mac Minis, Arduinos, microphones, projectors, laser-engraved acrylic, aluminum, and cellophane.

The software is written in Javascript and hosted on three web pages in order to connect to the browser’s cloud-based speech recognition tool, to retrieve synonyms from the WordsAPI server, and to generate responsive animation using the P5.JS graphics library.

Referencing the fundamental workings of screens and pixels, natural light is filtered through red, green, and blue cellophane, casting colored shadows into the space. Live-coded graphics synthesize color using Hydra, a networked web-based video synthesizer.