Motion reactive audiovisual installation and performance
2020
S.S.S.S. (or Synthesized Synesthesia Stimulated Simulation) is a motion reactive audiovisual installation with the accompanying performance are we still going to the movies?. While S.S.S.S. refers to the system and the organization of the installation, are we still going to the movies? is the specific performance related to the particular video input fed to the installation patch. This performance is meant to showcase a particular way of interacting with the installation.
Max / MSP patch of the installation is inspired by and takes foundational parts from the builds of Sam Tarakajian, David Tinapple, and Gregory Taylor. The webcam input is used to capture subsequent frame differences and therefore work as a simple motion detector. This input is vertically split into four sections and the “amount” of motion happening in each section is digitized through calculating the amount of changed pixels frame to frame. Each section’s motion input controls several parameters within the patch.
Audio output is based on generating, sequencing, and modulating waveforms in Max / MSP. The motion input is used to change the signal frequencies and volumes. Certain sounds are mapped to certain sections of the webcam input, therefore each quarter yields distinct results. These distinctions can be easily recognized by the user once they interact with the installation and can be manipulated to get relatively intentional sound compositions. As for the visuals; the same motion data is used to alter the color values and the temporal aspects of the video input. Much like the sound input, each section is mapped to a different quality of the video. Thus, it is possible to interact with the installation through the visual or auditory observation alone, or to interact with both interchangeably. If a single output is taken into consideration, this allows the other output to be produced accordingly as a discovery.
For the performance are we still going to the movies?, archival public domain footage is used to piece together the specific thematic narrative. The performance itself is based on gendered anxieties and feminine cynicism. However, the installation is open-ended in that it can be modified and developed further in different directions based on the visuals that may be fed into the patch (similarly; the base signals and the note sequences within the patch can also be modified to produce extensive variations of the installation).
