Music [Sticky] Notes
Music [Sticky] Notes was an environmental augmented reality exhibit that turned sticky notes into musical notes.
Built in TouchDesigner, the height of the sticky note on the wall determined the height of the musical note played. Each color played a different instrument, allowing participants to mix sounds or have multiple melodies overlap visually but played individually.
Having a physical item for participants to touch and place to make a visual outcome and create a digital melody combined three senses to expand an interpretation of musical notes on a scale.
This piece was exhibited at the Toronto Kids Digital Festival in March 2019, running for three days, and later at the 2020 Inclusive Science Day hosted by the Ohio Valley Museum of Discovery.
Families interacted with the piece in many ways: they created songs, experimented with the technology, learned motor skills, left messages, and produced visual designs. The interface made the project easy to calibrate and allowed participants to select which instruments they heard as they watched a "scanner line" read an image of the wall.
In addition to the main goals, participants explored how the technology works, leaving them with a sense of accomplishment and a desire to explore augmented reality.
Photo Set 1:
1. Early test of the TouchDesigner code. The song shown is the beginning of "Head Over Heals" by Tears for Fears.
2. Testing Music [Sticky] Notes in the GRID Lab.
3. First setup for the Toronto Kids Digital Festival. I added example notes for demonstration.
4. Rough-style foam "piano keys" provided a guide for participants.
5. I added labels to the inside of the buckets since that is the most visible surface from a standing position.
Photo Set 2:
1. Example of the interface and how it sees the wall.
2. The song, "Twinkle, Twinkle, Little Star" is shown in pink sticky notes.
3. Toddlers had fun bending, tearing apart, and sticking sticky notes to the wall.
4. Families worked together to create music.
5. This group worked together using the internet to create the "Happy Birthday" song.
Photo Set 3:
1. Friends worked together on the wall. Participants were able to watch in real time from the interface.
2. Siblings had fun creating musical murals and messages.
3. This smiley mural sounded interesting!
4. These participants created "Twinkle, Twinkle, Little Star" in pink sticky notes.
5. Tests and experiments by participants.
Photo Set 4:
1. This participant created the beginning of "My Heart Will Go On" by Celine Dion.
2. STEMKid hosts create "Sakura," a traditional Japanese folk song.
3. More experimentation by participants.
4. "Hi hi," says a participant.'
5. This participant experimented by having different songs played together.
Photo Set 5:
1. Toddlers had fun bending, tearing apart, and sticking sticky notes to the wall.
2. The wall after the toddlers had lots of fun.
3. More than one group was able to build songs at the same time.
4. Example of what the user interface looked like with a completed wall.
Technology & mediums used:
Derivative TouchDesigner
Python
Webcam
Sticky Notes
Buckets
Special Thanks:
Katie Micak
Jane Yang
