The color of music
Developed by Daniel Ortiz and Raúl Chávez
From immemorial times music has been a very important form of expression for humans and accompanies us as part of our culture
and traditions. Music is present in every moment of our life, however, it is an invisible company as only we can perceive with our ears and is imperceptible to the rest of the senses.
What if that was different? What if we could see it? Or touch? These are important questions for this experiment which aims to visualize something that is invisible to our eyes and thus discover the hidden patterns within the melodies that have the ability to give us goosebumps or make us feel a deep sadness.
Chromesthesia is a type of synesthesia in which, when certain sounds are heard the brain automatically makes the association with color and this is shown through the view. These visualizations are based on that concept.
The beginning
To develop these visualizations we decided to analyze songs they had a continuous main melody and they were easily identifiable.
Given these characteristics defined we decided to compare the "4 Seasons" by Antonio Vivaldi, taking into account that is a work of broad popular knowledge. In addition they were selected for its diversity of "moods" so we can compere the different stations and answer the questions proposed at the beginning.
Thus, these visualizations follow the movements of the concerts for violin and orchestra belonging to "Le quattro stagioni":
- Allegro, Mov. No. 1 del concierto No. 1 en E, Op. 8, RV269 "La primavera".
- Presto, Mov. No. 3 del concierto No. 2 en Gm, Op. 8, RV315 "L'estate".
- Allegro, Mov. No. 1 del concierto No. 3 en F, Op. 8, RV293 "L'autunno".
- Allegro non molto, Mov. No. 1 del concierto No. 4 en Fm, Op. 8, RV297 "L'inverno".
To develop these visualizations we decided to analyze songs they had a continuous main melody and they were easily identifiable.
Given these characteristics defined we decided to compare the "4 Seasons" by Antonio Vivaldi, taking into account that is a work of broad popular knowledge. In addition they were selected for its diversity of "moods" so we can compere the different stations and answer the questions proposed at the beginning.
Thus, these visualizations follow the movements of the concerts for violin and orchestra belonging to "Le quattro stagioni":
- Allegro, Mov. No. 1 del concierto No. 1 en E, Op. 8, RV269 "La primavera".
- Presto, Mov. No. 3 del concierto No. 2 en Gm, Op. 8, RV315 "L'estate".
- Allegro, Mov. No. 1 del concierto No. 3 en F, Op. 8, RV293 "L'autunno".
- Allegro non molto, Mov. No. 1 del concierto No. 4 en Fm, Op. 8, RV297 "L'inverno".
Coding
The process of transforming a complex language as music to data is actually a fairly simple task once the sound produced by an instrument is broken down into its most basic qualities.
This experiment considered as parameters the pitch, which refers to the specific frequency of each sound which allows us to identify musical notes. And the duration time that represents the vibration of the note expressed in fractions beat.
Then, each one of the notes with the corresponding color code would be represented as:
The process of transforming a complex language as music to data is actually a fairly simple task once the sound produced by an instrument is broken down into its most basic qualities.
This experiment considered as parameters the pitch, which refers to the specific frequency of each sound which allows us to identify musical notes. And the duration time that represents the vibration of the note expressed in fractions beat.
Then, each one of the notes with the corresponding color code would be represented as:
With the data structure built three different visualizations are built in order to generate the connection between the senses.
CHROMATIC FLOW
It represents the color fluctuation over time and helps us to discover patterns in the sequence of notes that may be associated with
what we perceive the "mood" of a song.
The linear interpretation of the movements of Vivaldi, allows us to obtain a linear display of the notes according to their temporal structure. In these visualizations we can see patterns of each "movement" of Vivaldi.
It represents the color fluctuation over time and helps us to discover patterns in the sequence of notes that may be associated with
what we perceive the "mood" of a song.
The linear interpretation of the movements of Vivaldi, allows us to obtain a linear display of the notes according to their temporal structure. In these visualizations we can see patterns of each "movement" of Vivaldi.
For example, SPRING, is a movement with many changes of colors and shades in a very short time. You can find patterns of change
in very short periods. On the contrary, we see that in WINTER, the changes are not so sudden and so close together, but there is a
pattern of change more spacious.
SATURATION TONE
The second visualization, helps us discover the range of notes that make up a song and allows us to build the "skeleton" on which the physical image of each song is built.
The "x" axis represents the eighth, or "height" of the note, with 0 being the lowest value and 8 the highest value which represents how low or acute is each note. The axis "y" represents the pitch, or the "name" of each note, and the range is Do (C) to Si (B), including the silences (S). Finally the diameter of the circles corresponds to saturation (number of repetitions) of each note allowing see which are the "strongest" notes in each of the pieces.
In the saturation tone we can see the mapping of the notes according to the number of repetitions in each movement. We mapped the number of times each note is repeated at different scales and we can see a "map" of the song. It depends which note and scale is repeated more times so we can see bigger "spots" on the map and see how it forms a "DNA" of the movement.
The second visualization, helps us discover the range of notes that make up a song and allows us to build the "skeleton" on which the physical image of each song is built.
The "x" axis represents the eighth, or "height" of the note, with 0 being the lowest value and 8 the highest value which represents how low or acute is each note. The axis "y" represents the pitch, or the "name" of each note, and the range is Do (C) to Si (B), including the silences (S). Finally the diameter of the circles corresponds to saturation (number of repetitions) of each note allowing see which are the "strongest" notes in each of the pieces.
In the saturation tone we can see the mapping of the notes according to the number of repetitions in each movement. We mapped the number of times each note is repeated at different scales and we can see a "map" of the song. It depends which note and scale is repeated more times so we can see bigger "spots" on the map and see how it forms a "DNA" of the movement.
CHROMATIC PRINT
Through an algorithm based on the number of times that each of the notes is repeated we generate a "fingerprint" which is unique to each song and represents the physical form of that song.
How does it work?
We start by counting the number of times the note is repeated in each "movement". The note that is repeated more times gets in the center and we start putting this note around in spiral in clockwise direction. When we turn to the next note, which would be the second most repeated note in the movement, we change the color of the new note and the direction of arrangement change and changes to counter-clockwise.
Through an algorithm based on the number of times that each of the notes is repeated we generate a "fingerprint" which is unique to each song and represents the physical form of that song.
How does it work?
We start by counting the number of times the note is repeated in each "movement". The note that is repeated more times gets in the center and we start putting this note around in spiral in clockwise direction. When we turn to the next note, which would be the second most repeated note in the movement, we change the color of the new note and the direction of arrangement change and changes to counter-clockwise.
In this way, we transform a collection of sounds in a visible creature to our eyes. This way we can imagine the shape of each song, as we have a variable color (notes) and variable direction (every time we change a note) that makes a particular and unique form of each movement.
The color of music
The sound made colors. It is is the premise of this visualization which assign sounds and colors through 3 visualizations we achive
to see the 4 different movements of Vivaldi. Comparing the colors, shapes, figures is a subjective exercise where each person does
their interpretation, since the senses of hearing and sight intermingle to create new sensations and interpretations.
Music generates feelings and awakens the senses. Usually music is associated with the sense of hearing. But what happens when
sound is transform into something visual? The idea of the color of music is to transform sound into visual patterns that can be
interpreted in shapes and colors. We conduct research on synesthesia that is a condition where people assign a color and shape
of letters and numbers. (http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4349591/)
In addition to synaesthesia condition, exists chromastesia where a color is assigned to the music and sounds heard. A person with
chromastesia makes a connection between a color (hue, saturation and value) and sound (pitch, amplitude and note). There are
several characters in the story that made their own representations chromastesia (http://rhythmiclight.com/archives/ideas/colorscales.html).
The sound made colors. It is is the premise of this visualization which assign sounds and colors through 3 visualizations we achive
to see the 4 different movements of Vivaldi. Comparing the colors, shapes, figures is a subjective exercise where each person does
their interpretation, since the senses of hearing and sight intermingle to create new sensations and interpretations.
Music generates feelings and awakens the senses. Usually music is associated with the sense of hearing. But what happens when
sound is transform into something visual? The idea of the color of music is to transform sound into visual patterns that can be
interpreted in shapes and colors. We conduct research on synesthesia that is a condition where people assign a color and shape
of letters and numbers. (http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4349591/)
In addition to synaesthesia condition, exists chromastesia where a color is assigned to the music and sounds heard. A person with
chromastesia makes a connection between a color (hue, saturation and value) and sound (pitch, amplitude and note). There are
several characters in the story that made their own representations chromastesia (http://rhythmiclight.com/archives/ideas/colorscales.html).
The representation of the colors and the sound is subjective and personal. To our representation we consider the color value that is
repeated more times and assign it to the notes. According to the eighth of note the color changes in hue, the higher the note is the
brighter the color is and the lower the eighth is the duller color is.
Developed by Daniel Ortiz and Raúl Chávez
