Algorithmic Chromo-synesthesia

Throughout history, composers (and many others) have described a phenomenon in which musical notes manifest as colors in the mind’s eye. Consider this chart by Fred Callopy showing three centuries’ of recorded synesthesia:

While I don’t personally experience synesthesia, I am a very visual thinker, and in self-teaching guitar I wondered at the potential of this system for improving learning. By studying the commonalities in the above chart, I created this system for color association:

I then painted the fretboard of my guitar according to these colors, which produced a beautiful artifact:

Then, I started mapping the colors to sheet music. Here are some of the results:

“Frère Jacques”, where red = C, yellow = E, blue = G, etc. and length of rectangle indicates length of note, such that a square is a quarter note:

Below, “Ode to Joy” introduced problems of identifying adjacent octaves; the low G in row 3 is differentiated from the higher G in other rows by a decrease in brightness, such that hue still corresponds to note but brightness corresponds to octave:

Unlike “Frère Jacques” and “Ode to Joy”, which are very bright rainbows, the color palette generated by sheet music for “Fur Elise” is decidedly unsettling, much like the tune itself:

Note the very light yellow note in the 5th line; this is the very high E, as higher octaves are identified by increased brightness in the same way that lower octaves (such as the mustard-colored low Es preceding it) are darker.

Future plans for this project include building a program that can read sheet music and automatically generate graphics like the ones above (which were manually generated), the ability to translate guitar tabs, using Processing and a microphone to generate synesthetic paintings in real time, experiments into chords and harmonic ratios (possibly building a harmonograph), and finishing painting my electric keyboard to match!

Leave a Reply

Your email address will not be published. Required fields are marked *