Using text to embellish speech: a response to some of the limitations of using sound to convey affective information

We discussed briefly in class on Monday the limited access that people with hearing impairments have in gathering affective information from auditory stimuli. Moreover, people like me who appreciate and thrive on visual information may find pure auditory presentations of information lacking. It struck me on my way home from class that night that I had previously experimented with a low-tech version of universal design before I knew what universal design was.

Art has always been a hobby of mine, and a few years ago I dabbled in (re)creating fonts by hand using pens, markers and paint. I enjoyed the fact that I could take spoken words and attach additional information on the delivery and emotion behind the text through stylistic representation.

The following is an example of what I’m talking about. The song is Without Mythologies by my favorite band The Weakerthans. Permit me to now take some time to do a “What Makes it Great” discussion on this particular song. First of all I appreciate The Weakerthans because of their masterful use of the English language with appropriate accompanying instrumentals that always add to the quality of the lyrics.

• The word choice in Without Mythologies is so vivid, exact and well chosen that even a visual-dependent person like me can easily transfer the spoken words into detailed imagery.
  


• Also, the music behind the lyrics is simplistic to the point that it only offers only occasional prominence while continually displaying the lyrics.
  


• Lastly, the singer (and songwriter) John K. Samson uses cadence, delicate emphasis and pauses to accentuate poignancy and emotion.
  

Of the three qualities that I mention here only the words themselves are accessible to all people. Therefore, you are about to see how I took the text and transformed it to imitate Mr. Samson’s vocal styling and the mood of the music itself through color, fonts, direction and spacing.

This particular piece is oil on canvas, luckily unstretched, so I was able to put it on my scanner in various positions. I won’t give you too much information about how I interpreted the sounds into text because I think you can do that on your own. Lastly, go easy on me, I’m only a hobbyist; I just did this for fun a few years ago (and there are some typos).

You have two options to view the art and music pairing. First, using Windows Movie Maker I made a movie that changes the images with the music. But, since it is small, it's hard to read so I've included the pictures of the painting that you can pair with the song that is on the playlist at the top right of the blog. You may want to play the song on the playlist first without the visuals. Just so you know, the playlist version is live and slightly different.

Let me know what you think!