Musical Vision

Musical Vision is a bio-inspired system for converting images into polyphonic music. Initially, its clearest potential of use consists in allowing people suffering from visual impairments to “see” through music. However, Musical Vision is an extremely flexible system, meaning that it can be configured to convert an image into either a few seconds or a several minutes long musical piece. Thus, it can also be used with artistic purposes.

Musical Vision converts pixels into notes by mimicking the human visual system in terms of its field of vision and photosensitive sensors. Thus, image contents located in the central region are given more importance than that appearing in the peripheral regions. Moreover, each pixel is transformed into three notes (one per color channel), the intensity of which depends on the pixel’s luminance. Furthermore, the color space is mapped onto a fully configurable instrument space.

So as to generate pleasant polyphonic music, Musical Vision incorporates a harmonizer, which modifies the notes corresponding to the pixels to build melodies using harmonic chords.


The soundtrack music of the video above corresponds to the translation into music of the following image made by Musical Vision:


Musical Vision was developed as part of the MSc thesis of student Antonio Polo, supervised by Xavier Sevillano.