Posts Tagged: sonification

Hue Music

The paper accompanying this page is available here: Hue Music

Associating Hue with Timbre

Initially 8 different hues were associated with a recorded timbre each. The images and audio clips below show this. Click on the colour names to hear the associated timbre.

White – No Sound

Mixing Different Timbres Together

A virtual audio mixer is then used to combine the sounds dependent on the quantity of each hue in the analysed image. The simple split blue-yellow image below creates a histogram of values showing equal quantities of blue and yellow pixels. This information is used to drive an 8 channel stereo audio mixer. This particular picture produces a sound that has equal amplitudes of the blue and yellow associated timbres.

Blue Yellow Matrix

Blue Yellow Histogram

Quantising Hue Values from Real Pictures

If a regular picture is used with non-standard hues then the image has to be quantised to a series of closest value hues. This process has 2 stages, first splitting the image into a 8x8 matrix of smaller submatrices. Then each submatrix is analysed to find the average hue value which is then quantised to one of the 8 chosen hue values.

Original Picture

8×8 Image Matrix

Matrix of Quantised Hue Values

The complete mixed audio result is given here.

Probing the Picture

The program also allows a probing scan to occur which allows a user to zoom into an area of the picture. This emphasises different sounds depending on the focus of the probe. The audio file above was recorded while probing into several different areas of the image. Most notably the timbre for cyan can be heard clearly around 12 seconds into the music.

More Music

To demonstrate the process further some more 'rendered' compositions are presented below.

Wedding – This picture produces a similar soundtrack as the aquarium one due to the similar colour content

Bark – The bark quantises to produce a lot of white space. The resulting audio is therefore quieter

Flowers – In this picture green and yellow dominate and can be clearly heard in the music