Shazam: How does it works?

With the iPhone, phone or smartphone application Shazam, you can identify music tracks played on radio, TV, CD PLayer, Movies, etc..

Where ever you are, at home, in a store, in a movie theater or in your car hold your iPhone next to the speakers and in 10 secs Shazam identifies the song (Song and Album titles, Artist name), shows the cover and links you to the iTunes store or YouTube to watch the video.

Last night, I was at my friends Pascal and Christian for a wonderful diner. We were 9 around the table when I was ask to show how Shazam works on my iPhone. After being impressed everybody was wondering how it works. I started to explain.......

After listening 10secs Shazam transmits over the Internet the audio in real time to a large call center type place overseas, where there are a bunch of workers who are very familiar with music and can recognize 90% of all songs instantly. If one person doesn't know the song, it rolls over to one of his colleagues, who will know it. Then in no time the informations are send back to my iPhone...

No one believed me.

Last night, I was told that the music played always send a meta file (?) with all the informations. Since my friend was not able to explain what he meant, I search today to try find the principal of this application.

The recognition technology used by Shazam is owned by Landmark Digital. It is a patented algorithm (fingerprinting technology) that recognizes practically any audio that is within the recognition database, very quickly and with a very small amount of audio.

An algorithm is a procedure or formula for solving a problem. The algorithm is able to recognize a short audio sample of music that had been broadcast, mixed with heavy ambient noise, subject to reverb and other processing, captured by a little iPhone or cellphone microphone.

The algorithm also have to perform the recognition quickly over a large database. The audio can be subject to voice codec compression, and network dropouts, all before arriving at the database servers.

The application listen, analyses, recognizes and then recalls it. Like our brain the technology is specialized in a complex pattern recognition.

Each audio file is "fingerprinted," a process in which the needed audio informations are extracted. A time-frequency analysis is performed, marking the spectrogram, to reduce the audio file down to a relatively sparse set of time-frequency pairs. To perform the search the fingerprinting (empreinte digitale) is sent to the database to find a match of the time-frequency pairs.

What makes Shazan works, is its ever-expanding database. It store more audio information than any human brain ever will.

An other application made that possible: midomi.

midomi is a smartphone and iPhone apps and also a web application ( http://www.midomi.com) which make possible to find your favorite music by simply singing, humming or whistling it.

If you want to try it, simply go to the midoni site to sing or hum for 10 seconds and find who sing the song you want.

These two apps are truly amazing.

I am officially impressed! Are you?