Math Equation Predicts Musical Reactions

[MarkB]

From Discovery News:


Math Equation Predicts Musical Reactions
By Anna Salleh, ABC Science Online
Font Size: [small] [medium] [large]
May 25, 2004 —The power of music to convey a certain emotion can now be predicted by a mathematical model, an Australian psychologist has found.

Emery Schubert of the University of New South Wales will report the first study of its kind to mathematically quantify the emotional impact of music at the International Conference on Auditory Display in Sydney in July.

Schubert has developed a formula that relates features such as the music's loudness, tempo and pitch to an emotion perceived by the listener.

"With my formulas I can play a piece of music and predict what emotional impact it will have," Schubert told ABC Science Online.

He found that the loudness of a piece of music was the most powerful predictor of whether the music was arousing. Next in line was tempo: the faster the music, the more arousing the music.

He also found some evidence that rising melodies and large numbers of instruments meant listeners thought the music was happy music.

For decades music psychologists have been studying how different aspects of music such as rhythm, harmony, tempo, melody, mode and loudness affect emotions. Most people support the idea that a piece of music will evoke a sad feeling if it is in a minor mode, has a slow tempo, low pitches, firm rhythm and complex harmony.

But this study is different because it tries to put numbers on the effect.

"If music is changing and the emotions are changing then it should be mathematically possible to use the musical features as predictors of the emotional response," said Schubert.

Schubert said that previous studies had tended to ask participants to listen to a piece of music and then respond, but he wanted to use a method that took into account the ebb and flow of emotion as people listened to a piece of music.

He also used a more sophisticated method of measuring emotion than previous studies. Schubert not only looked at whether particular pieces of music induced people to feel positive or negative, he looked at an independent dimension: whether they were arousing or made people feel more sleepy.

This "two-dimensional emotion space" identified the difference, for example, between someone who felt angry (negatively aroused) and someone who felt joyous (positively aroused), blissful (positively sleepy) or depressed/bored (negatively sleepy).

A complex effect
He set up 67 volunteers to listen to four pieces of music expressing a range of emotions. The volunteers indicated the emotion being expressed musically by moving a mouse over a computer screen. The mouse movements indicated whether they found the music happy or sad, arousing or sleepy.

Schubert monitored how the emotions changed every second through the whole 20 minutes of music. He also analyzed how different musical features changed during each second and then modelled how the emotions and music features changed over time.

The music features he measured were loudness, tempo, pitch, texture and brightness (harsh/warm).

But Schubert warned it was important to take into account the personal factor.

"Our emotional response to music is highly complex and has a lot to do with what we bring to the listening experience, such as memory, expectation and conditioning," he said. "Before we can compose musical emotions by numbers, we need to convert human experience and cultural knowledge variables into numbers, too. It will be some time before we can do this."

Schubert used classical music in the romantic style, common in the 19th century, in his experiment. But he said the approach could be relevant to contemporary music, which would need further work.

"In romantic music, you're seeing a constant ebb and flow of emotion whereas in popular music, like baroque music, the fundamental aesthetic is to portray a single emotion and it doesn't change much throughout the piece," he said.

"Composers through the ages have exploited mathematical relationships among rhythms, melodies, harmonies and other aspects of music to create and change emotion.

"What we've shown is that it is already possible to locate and quantify some of these emotions with some precision," he said.

This makes my head hurt!

MarkB

[antstastegood]

No mention of different kinds of rhythms, sycopations, etc...?

Also, I've heard slow, quiet melodies that were actually quite happy, possibly due to certain cadences and things like that.

[buddhu]

What kind of person would you have to be to even *want* to explain music away in terms of maths?

The researchers should get out to more gigs.

[Wombat]

I want to emigrate. No I don't, I want to find out the number which will make him want to emigrate and beam the appropriate music into his office until he gets the message. I'm close enough to Sydney to do it too.

[Nanohedron]

"If music is changing and the emotions are changing then it should be mathematically possible to use the musical features as predictors of the emotional response," said Schubert.

I suppose this is true as far as it goes; cultural tonal associations count, too, though. Take the Lament for Staker Wallace: Gmaj with an Fnat turn in the B section. To conventional Western hearing, when played on flute it seems rather pretty, pastoral, and a bit wistful, but it IS a lament, and was composed on the occasion of the hanging of an Irish patriot. I've heard other "happy"-sounding Irish laments in G as well, which seems to fly in the face of Schubert's ideas.

I'm presuming that Schubert is operating from a conventionally Western angle.

[Ridseard]

But Schubert warned it was important to take into account the personal factor.

Until you do, it's sh*t.

"Our emotional response to music is highly complex and has a lot to do with what we bring to the listening experience, such as memory, expectation and conditioning," he said. "Before we can compose musical emotions by numbers, we need to convert human experience and cultural knowledge variables into numbers, too. It will be some time before we can do this."

I'm not going to hold my breath.

[BillChin]

This is not a new idea. At least one company is making money doing this. Several record companies either were or are clients. For the curious:

http://www.hitsongscience.com/

From their FAQ:
How does the technology work?
We use artificial intelligence applications as well as other methods to analyze the underlying mathematical patterns in music. Our technology does something called spectral deconvolution which is a fancy way of saying that we can isolate and separate many musical events that occur in a song. Some of these event are patterns in melody, harmony, chord progression, brilliance, fullness of sound, beat, tempo, rhythm, octave, and pitch. We then compare the patterns in new music (yours for example) to patterns in recent chart hits. By doing this combined with other mathematical calculations we've been able to develop a highly accurate and scientific tool. While this may sound like science fiction and that the results can't possibly be meaningful please keep in mind that we are working for major labels on both side of the Atlantic who trust and use our results to make decisions. Please see our explanation of the technology.

[Nanohedron]

The Borg have landed.

[MarkB]

Count the number of empty pints on the tables after a session, divide by the number of players, multiply by the number sets that were played adding in the speed in which they were played or weren't played = a great session

Our forumla for a good session.

MarkB

[John S]

These people never look at the power of trad dance type stuff, which is sad, because the structural constraints of the music make it a good place to start.
To my ears good tunes seem to have a dialog between entities embedded in then, a sort of sophisticated emotional question and answer.
Of cause the bottom line is that any mathematical explanation of a tune will never come near to the "in the moment" experience they generate, let alone their transcendent power.
John S

[GaryKelly]

I've heard other "happy"-sounding Irish laments in G as well, which seems to fly in the face of Schubert's ideas.

No, I think that's just the Irish...

One of my favourite songs, which always conjures images of gaily dancing elves in my mind's eye, turned out in translation to be about some bloke who's missus has just run off with another bloke leaving him lamenting the betrayal and swearing he'll never trust another woman again... Amazing.

[Sam_T]

I feel I ought to come to the rescue of Emery here, since I know him and have worked with him professionally. For start, he's a very nice bloke and a very good musician who gets to plenty of gigs (often conducting them, in fact). Secondly, I can bet you that the reports of his work paint a much cruder picture of what he's been doing than the reality. But this always happens when the media get hold of an interesting music-related story (remember the "Mozart Effect"?).

Generally, though: I've never undestood this idea that music should somehow be immune from demystification. Sure, people have strong emotional responses to it, but that doesn't mean it's magic. We're all human, with broadly the same physisological and psychological make-up. It should be no surprise if responses to music are broadly predictable, provided sufficiently careful measuring and modelling is carried out. It's not like we'll somehow "spoil" the music by knowing why it has the effect it does.

Sam

[GaryKelly]

Evryehtnig is jsut Ptatren Reoctnigoin aynawy, from raednig to muisc. All vibriatons are mahtametcial. The brain is a cool toy. You shulod have no torulbe raednig this, for emxalpe.

I have no trouble with the concept of delving deeper into pattern recognition, which is what Mr Schubert's work appears to be doing, any more than I'd have trouble with a work suggesting a link between various natural and man-made substances having a measurable effect on mood and emotion. I'm off down the pub.

[MarkB]

Sam, I agree. I think it's fascinating that someone was curious enough to study it! But what scares me is that someone, somewhere will use that knowledge to write music, as someone said above, that will puposely use to manipulate us, which is the case now in all advertizing.

Coming out of the visual arts, I have often wondered if there was a formula(s) that could be used make art that is consistently saleable.

If if weren't for the curious of the world we wouldn't be where are today, music or anything else.

MarkB

[BillChin]

...
Coming out of the visual arts, I have often wondered if there was a formula(s) that could be used make art that is consistently saleable.
...
MarkB

There certainly are universals based on naturally occuring ratios, such as Fibonacci numbers, the spiral of a Nautilus and the like. I don't know if there is a similar outfit to HitSongScience for visuals. It seems like it would be easy enough for programmers to try and adapt the algorithms to analyze paintings, photos, magazine print ads, and such. If there isn't one, I'm sure some folks are working on trying to become that kind of company.
+ Bill