I have heard the music in games, and I have an idea that might be useful to help games become more immersive. Put AI in charge of composing. I've studied music, and have learned that changing one note can alter the mood of a song from happy to sad, carefree to intense, etc, etc. AI could dynamically compose the music to reflect the mood of the situation. Of course, AI in charge of the soundtrack would need a lot of troubleshooting, lest someone be surprised by it freaking out by an event that it was not programmed to respond to. Am I on to something here, or should I check my home for toxic fumes? Feel free to nag/hurl venom/brag/reply/muse if you feel the need to...

Funny you should mention that. There's actually talks of the Aphex Twin (check warprecords.com if you have no idea who he is, and if you have no idea who he is, for shame!) writing a program to ... well, write music for you. Kind of. Within reason anyway. And not actually within Reason, the program. Ho ho. *sigh* So yeah, there is talk of AI composing already being done, and I'm sure it's already been done in some small way before. Surely? Chaps at LH probably know - *glances at Russel and Alex*. Go on, reply. Please?

Well there are very basic AI based music changes in most games these days, that switches the music from an ambience track to an action oriented track based on the situation the player is in. But that's not really what you're going on about. That said, the idea has been explored in a few titles, most notably 'Rez', which has a concept heavily influenced by the ideas of Russian abstract artist Wassily Kandinsky. Kandinsky said that when he saw colours he heard sounds, and this conflict of the senses a condition known as 'synaesthesia', which is what Rez is based on. This following paragraph is from an article about synaesthesia in Rez...

What elevates the title beyond mere eye and ear candy is its interactive nature. The stages' music and visuals are strongly bound together; changes and mutations in one are mirrored or countered in the other. More importantly, everything the player does creates sound and vision - not mere sound effects, but music. Every time the player presses the lock-on button, a snare drum snaps. Each enemy lock-on brings forth a hi-hat click. Firing plays out a musical phrase, different according to the number and type of targets, stage, layer, and avatar. Explosions are hard-hit power chords; layer changes are swelling crescendos. All of this is matched by a frenetic display of cycling colors, rotating perspectives, oscillating boundaries, and non-Euclidean geometries. The Dual Shock's vibrations are also carefully balanced between accenting the music and reinforcing the player's actions.

Basically, if you're interested (and aside from being a good concept it's actually a good game too) then for about £50 - 60 you can pick up Rez and a Dreamcast off eBay, or you can get it for about £35 retail on PS2 (i've seen it in a few shops but it's discontinued so you'll have to keep your eyes open). As an aside, the ideograms used for stop (square), play (forward-pointing triangle), pause (interrupted square), and fast forward (double forward-pointing triangle), that you see so much in audio and video recording and playback equipment, were based on Kandinsky's artwork.

Yeah, that's interesting stuff. Several of the great composers are thought to have had that condition. But their's was a case of hear music and "see" colour - "C major is green"!! What's interesting is that when you compare the results of two different sufferers, they do not agree - one may "see" blue and the other may "see" pink based upon them hearing the same sound. I dont think that true real-time composition would work; there would be a lack of coherence in the performance (not generally desirable). If the composition was based upon pre-composed themes then we're at the stage that interactive music has reached right now. I think Black & White is a superb example of variable "interactive" music. It was also an excellent example of when not to use music, a concept alien to most interactive media composers.

There is enough terrible music in the games industry.... We certainly don't need any more. AI composed music? that would kill my love of music straight away.

Unfortunately I know very little technical information about musical theory, but this concept has always intrigued me. Much about the art of music composition can be taught (or so I understand), and therefore it must come down to a set of (loose) rules. Everyone has some of the same basic opinions on what sounds "good" and what sounds "bad," so surely these could be taught to a computer? There must have been scientific studies into what precisely music is. But how far off is an effective program of this type? Arthur C. Clark once wrote a short story about this subject; something about a computer that wrote the "ultimate" song. A far-fetched idea, but an interesting one. The implications go far beyond mere games, I think.

There are already programs created that use a formula to 'compose' music... Some guy would feed in existing compositions and it would use the structures as a referance to create its own compostitions. It wasn't very good. Lets be honest, there is a hell of a lot more to music than just put some notes together. Even if it managed to provide a passable composition, it would also need to know how to orchestrate and mix it (fields that could fill up concrete block sized books) etc.. You may be able to teach some elements of music... but there are thousands of people who, tho taught how to play an instrument and read and write music... would not be able to write good music of thier own if their life depended on it. Some people are good composers, most are not... and all the training in the world wouldn't give a non-musical person any musicality. With regards to people generally all agreeing what is 'good' and what is 'bad'... thats not really true either, there are so many different types of music, play one piece to one person and he might love it, play the same piece to another and they may very well think it is awful. It's like those randomly generated levels on soldier of fortune 2.... they pale in comparison to 'designed levels'... And thats why I would choose john williams over "AI" every day of the week.

regretably mr. Williams doesn't write game music. but there's always Russel Shaw and Nebuo Uematsu

James: perhaps my usage of the words "good" and "bad" was a bit broad. I meant more along the lines of the basic ideas of music: such as how most people can tell if a sound is discordant or condordant, for example, or how a D note is not typically used to end a melody.

Hmm... good point about the randomly-generated level, I was thinking of that myself too. I suppose you could create a basic track with some basic rules. Eg, for a track to fit an "angry" scene, you'd need a fairly-fast tempo, short, loud notes, etc. This would need to be fed into a program to generate music, then the actual music would need to be composed. And anyone who has done the basic grades in music will testify that the sight-reading test given in the exam, which is made up of some notes in a particular range that increase in complexity as the music grade level goes up, sounds like utter shite and not like music at all. So the best approach (imo) would be to take an approach akin to MGS 2, where specific tracks are written for specific event, and the program decides when an event has taken place and so triggers the track. So if you are discovered, a certain track would play and so on. This has the disadvantage that all the tracks must be samey enough to repeat ad-infitum, as the given event could last forever. Thus the impact of music that is properly choregraphed for specific instances a la films, with specific events in the music coinciding with specific events in the movie. Just imagine the the final fight scene in Phantom Menace without the excellent orchaestral track (which my friend sung in BTW) properly matched to the fight. But that just leads to the question of scripting in games, which is a bit OT.

Matthijs: True, John Williams doesn't make music specifically for games, but that's not to say that for example, having his Star Wars score in Tie Fighter, wasn't fitting music. Also, if Harry Gregson Williams can have a crack at the Metal Gear 2 music, then I guess at some point John Williams might give it a shot. But this is all a little off topic, so don't let me side track people.

Unity. Oh yes.

This a topic that really fascinates me. AI/generative music and "interactive" music. There's a freely downloadable program, "Microsoft DirectMusic Producer" that's worth checking out if you're into this sort of thing. I used it as the backbone of a musical composition device/thingy I made that can be downloaded from here: http://members.iinet.net.au/~snderson/modgeulator/bowlophone/ Don't know if this is an old, dead thread or not - but anyway... Any feedback would be appreciated too if anybody is interested.

I think I remember that Brian Herbert wrote something on AI formula-based musical composition in his continuation of the Dune series. They wrote scores were mathematically perfect, but lacked the sensibility/spirit of human compositions. This irked the robots to no end as they searched for a way to make their work as good as humans'. Music is kind of like poetry in that way. You can have flawless parsing and the most creative combination of words and phrases, but the best poetry has meaning beneath its surface.

Remember when there were people saying "We've studied what music people like and have created a formula to create the perfect track"...... ....funny that none of them are millionaires right now eh? I think as far as creating the motifs and overriding concepts go, it's a bad idea. But for changing the direction/key and basic stuffs (with in a given limit of ideas) is a good idea.