Christopher Milam's List: Interactive Audio

• Generation One, 1970-1980
• From the mid 1970s to 1980 or so, video game music started as the most horrendous
• ust when the microprocessor ‘chip’ entered the video game audio arena but it is fairly evident that by 1980 it was  in fairly widespread use. Most of us shudder at the thought of such primitive days, but it was a tremendously exciting time for the engineers  involved.
• Generation Two, 1980-1990
• Video game music grew by leaps and bounds in this period, as did games in general in all aspects of their technology. Vector graphics began, using  lines to draw objects instead of the blocky pixels, moving cockpits were used in various titles (Sega’s “Afterburner II”, for instance), and audio  turned into full-fledged chip processing.
• “From the time you press start until the games over, the beat continues. Music always changes at a musical boundary (beat, measure, 1/2 measure, etc).  Some sound fx (the pop bumpers on the upper playfield) are timed to 1/16th notes. When you lock a ball, the sound is on a beat boundary. Also the key of  the sound effect matches the underlying chord of whatever the background music is playing...if you have the glass off, try locking the ball when you know  the chord's about to change, and you'll hear the sound effect transpose in mid-stream. Graphics (lights, flashes, visual display) are all very, very  tightly synchronized with the music. A further trick...the vocal singing (the 'aaah's in particular)...Memory was REALLY tight. The main song is in E minor.  I recorded a vocal "aaah" of an Emin chord.  I use that same sample as Emin chord, CMaj 7 Chord and Bsus,  so it sounds like there’s  a lot more singing samples than I actually have.  (listen to the part in the main music where after the "...beat the black Knight!" is sung...  chords go Em, CMaj7..Bsus...B7...EMin.) Each mode has it's own music...main play...one ball left for mball...mball...jackpot...ball in shooter (waiting  to plunge)...they are all harmonically related, move from one to the other seamlessly.  Also, as the music progresses, if you go to another mode  (say a timed mode), when the mode's over, it doesn't always go back to the beginning of the 1st piece. It might pick up in the middle.”
• Generation Three, 1990-present
• "All the music is streamed, but we were able to still make it somewhat interactive. Haunted House Level - Organ music fades in/out depending on how close you are to the organ - it was written to go with the existing music bed regardless of when in the music you happen to go near the organ. Maze of illusion - music changes when playfield changes. Carnival of the lost - music changes as you pass through various sections of the playfield."
• “Adaptive audio systems provide a heightened user experience through a dynamic audio soundtrack which adapts to a variety of emotional and dramatic states resulting, perhaps, from choices the user makes. What does this mean to the pro as well as the layman? Interactive audio is audio that happens when a user does pretty much anything with any kind of device, whether it be click a mouse or hit a key. Adaptive audio refers to something that happens most often in video games (at times in websites as well) when the user goes through more than just simple interactivity."
• Early examples of adaptive incidental music are seen in such games as “Vanguard” (SNK), when the player flies through a fuel ‘depot’ (a lovely little pixellated flashing tunnel with the word “FUEL” written above), the music changes from the main theme which begins the level (derived from Paramount’s “Star Trek: The Motion Picture”, music by Jerry Goldsmith) into a triumphant theme (derived from Thorn/EMI’s picture “Flash Gordon”, music by “Queen”) that lasts for as long as the player is invincible… around 15 seconds, during which the player can fly through anything and destroy it.
• Certainly, the public isn't exactly clamoring for adaptive audio, but perhaps that's because there isn't much of it out there to clamor for. If we look at the increase of music based titles however, adaptive audio seems to be a good thing to keep your eye on, and if you didn't know all the games demoed above had adaptive audio and enjoyed them anyway, then the authors did a good job. Again, for all you out there who wish another title to be featured in a future article... write in, and happy AA hunting!

9 more annotations...

• Creating an Interactive Audio Environment
• Audio Object Vocabulary
• An audio object vocabulary is a method by which game objects (not necessarily just speaking ones) talk to each other and the player. The methods of communication vary from object to object and from context to context. There are three types of object interaction: direct, indirect, and environmental.
  
• Direct Communication
• An object communicates directly as a cause of direct action on its part. When the ball hits the paddle in the old arcade game Pong, it makes a bleep. This is direct object interaction.
• Indirect Communication
• This is an indirect method of object interaction. That is, by causing something to happen in the game, something else responds sonically. A typical example of this is a "sighting" state for an enemy.
• Environmental Communication
• A character or object in the game may generate a system of audio cues on its own, irrespective of its communication to the player. This is purely a function of a character's existence in its environment.
• Character Development
• A weak character may be depicted through the use of a humorous voice. A stronger character's dramatic personae can be highlighted through the use of a deeper and more resonant voice, as well as a script that relates without question his or her authority.
• TIPS AND TECHNIQUES.
• Always use professional voice actors. Trained voice actors are professionals who specialize in giving your character the voice it deserves. Whether it is a cartoony or a deep resonant voice, a single talented actor may help develop your ideas for multiple characters and realize them in ways that you haven't conceived. When in doubt of how to find a good voice actor, look to talent agencies and talent search services for help. Moreover, making a trained voice work with the rest of your mix is quite a bit easier than trying to amplify or equalize a weak voice. All sound engineers can attest to this.
• Spend a little more time in sound design. As in all cases, don't just pull sound effects off of a CD. Create sound effects from your own sampled sounds as much as possible. A portable DAT recorder and a good microphone in the field will take you much further than a commercial CD sound library ever could. Nothing kills a unique audio environment more quickly than the phrase, "I've heard that somewhere else before...."
• Collaborate with professional scriptwriters. Writers would jump at the opportunity to write a couple of hundred lines of dialogue for some game characters. The results will definitely be worth the investment.
• Don't be afraid to inflate the vocabulary. Minimize silent time. If you have the space for audio, use it. Set the limit with the programmers and designers early as to your memory budget for audio, and use it wisely.
• Ambient Sound
• Ambient sound refers to the sound world that is generated from a player's location in the game space. It is a system of indirect and environmental cues that immerse a player in a particular setting.
• Environmental ambiences fully transport a player into the world presented by the game.
• Tips and Techniques
• Try to use consistent reverb settings. All sounds within a given environment should have a similar set of reverb settings that place the entire sound world within a consistent acoustic space. There are foreground and background elements that do stand out from within the ambience, but not so far as to mistake these sound elements for characters or objects that a player must encounter.
• Make your loops seamless. The looping ambiences in the game need to be smooth and unnoticeable. Large variations in pitch or amplitude will make the loop quite recognizable and annoying after a while. A rhythmic pattern works well (like the sound of crickets), if it's cut perfectly. Also, a longer sound sample will help mask the loop point.
• Avoid loops. Though seamless loops are not an impossibility, it's best to use trigger ambiences whenever possible. Trigger ambiences help mask the loop point, as well as provide overall variety in the ambience. In Kesmai's Air Warrior, I used trigger ambiences to convey the sound world of a World War II airfield. During any given time, an airplane fly-by sound, a vehicle drive-by sound, and an airplane startup sound would be selected and played from a set of 50 or so trigger ambiences. Since these trigger ambiences were selected randomly and played at random times, the sound world was always changing and seldom repetitive. Another method of avoiding loops is to queue similar sounds one after another. A set of three or four sounds that fit seamlessly end-to-end will work well if they are selected to play on a single channel randomly. This helps break up the pattern created by a single looping sound.
• Try to create fine gradations of ambiences. Say we're walking from a forest into a mountain pass. We start out in a deep forest then walk through a leafy forest then into a meadow before reaching the mountain pass. If we have a single sound for the forest ambience, no matter how the forest changes, the ambience will remain the same until we change scenery drastically when we reach the mountain pass. However, if we subdivide the forest into three gradations (deep, leafy, meadow), we'd be better able to convey to the listener the transition of environments from forest to mountain pass.
• Adaptive Music
• The nonlinear medium of computer gaming can lead a player down an enormous number of pathways to an enormous number of resolutions. From the standpoint of music composition, this means that a single piece may resolve in one of an enormous number of ways.
• Control logic - a collection of commands and scripts that control the flow of music depending on the game state.
• Segments - audio segments that can be arranged horizontally or vertically according to the control logic.
• Tips and Techniques
• Music comes first. Remember that no matter how closely your music follows the game play and how interactive it is, if it doesn't gel as a musical composition, you're better off writing a linear score. Always explore all possibilities of transitions from one game state to the next, and see if the music reacts the way you meant it to react. Make sure that you write transition sequences and that the engine is intelligent enough not to change game states midmeasure or midphrase.
• Decouple segments horizontally and vertically. Compose your music so that different segments may be combined end-to-end (horizontally), as well as on top of each other (vertically). This way, you can combine different melody lines with bass lines, use different ornamentation, and so on.
• Don't give away too much information. Sometimes a musical cue might say too much, when it was meant just to highlight the game state change. For example, in a certain game, an upward chord progression always signifies to a player that a starship is on his tail. When working on game state changes, make sure your event-driven music isn't used as an early warning system for the game.
• Define a series of translation tables to track game state changes. For example, in Multiplayer Battletech, a game state change from "winning" to "advantage" implies a losing trend. The music reacts to this change by selecting a different set of segments than it would if the change occurred from "advantage" to "winning." By composing in a nonlinear fashion, and by having the music react to the player's actions directly and indirectly, we introduce a new level of interactivity. Emotionally, the soundtrack carries the person seamlessly along with the action in much the same way as the static, linear media of film. In this fashion, music becomes the gateway to the player's emotional response to the game.
• Total Immersion through Sound

• As game designers and audio producers, we should be constantly aware of the impact that a well thought-out audio environment can have on the product. It can make a graphically simple and uneventful scene become awe-inspiring. Effective use of an audio object vocabulary can enhance the impact a character may have on the game player. Ambient sounds, in all of their variety, can transform a game scene from a virtual one to a believable one. Surreal textures and atmospheric gestures can generate emotional responses in a player as varied as the soundscapes themselves. As games become more and more complex and graphically spectacular, we must not overlook the role of audio in enhancing and completing that feeling of total immersion.

   

33 more annotations...