by Matt Fox
When we set out to make G: Into The Rain, we came to understand that sound design was an area we did not want to neglect. This is not to say that we were immediately aware of the importance of having good sound. In fact, to be honest, we knew that G was going to be first deployed on the iPhone. The on-board audio playback hardware on the iPhone is not exactly high quality – unlike the netbooks which we eventually ported G to late in 2009. During the development, we started to realize that we weren’t necessarily tied to one platform, and that devices like the netbook could impart a much richer game play experience in terms of sound.
In the early conceptual phases of our development, we decided to have a look around at the games that we all liked. Sound design wasn’t something that jumped out as a priority. Frankly, the first non-visual game element that most people notice is music; and while we were blessed with a very talented composer for the musical score of G, that is a topic for another post. Sound, as a game element, is often overlooked. If you ask many people what they enjoyed about a particular game or application, more often than not you will hear about how they liked the gameplay, or that the art was stunning, or that the storyline moved along really well, or that they really liked (or hated) a particular character. Sound design, or rather good sound design is not something that is in your face. It’s subtle, and most often it’s only really noticed after the visual. For instance, when you walk into a room, unless there’s a buzzsaw running, most likely the first thing you’ll notice is what it looks like. We are visual creatures, and hearing most often is employed after sight.
Now I don’t want to convey the idea that because we’re primarily visually oriented, that good sound design can be left for the back burner. Quite the contrary. When we looked around at the games that we all liked to play, we started to catalog and attempt to define what it is that we enjoyed about those games. Eventually, we began to pay close attention to the sound design. In doing so, we had to look beyond the iPhone, and plan accordingly. We came to the conclusion, that one of the reasons we liked the games we did was because of the sound. In most cases, it wasn’t an obvious, in-your-face sort of revelation. Sound was used here as a way to augment the look and feel of the games. Looking ahead, we made the choice to take sound design seriously – which is especially vexing considering the less-than stellar (external) speakers on the iPhone. But if we had taken the approach that “nothing will sound good on the iPhone” and had half-hearted sound design, then G certainly wouldn’t have sounded as good as it does now on a netbook.
We began to notice different types of sounds – or rather we began to classify them for our purposes. Ambient sounds, event driven sounds, UI sounds. Each of these sounds had its own impact in the overall game. Earlier I mentioned a few of the things that a person might remark about what they enjoyed about a particular game. As it turned out, it was surprising just how many of those items included sound. Gameplay, for instance would be a pale, and shallow thing without sound effects. Whether that means the over-the-top and cartoony explosions, or more realistic sounds of blades and armor, sound has a very important part in crafting the game experience. Storyline is another game element that benefits greatly from sound design. The talent of a voice actor has the obvious ability to move the story along, but ambient sounds that accompany that voice presence, can go a long way towards adding depth to the story. One well known example would be our pal Darth Vader. He never went anywhere without his bionic wheeze. While that wasn’t specifically part of the voice work, it certainly is a part of his identity. Whether tied directly to a voice actor, as the case with Vader, or ambient noise, sound communicates a tremendous amount of feeling and emotion that can really deliver a rich gaming experience.
Planning the sound
As it turns out, sound design can’t simply be tacked on to the end of the development process. The overall project benefits greatly from planning out sound in the storyboard phase of development. That is a time when it is much easier to allow sound to be part of the creative process, rather than trying to assign sounds to visual elements that have already been defined. A completed storyboard should include descriptions of events and the sounds that accompany them. A perfect example in this game was the sound of the rocket ‘ping’ which was cobbled together from several submarine sonar sounds. We wanted the game to feel more nautical than sci-fi and the use of this one sound did wonders to create that emotional impression.
Event Sounds
Event sounds are probably the most obvious sounds in a game. Event sounds are the in-game equivalent to the sounds that accompany real world events. A car crash in the real world without the screech of tires and breaking glass and shrieking metal, would be strange and… incomplete. Likewise, in a game there is certainly the option to add all of these things. In the making of these games, there is certainly the ability to tie every single action or event to a sound. Although sound gives us the ability to selectively add sound to the game, for the purpose of emphasis.
User Interface
In G, we ended up with at least as many sounds augmenting the User Interface, as we had sounds for events. This was one area, where we had to be quite creative. Many of the user interface elements in games either don’t have a recognizable real-world counterpart, or if they do, it’s not quickly apparent. This meant that we were free to come up with our own sounds to convey changes in the user interface. The launch angle indicator sound in G was a metallic click. We went through several different types of sounds before we could settle on one we liked. It just so happened that the chamber rotation click on a .357 magnum was the ideal sound for that. Other sounds had to be combined and processed. The Fire Control interface that slid in and out was the processed sound of an old drawer being pulled out of a chest. Since we were going for a more Steampunk-like feel, we didn’t want a sound that was too “clean.”
Creating Sounds
In some of the early development for G, our team recorded and altered some sounds for various events in the game. The SDER ping that the sounding rocket uses, as well as the early explosion sound were both recorded mouth sounds – courtesy of David Harrison. However, these were primarily for development. The sounds we used for G, were primarily taken from professional sound archives, and purchased or licensed for our use. By the time we went looking for pre-existing sounds, or recording new sounds, all the sounds for the game had already been determined. We didn’t know what the sound for the rocket impacting on an icteroid would sound like, but we knew we needed a sound to describe that event. That gave the sound designer, both more freedom and focus to deliver that particular sound.
Occasionally we would find a sound that without any changes, was perfect for what we wanted to convey. The main rocket thrust we used was taken from a liftoff tape of a Saturn V rocket. Now this presented an interesting problem. We needed a thrust sound that could be intermittent, or could be one long thrust sound until the tanks ran dry. We’ve found that looping the thrust sound yielded a product that seemed very artificial. Our ears are very good at picking up repetition, and the looped rocket sound sounded very… well… looped. Our solution was to use a length of recording (the actual recording was a few minutes long) that accommodated the longest possible rocket burn, and selecively sample as much or as little as we needed for the length of the burn as determined by the player. That was a good example of how game coders needs to be flexible in order to integrate sounds into the game. However, the majority of sounds aren’t processed for length or sound quality in-game. They need to be delivered, ready to play. The ever-present SDER ping has an interesting origin. We wanted this obviously to be something high pitched, and reminiscent of radar. For the ping, we actually started with an old-style sonar ping, like that you might have heard in the movie Das Boot. We then processed it for length, tempo, pitch, and then played a portion of it backwards and layered it onto the original sound. The result was something almost entirely new, but that triggered a sense of familiarity.
Processing the sound
For the majority of sound processing, we used Audacity. At the time, the price was right, and we were primarily altering sounds instead of creating new ones. It has plenty functionality to adjust timing, cropping, tempo, pitch… the list goes on. It also allows sounds to be layered, and a series of filters to be added to sounds. As sound editors go, it has proven to be a very capable tool. I have no doubt that we will likely explore other solutions , especially when we begin to actually create sounds from scratch. However, much can be accomplished using this modest tool.
iPhone to AppUp
When it came time to switch platforms and port G to AppUp, we were very pleased with the ease of transition from the Apple devices to those served by the AppUp store. Technical challenges ran from the minor to the non-existent, as far as delivering sound. And now, in hindsight, we’ve discovered that the attention to sound details, and solid planning of sound events and effects has really paid off on this new platform. The superior playback on netbooks and PCs has led us to reailze that the initial planning and effort was well worth it.
Overall, the sound development for G: Into the Rain, and other apps and games we’ve worked on, has been a lot of fun. As we progress in the scope and range of games and apps, I’m looking forward to expanding our expertise in this area. I hope that this little glimpse at one aspect of our development has been useful.
——–
This article first appeared on the Intel AppDeveloper Blog here: http://appdeveloper.intel.com/en-us/blog/2010/07/12/sound-design-appup-games-and-apps