Sound Effects for Games & Multimedia PDF

6 - Sound effects Sound Design - Games and Multimedia Miguel Negrão ©2023 CC BY-NC-ND 4.0 What is a sound Effect ? Example  What is a sound effect ? Consider all the sounds in a movie All the sounds which are not speech nor music can be considered to be sou...

6 - Sound effects Sound Design - Games and Multimedia Miguel Negrão ©2023 CC BY-NC-ND 4.0 What is a sound Effect ? Example  What is a sound effect ? Consider all the sounds in a movie All the sounds which are not speech nor music can be considered to be sound effects. Sounds of footsteps, clothes, cars, animals, wind, rain, weapons, etc... In filmmaking and digital games sound is at least half of the experience. The objective in both cases is to suspend disbelief in order to be "transported" to another world. What is believable turns out to be very subjective and culture- dependent. Realistic and believable is not the exact same thing. Sound design helps immensely to make an audio-visual experience believable. Sound also has a strong emotional impact. Most Hollywood clichés need music to work: romantic scenes → romantic music suspense scene → high pitch, scratch, dissonant, repetitive chords, etc. heroic scene → epic chord progression, drums, military marches, etc.  Sound effects are used in many contexts: Film Television Radio Theater Multimedia Digital games User interfaces Categorization of sound effects  Categorization of sound effects By function 1. Background (atmosphere or ambience) 2. Discrete (single sound)  Categorization of sound effects By sincronization 1. Hard sound effect: must be perfectly synchronized with image, e.g. gunshot, door closing, punch, footsteps. 2. Soft sound effect: exact synchronization with image not required, e.g. wind, rain, birds, ocean surf, crowd.  Categorization of sound effects By method 1. Foley (creating sounds through performance) 2. Field recording (recording in the "real world") 3. Sound design (creating a new sound through editing, processing, mixing and synthesis)  Background sound effects Help to give a sense of the location and the surrounding environment. Not associated with a specific action. The typical soundscape of a given location. Usually captured through field recording.  Background sound effects Examples: Street: traffic, people walking and talking. Beach: ocean surf, wind. Forest: birds, tree movement, wind, water stream. Train station: people walking, announcements over PA, trains moving, electronic sounds from ticket machines.  Discrete sound effects Synchronization very important Associated with a specific action Usually a single recorded sound (through foley or field recording) Can also be assembled (sound design)  Discrete sound effects Examples: Door closing (house, building, car) Punch, gun shot, sword hit, light saber hit Putting down glass on table Hyperdrive activating; entering transdimensional portal. Foley example  Foley Named after Jack Foley, Hollywood pioneer of many sound effect techniques. Involves performing actions with objects in a recording studio close to a microphone in order to create the desired sounds. Makes it much cheaper to record certain sound effects. Others are just impossible with field recording (don't exist, too expensive, etc). Examples: footsteps, car tyres screeching sound, objects breaking. Sound design  Sound design Creating sounds that don't exist in the real world: alien and fantasy creatures, spaceships, advanced future weapons, physical anomalies, etc. Using recorded sound: editing, processing and mixing sounds. Using sound synthesis. Example Deadspace 3  Field Recording Field recording material Recorder Accessories Headphones Microphone Basic field recording kit  Microphone Shotgun microphone High directivity (except at low frequencies) Reduces ambient noise Cardioid Less directivity May feel more natural  Microphone Stereo/multichannel To record ambiences Techniques Spaced (A-B, ORTF, etc) Coincident Techniques (X-Y, Mid-Side) More information see here.  Stereo Techniques A - B : time-of-arrival stereophony Left Right 50cm  Stereo Techniques X - Y : intensity stereophony Right Left  Stereo Techniques Mid/Side mid: cardioid, hyper-cardioid M S+ S- Master or shotgun Phase Phase Phase side: figure of eight Pan Pan Pan L R Stereo Techniques ORTF 110º Left Right 17cm  Recorders Nowadays always digital using solid state storage. Features: Quality of microphone pre-amps Quality of analog-to-digital converters Number of independent inputs (1,2,4,8,etc) Display Battery capacity File transfer/storage/filesystems allowed Timecode (e.g. syncing with video)  Recorders Configure the device with the intended recording format settings. Check available storage. Check battery charge. Check high-pass filter, phantom power, limiter and trim settings.  Recorders - Settings Default should be uncompressed (WAV), 24bits, 44.1Khz or 48KHz. When there is the possibility of changing the playback-speeed of the sound (pitch-shifting) record at higher sample rates.  Headphones It is necessary to monitor sound before and while recording to make good use of the microphone directivity, influence timbre and control noise and distortion. Monitoring is done with headphones. Should be closed-ear headphones (circumaural) to reduce sound spill (both ways).  Microphone Accessories Shock Mounts Microphones are subject to handling noise. Vibrations are transmitted to the microphone through direct physical contact. A shock mount absorbs the vibrations.  Microphone Accessories Windshield - blimp http://www.rode.com/accessories/blimp Windshield - foam  Windshield - deadcat Minimizes wind noise in high wind conditions. Windshield - stand Always preferable to not be in physical contact microphone when recording.  Field recording Other accessories Flashlight Umbrellas Connectors/adapters Blankets Tape Hair rubber bands Cables Food/water Swiss-knife Chair Walkie-talkies Hat/sun glasses The process of recording in the field  Recording When going to a location make sure you take all the equipment that is needed. It can be very costly to have to return to a location.  Recording Record at least 3 seconds of sound before and after the intended sound. A sound event takes some time to decay, be careful not cut the decay.  Recording Remain silent and don't move while recording. If a take is interrupted, the recording has to be repeated... everyone on the team should be silent during recording. If necessary talk with your hands. Make sure you've eaten to avoid stomach sounds.  Recording Eliminate as much as possible background noise. Background noise can be caused by: AC buzz/hum, air conditioning, heating, traffic, clocks, fluorescent lights, insects, airplanes, refrigerators, TV sets, etc.  Recording Check levels for each new situation, setup or location. The level should be high enough in order to avoid noise, but low enough to prevent clipping. Again, remember to always monitor what is being recorded with headphones. Don't change the level while recording ! (This becomes a non-issue with 32-bit recorders. For the moment there aren't many.)  Recording Point the microphone at the sound/object/area. This is also important when using stereo microphones, in that case point at the center of the area being recorded. Don't record a single object off-center. Try from different angles. Avoid moving a stereo microphone during recording.  Recording Record enough material (usually more than you will use). Why ? A take might have a mistake. For variation: record different versions, only later you will decide which is more appropriate. To accumulate sound material which can be used in different projects.  Recording - location Select the location based on the sounds you expect to encounter. Select the time of day. Investigate traffic situation. If there is risk of suspicion notify police.  Recording Log information describing each recording. Normal to have hundreds of recordings after recording session. Log information using paper, computer/smartphone or recorded speech.  Logging What to write down: object location action microphone microphone position time of day date name of the person  Recording Watch out for copyright violations. While doing field recording you cannot use recordings which capture copyrighted material such music or speech. Examples: Anything playing on a TV or radio. Band playing in the street. Video game sound and ring tones.  Recording Room tone is the constant background noise that is heard in a room when all else is silent. Is is important to record room tone: To simulate that a sound which was recorded in a very dry studio is actually in that room. In that case the studio sound and the room tone are played together in the editor. To obtain a noise profile which can be used by noise removal software. Foley  Foley Example: Transformers movies https://www.youtube.com/watch?v=RRM--iEWGGo  Foley are sound effects created by performing with objects in special sound studios. image from https://beatproduction.net/foley-sfx-pack/ Like any sound studio it must be insulated regarding sounds from outside and treated in order to minimize reverberation. In the ground there are foley pits with different materials used for footsteps and other sounds (e.g. concrete, hard wood, hollow wood, carpet, etc). When doing footsteps different types of shoes are used as well (heels, wood, rubber, etc).  Using objects you can simulate: Fight (e.g. swords) Tools (e.g. hammer) Kitchen (e.g. plates) Sport (e.g. tennis racket) Clothes. Use different types of fabric Materials for use in Foley can be obtained from: Home improvement stores 1eur stores Debris from materials destroyed during foley session. Junkyards The street Food (organic sounds)  Material for foley recording Axe Sandbags Rags Cotton Gloves Baseball Bat Sledgehammer Rope Dust Masks Broom/Dustpan Sound Blankets Rubber Bands Gaffer’s Tape Buckets Spring Clamps  Material for foley recording Rubber Gloves Knife Can of Air Towels Rubber Mallet Leather Gloves Carpenter’s Hammer Weights C-Clamps Mop Safety Goggles  Tips for doing foley: Experiment touching, rubbing, pressing or hitting a prop with the eyes closed. Where the prop is held affects the sound. Experiment holding the prop in different positions and more or less pressure. Try to perform, think of it as almost music.  Take care when recording: No talking. Smartphone off. Wear clothes that don't make any sound when you move. Don't wear jewelery. Use safety equipment when needed. Editing  Guidelines for editing Organization: Use a meaningful filename (more on this later). Save a session/file often. Backup original recordings and work from a copy. When appropriate work non-destructively (multitrack editing). Keep all files used by a multitrack session in the session folder (prevents unlinking).  Guidelines for editing For source material: Crop each sound. One sound → one file. Remove silence before and after sound. The sound should start immediately. Be careful not to remove the attack or decay of the sound. Make sure file starts and ends at a zero crossing. In some cases a very short fade-in or fade-out is needed.  Guidelines for editing For sounds created from multiple recordings using multitrack editing: Clicks and Pops Always cut a sound at a zero crossing. If necessary use a cross-fade. Pacing, speed: can be altered by controlling distance between sounds in the timeline. Correcting errors: removing an unwanted sound.  Guidelines for editing Balance stereo field  LISTEN - mnemonic L - Listen Critically I - Identify Clicks, Pops, and Errors S - Signal Process (EQ and Compression) T - Trim/Crop the File E - Examine Fade-Ins/Fade-Outs at the Zero Line N - Normalize/Name File Organizing: filenames and metadata  Filenames Three possible filename systems: Category-Based Effect-Based Numeric-Based  Filenames - Category-Based 1. Category 2. Noun 3. Verb 4. Description 5. Number (Zero-Filled) Example: "technology button press dvd player eject tray open 01.wav"  Categories AMBIENCE AMB HUMANS HUM ANIMALS ANM IMPACTS IMP CARTOONS CAR INDUSTRY IND CRASHES CRASH MULTIMEDIA MULT CROWDS CRWD MUSICAL MUS EMERGENCY EMER OFFICE OFC ELECTRONIC ELEC SCIENCE FICTION SCIFI EXPLOSIONS EXP SPORTS LEISURE SPL FIRE FIR TECHNOLOGY TECH TRANSPORTATION TRAN FOLEY FOL WARFARE WAR FOOTSTEPS FOOT WATER WAT HORROR HOR WEATHER WEA HOUSEHOLD HOU  Filenames - Effect-Based 1. Noun 2. Verb 3. Description 4. Number (Zero-Filled) Category is placed in the folder: "technology/button press DVD player eject tray open 01.wav"  Filenames - Numeric-Based Numeric-based file names are used to reference a data sheet or track list. button press dvd player eject tray open 01.wav becomes BUT 01.wav Metadata: additional information about the sound. Enables advanced searching.  Metadada OS comments (MacOS) External file (spreadsheet) Proprietary applications (e.g. NetMix and Soundminer)  Attribution: Images taken from wikipedia with Creative Commons Attribution- Share Alike 3.0 Unported licence.  Resources for studying: “The Sound Effects Bible: How to Create and Record Hollywood Style Sound Effects”; Ric Viers; Michael Wiese Productions; 3rd edition, 2008.