Game Audio Basics in Engines

Game Audio - Sound in Game Engines

• Current digital games often use PCM audio file playback.
• Audio files accommodate various sounds and music types with diverse quality.
• Each game engine offers unique audio features.
• Users typically employ the game engine's audio engine or external audio engines (code-from-zero, open-source, proprietary).

Typical Audio Features in a Game Engine

• File Playback:
  • Initiating audio file playback.
  • Terminating audio file playback.
  • Seamlessly looping audio files.
  • Importing audio files for playback.
  • Common import formats: 44.1kHz or 48kHz, 16-bits or 24-bits.
  • Audio files in shipped games are frequently compressed (lossy) to conserve space.
  • Files can be loaded into RAM or streamed from disk.
  • Real-time volume and pitch adjustments are possible.

Simulating Sound Sources in 3D Environments

• A listener (like a microphone) positioned in the camera coordinates.
• Sound sources are positioned within the 3D environment.
• Level attenuation and panning techniques simulate sound source position.

Level Decrease with Distance

• Sound levels decrease with increasing distance from a source.

• Level is calculated as a function of the distance. (a(distance) = a(||pos source - poslistener||))

• Various mathematical curves (linear, logarithmic) are utilized to determine attenuation.

• The equation is using the distance between a virtual listener and source, with position coordinates.

Panning

• Stereo panning for speakers applies varying attenuation to left and right channels.
• The attenuation for each channel depends on the angle between the listener and the sound source. (θ).

Headphones - Binaural

• HRTF/Binaural is used for headphone audio in VR.
• Head Tracking in helmet helps.
• The problem with stereo in headphones/speakers is to track the sound coming from the front correctly, since audio sources might be placed all around the listener's position.

Panning Considerations

• Attenuation and panning settings can be controlled individually.
• Panning is typically disabled for widespread sound sources like those from an entire forest.
• Only disable panning for closer listener-source proximity.

Reverb

• Reverberation effects can simulate sounds inside buildings.
• Reverberation effect should only be applied while the source is within a building.
• Game engines offer mechanisms for designating areas affected by reverb.

Low-Pass Filter

• High frequencies are more absorbed by the air during long distances.
• This attenuation is simulated by a low pass filter (depends on distance).

Directional Sources

• Some sources (like horn loudspeakers) are highly directional.
• Game engines can reproduce this directionality.
• This is simulated using low-pass filters and attenuation based on the listener-source direction angle.

Occlusion

• Physical obstacles between sound sources and listeners affect sound.
• Some game engines use ray tracing to dynamically verify obstacles and simulate occlusion, calculating the sound level accordingly. This effect is named Occlusion.
• Use cases are sounds in rooms from outside the room, or sound that is hitting a wall/pillar from inside.

Dynamic Occlusion

• Changes in building structures must enable real-time adjustments to occlusion and reverb settings.

Doppler

• Game engines can simulate the Doppler effect for moving sound sources.
• Alternatively, sound designers can record the Doppler effect separately.

Dealing with Time

• Implement time-based actions (e.g., waiting n seconds).
• Use curves (breakpoints) to control parameters over time.
• Utilize general programming or audio engine-specific functions (e.g., coroutines, timing events).

Mixing

• Mixing involves combining sources and affecting their volumes.
• Typical arrangements in games include music, sound effects, ambient sounds, and dialogue.
• Approaches to mixing include VCAs for simple volume controls and more elaborate mixing parameters using audio groups.
• Using mixing groups usually adds more flexibility for effects per group and/or per parameter, adding higher computing resource usage.

Audio Effects

• Some game engines facilitate audio effects on individual sound sources.

Automatic Mixing

• Approaches to automatic mixing in games.
• "Ducking" reduces other sounds when voice lines are played.
• Sound layers are increased when listeners approach sounds in the game.
• The sound of music may change based on the listener's emotional state/stress level during play in the game.

Limitations

• Audio Budget: The available space for audio files.
• Examples include restrictions in the sizes for 4GB game discs(1GB limit or something similar).
• Maximum Number of Voices: The maximum concurrent sounds the game can play.

Audio Compression

• Files used for music, dialogue need to be compressed.
• Sound effects may be uncompressed (or not compressed).

Prioritization

• Sounds are assigned priorities to appropriately manage instances of multiple sound sources.
• When the maximum number of voices is met, the rule is to automatically pause the lower priority ones.
• A stop rule is stopping the playing of softly/quiet sounds.

Audio Culling

• Game engines remove sounds that are too far away from the listener in order to optimize computation/performance.

Game Engine Audio Capabilities and External Middleware

• Game engines usually have basic audio capabilities, but using external middlewares can expand these capabilities.

Audio Middleware

• External libraries (e.g., FMOD, Wwise) that integrate into game engines

Composed of

• API or engine integration (Unity, Unreal).
• GUI application (similar to a DAW).

Audio Middleware Considerations

• Game audio events trigger changes in the middleware.
• Continuous parameters are set by the game engine and used by the middleware.

Audio Middleware Features

• Random playback with variable delays.
• Advanced timeline logic for complex sound playback.

Audio Middleware Usefulness

• Sound designers can use middleware without understanding game engine programming.
• Sound playback, transitions and logic can be tested within the middleware without needing the game engine to be running.
• Audio programmers export the middleware data into the game engine.
• Sound designers and programmers agree on parameters to use.

Interaction between Audio and Game Systems

• Example: dynamically altering sounds based on game mechanics/events. (e.g. sound of a car would vary depending on speed, a parameter such as speed would be retrieved from the game engine.)

Additional Notes

• The presented notes cover the essential concepts of sound/audio in video game development.
• Individual games, engines, audio middlewares have different/more specific characteristics.

Explore the fundamentals of audio in game engines, including PCM playback, file import formats, and unique audio features. Learn about the methods for simulating sound sources in 3D environments and how real-time adjustments enhance the gaming experience. This quiz is perfect for those interested in game development and audio design.