Elements of Multimedia Sound PDF

ELEMENT OF MULTIMEDIA SOUND Compiled by: Ts. Dr. Rashidah Mokhtar OUTLINE The Nature of Sound Digitizing Sound Processing Sound Compression 2 Formats MIDI Combining Sound NATURE OF SOUND 3 275–276 THE NATURE OF SOUND • Conversion of energy into vibrations in the air (or some other elas...

ELEMENT OF MULTIMEDIA SOUND Compiled by: Ts. Dr. Rashidah Mokhtar OUTLINE The Nature of Sound Digitizing Sound Processing Sound Compression 2 Formats MIDI Combining Sound NATURE OF SOUND 3 275–276 THE NATURE OF SOUND • Conversion of energy into vibrations in the air (or some other elastic medium) • Sound waves vary in sound pressure level (amplitude) and in frequency or pitch. • Vibrations in the air create waves of pressure that are perceived as sound. • ‘Acoustics’ is the branch of physics that studies sound. • Most sound sources vibrate in complex ways leading to sounds with components at several different frequencies • Sound pressure levels (loudness or volume) are measured in decibels (dB). • Frequency spectrum – relative amplitudes of the frequency components • Range of human hearing: roughly 20Hz– 20kHz, falling off with age 276–280 WAVEFORMS • Sounds change over time • e.g. musical note has attack and decay, speech changes constantly • Frequency spectrum alters as sound changes • Waveform is a plot of amplitude against time • Provides a graphical view of characteristics of a changing sound • Can identify syllables of speech, rhythm of music, quiet and loud passages, etc MULTIMEDIA SYSTEM SOUND • System sounds are assigned to various system events such as startup and warnings, among others. • Macintosh provides several system sound options such as glass, indigo, laugh. • In Windows, available system sounds include start.wav, chimes.wav, and chord.wav. • Multimedia sound is either digitally recorded audio or MIDI (Musical Instrumental Digital Interface) music. DIGITIZING SOUND 7 DIGITAL AUDIO • Digital audio data is the actual representation of sound, stored in the form of samples. • Samples represent the amplitude (or loudness) of sound at a discrete point in time. • Quality of digital recording depends on the sampling rate, (or frequency) that is, the number of samples taken per second. 281–282 DIGITIZATION – SAMPLING • The three sampling frequencies most often used in multimedia are CD-quality 44.1 kHz, 22.05 kHz and 11.025 kHz. • The number of bits used to describe the amplitude of sound wave when sampled, determines the sample size. • Digital audio is device independent. • CD: 44.1kHz • Sub-multiples often used for low bandwidth – e.g. 22.05kHz for Internet audio • DAT: 48kHz • (Hence mixing sounds from CD and DAT will require some resampling, best avoided) 283–285 DIGITIZATION – QUANTIZATION • The value of each sample is rounded off to the nearest integer (quantization). • 16 bits, 65536 quantization levels, CD quality • 8 bits: audible quantization noise, can only use if some distortion is acceptable, e.g. voice communication • Dithering – introduce small amount of random noise before sampling • Noise causes samples to alternate rapidly between quantization levels, effectively smoothing sharp transitions DIGITAL AUDIO • Crucial aspects of preparing digital audio files are: • Balancing the need for sound quality against available RAM and hard disk resource. • Setting appropriate recording levels to get a high-quality and clean recording. HOW TO CALCULATE - DIGITAL AUDIO Size of a monophonic digital recording = sampling rate x duration of recording in seconds x (bit resolution/8) x 1 Size of stereo recording = sampling rate x duration of recording in seconds x (bit resolution/8) x 2 EXERCISE 1 4 PROCESSING SOUND 289 SOUND EDITING • Timeline divided into tracks • Sound on each track displayed as a waveform • 'Scrub' over part of a track e.g. to find pauses • Cut and paste, drag and drop • May combine many tracks from different recordings (mix-down) 290–295 EFFECTS AND FILTERS Noise gate Low pass and high pass filters Notch filter De-esser Click repairer Reverb Graphic equalizer Envelope Shaping Pitch alteration and time stretching etc EDITING DIGITAL RECORDINGS Trim • Removing extra recordings from the beginning or end. Splicing • Splitting the audio file from any location into two or more slices. Digital equalization (EQ) • capabilities that allow you to modify a recording’s frequency content so that it sounds brighter (more high frequencies) or darker (low, ominous rumbles). Time stretching • alter the length (in time)of a sound file without changing its pitch. EDITING DIGITAL RECORDINGS Digital Signal Processing (DSP) • Some programs allow you to process the signal with reverberation, multitap delay, chorus, flange, and other special effects. Resampling Also known as Downsampling , is the process of reducing the number of samples on a file to lower its file size considerably. Assembly • Assembling two or more sounds on a single track. COMPRESSION 19 295 AUDIO COMPRESSION • In general, lossy methods required because of complex and unpredictable nature of audio data • CD quality, stereo, 3-minute song requires over 25 Mbytes • Data rate exceeds bandwidth of dial-up Internet connection • Difference in the way we perceive sound and image means different approach from image compression is needed 300–301 MP3 • MPEG Audio, Layer 3 • Three layers of audio compression in MPEG-1 (MPEG-2 essentially identical) • Layer 1...Layer 3, encoding proces increases in complexity, data rate for same quality decreases • e.g. Same quality 192kbps at Layer 1, 128kbps at Layer 2, 64kbps at Layer 3 • 10:1 compression ratio at high quality • Variable bit rate coding (VBR) 301 AAC • Advanced Audio Coding • Defined in MPEG-2 standard, extended and incorporated into MPEG-4 • Not backward compatible with earlier standards • Higher compression ratios and lower bit rates than MP3 • Subjectively better quality than MP3 at the same bit rate AUDIO FORMATS 302 AUDIO FORMATS • Platform-specific file formats • AIFF, WAV, AU • Multimedia formats used as 'container formats' for sound compressed with different codecs • QuickTime, Windows Media, RealAudio • MP3 has its own file format, but MP3 data can be included as audio tracks in QuickTime movies and SWFs DIGITAL AUDIO • Common Digital Audio Formats Type Extensions Codec AIFF (Mac) .aif, .aiff *PCM AU (Sun/Next) .au *u-law CD audio (CDDA) N/A PCM MP3 .mp3 MPEG Audio Layer-III Windows Media Audio .wma Proprietary (Microsoft) QuickTime .qt Proprietary (Apple Computer) RealAudio .ra, ram WAV .wav Proprietary (Real Networks) *PCM Table 1 - Common Digital Audio Formats MIDI 26 303–304 MIDI • Musical Instruments Digital Interface (MIDI) is a shorthand representation of music stored in numeric form. • It is not digitized sound. • Instructions about how to produce music, which can be interpreted by suitable hardware and/or software • vector graphics as drawing instructions • Standard protocol for communicating between electronic instruments (synthesizers, samplers, drum machines) • Allows instruments to be controlled by hardware or software sequencers • A sequencer software and sound synthesizer is required in order to create MIDI scores. • MIDI is device dependent. 304 MIDI AND COMPUTERS • MIDI interface allows computer to send MIDI data to instruments • Store MIDI sequences in files, exchange them between computers, incorporate into multimedia • Computer can synthesize sounds on a sound card, or play back samples from disk in response to MIDI instructions • Computer becomes primitive musical instrument (quality of sound inferior to dedicated instruments) • Since they are small, MIDI files embedded in web pages load and play promptly. • Length of a MIDI file can be changed without affecting the pitch of the music or degrading audio quality. • Working with MIDI requires knowledge of music theory. CREATING MIDI FILES • You will need: • Notation software • Sequencer software • Sound synthesizer • MIDI keyboard COMBINING SOUND 30 ADDING SOUND TO MULTIMEDIA PROJECT • File formats compatible with multimedia authoring software being used along with delivery mediums, must be determined. • Sound playback capabilities offered by end user’s system must be studied. • The type of sound, whether background music, special sound effects, or spoken dialog, must be decided. • Digital audio or MIDI data should be selected on the basis of the location and time of use. AUDIO RESOLUTION MONO AND STEREO PRODUCTION TIPS Recording on inexpensive media rather than directly to disk prevents the hard disk from being overloaded with unnecessary data. The equipment and standards used for the project must be in accordance with the requirements. It is vital to maintain a high-quality database that stores the original sound material. Sound and image synchronization must be tested at regular intervals. SUMMARY • Vibrations in air create waves of pressure that are perceived as sound. • Multimedia system sound is digitally recorded audio or MIDI (Musical Instrumental Digital Interface) music. • Digital audio data is the actual representation of a sound, stored in the form of samples. • MIDI is a shorthand representation of music stored in numeric form. • Digital audio provides consistent playback quality. • MIDI files are much smaller than digitized audio. • MIDI files sound better than digital audio files when played on high-quality MIDI device. Digital Multimedia, 2nd edition Nigel Chapman & Jenny Chapman REFERENCES Multimedia Making It Work, Tay Vaughan, 2014, 9th Edition, McGraw Hill. https://yslaiseblog.wordpress. com/ https://optimus.keycdn.com/s upport/lossy-vs-lossless/ 36