Multimedia IS411P Lecture 02 - Digital Audio

Multimedia (IS411P) Lecture Notes

• Course: Multimedia (IS411P)
• Faculty: Computers & Information Science
• Department: Information Systems
• Academic Year: 2024-2025
• Lecture: 02
• Date: 15/10/2024

Digital Audio

• Amplitude: Determines sound volume, measured in decibels (dB).
• Period: Time between two crests (peaks) in a sound wave, measured in seconds.
• Frequency (pitch): Number of peaks per second, measured in Hertz (Hz).
• Bandwidth (BW): Difference between highest and lowest frequencies in a signal.
• Wavelength (λ): Distance from midpoint of one crest to the midpoint of the next.
• Human Hearing Range: 20 Hz to 20 kHz.
• Human Maximum Sensitivity Range: 2 kHz - 4 kHz.
• Velocity of Sound: Varies with medium, but can be determined by measuring the time required for sound waves to travel a given distance. Values for various mediums are shown in the table on page 5.

Doppler Effect

• Sound waves are compressions and rarefactions of air.
• When a sound source moves towards a listener, the frequency increases.
• When a sound source moves away from a listener, the frequency decreases.

Harmonics

• Most vibrating objects produce complex sounds.
• Harmonic series: frequencies are whole number multiples of a fundamental frequency.
• A complex sound wave is comprised of various frequencies.

Basic Characteristics of Audio Signal

• Audio is caused by disturbances in air pressure.
• Frequency range of audible sound: 20-20,000 Hz.
• Amplitude: dynamic range is large, from audibility threshold to pain threshold; measured in decibels (dB).

Digital Representation of Audio

• Continuous audio waveforms converted to an electrical signal by a microphone.
• Analog signals converted to digital signals for processing/communication (ADC, Analog to Digital Converter). Three stages of ADC: sampling, quantization, coding.

Sampling

• Continuous time converted to discrete values.
• Time axis divided into fixed intervals.
• Value of analog signal taken at the start of each interval.
• Determined by sampling rate (sampling frequency)

Quantization

• Process of converting continuous sample values to discrete values.
• Signal's range divided into fixed intervals, each interval has a number.
• The size of each interval is referred to as the quantization step.

Coding

• The process of representing quantized values digitally (e.g. using binary values).
• Eight quantized levels can be represented via 3 bits.

Digital-to-Analog Converter (DAC)

• Reconstructs original analog signal from digital data.
• Each quantized value is held for the sampling interval, resulting in step signals.
• Step signals passed through a low-pass filter to approximate the original signal.

Signal-to-Noise Ratio (SNR)

• Measures signal quality in decibels (dB).
• Defined as: SNR = 20 log₁₀(S/N), where S is the maximum signal amplitude and N is the quantization noise.

Nyquist Theorem

• Two samples per cycle (per wave) are necessary to represent a given wave.
• To represent a 440 Hz sound, sampling rate must be 880 samples per second at minimum.
• Sampling rate= 2 x Highest Frequency.

Aliasing

• Distortion that happens when a signal has frequencies greater than half the sampling rate.
• A serious problem in sampling systems, cannot be removed by post-processing after recording.
• High frequencies filtered out prior to sampling for prevention.
  • falias = fsampling - ftrue for ftrue < fsampling < 2 x ftrue

Quality of Sound

• Telephone conversation bandwidth = 3300 Hz.
• CD-ROM sampling rate = 44 kHz per channel.
  • CD ROMs are important media for multimedia.

Sound Formats

• Stereo recordings (2 channels) are more realistic, but take twice the storage space of mono recordings.

• Formulas to calculate storage space for mono/stereo recordings:

  • Mono: File size = Sampling rate * Duration * (bits per sample/8) * 1
  • Stereo: File size = Sampling rate * Duration * (bits per sample/8) * 2

Use of Audio in Multimedia

• Audio can serve as either content (dialogues, instructions) or ambient (background music, sound effects).

Video

• Delivery of information per second is greater than with any other multimedia element.
• DVDs make large video distribution easier, similar to the ease of analog-to-digital conversion of audio using CDs.

Analog Video

• Only analog video is used in broadcasting for now.
• Some movies/video may be digitally processed prior to broadcast.
• Three analog video standards globally: NTSC, PAL, SECAM.

Analog Video Standards

• NTSC (National Television System Committee): 30 frames/sec, 525 lines, 16 million colors.
• PAL (Phase Alternation by Line): 25 frames/sec., 625 lines
• SECAM (Sequential Couleur Avec Mémoire): 25 frames/sec., 625 lines

Digital Video

• Increasing trend towards digital video, even in consumer electronics.
• Digital video is easy to access and edit.
• Video editing involves various operations: removing/inserting frames, mixing audio, etc.

Timecode

• Unit for measuring video clip duration.
• Can serve as a frame address.
• SMPTE standard for timecode: hours:minutes:seconds:frames

Digitizing Analog Video

• Video capture cards accept video input from devices, audio is sampled separately, software synchronizes both.
• Cards must support appropriate frame rates (e.g., 30 fps) to avoid issues.

Keyframes

• A complete image frame, unlike frames that only show differences.
• Serve as reference points in video; rest of the frames rely on them to reconstruct the full picture.
• Keyframe interval indicates how often a keyframe appears in a video stream.

Compression

• Restructuring data to reduce file size.
• Video files are compressed during capture, decompressed during playback.
• Various codecs (compression/decompression algorithms) are available for digital videos.
• Important codec characteristics: whether they are symmetric or asymmetric.

Factors Affecting Compression

• Frames per second (fps)
• Number of keyframes
• Data rate for playback

File Formats for Video

• AVI (PC) and QuickTime (Macintosh) are common formats for saving edited digital videos.
• Both utilize similar video compression/decompression strategies.
• Conversion programs are available.

Video on the Internet

• Streaming video enables real-time video transmission via the Internet.
• Quality depends on factors like bandwidth, sound intensity/frequency, and difference in information between successive frames.

Surround Video

• Provides photorealistic visuals in Web pages, enabling real-time navigation around a 360-degree image.

Quiz (Example Questions)

• Total frames in a clip with a given timecode and frames per second are given.
• Uncompressed video file size calculation given video dimensions, color depth, and frames per second.
• Given fps and keyframe interval, calculating number of keyframes in a second.

This quiz covers key concepts from Lecture 02 of the Multimedia (IS411P) course, focusing on digital audio fundamentals. Topics include amplitude, frequency, bandwidth, and the Doppler Effect. Test your knowledge on the properties of sound and human hearing ranges.