Speech & Music PDF

SPEECH Sounds of Speech: Phonemes Phoneme: smallest units of sound that if changed would change the meaning of a word International Phonetic Alphabet (IPA) where each symbol stands for a different speech sound Producing the Sounds of Speech Most speech sounds begin with an exhalation of air f...

SPEECH Sounds of Speech: Phonemes Phoneme: smallest units of sound that if changed would change the meaning of a word International Phonetic Alphabet (IPA) where each symbol stands for a different speech sound Producing the Sounds of Speech Most speech sounds begin with an exhalation of air from the lungs ◦ Trachea = windpipe ◦ Larynx = voice box Vocal folds = vocal cords Can be relaxed and open, allowing air to pass silently Can be tensed, causing them to vibrate when air passes Frequency of vocal fold vibration depends on size and thickness of the vocal folds and size/shape of larynx Male = 85-180 Hz Female = 165-255 Hz ◦ Pharynx: upper part of throat ◦ Uvula: flap of tissue at the posterior edge of the soft palate Speech sounds divided into 1. Vowels: ◦ Vowels: unrestricted flow of air through the pharynx and oral cavity ◦ Contain harmonics (multiples of fund. frequencies) ◦ Modifying the position of the jaw to have different shapes for the oral cavity determines resonance and frequencies ◦ Formants: frequency bands with relatively high amplitude in the harmonic spectrum of a vowel sound 2. Consonants: ◦ Consonants: restricted flow of air at one place or another along the path of the airflow from the vocal folds: 1. Place of articulation: point at which this restriction occurs and the anatomical structures 2. Manner of articulation: nature of the restriction 3. Voicing: whether the vocal folds are vibrating or not Perceiving the Sounds of Speech Difficulties in translating speech to phonemes in brain 1. No 1-to-1 correspondence between speech and phonemes ◦ Different talkers produce sounds with different fundamental frequencies ◦ Sounds of specific phonemes differ in different dialects ◦ Same phonemes produced at different times can differ greatly Whisper, shout, speak, sing, having a cold, etc. 2. Indistinct boundaries between words in the sound stream of normal speech ◦ Pauses between each words Coarticulation and Perceptual Constancy Coarticulation: influence of 1 phoneme on the acoustic properties of another, due to the articulatory movements required to produce them in seuqence Articulation for the final vowel influences the first and vice versa Categorical Perception of Phonemes Categorical perception: perception of different sensory stimuli as identical, up to a point at which further variation in the stimulus leads to a shrp change in the perception Sharp changes in perception as the stimulus varies Voice onset time (VOT): in production of stop consonants, the interval between the initial burst of frequencies and the onset of voicing Phonemic boundary: VOT at which a stop consonant transitions from being mainly perceived as voices to being mainly perceived as voiceless ◦ Transition near phonemic boundary is very abrupt Behind categorical perception "detectors" in auditory system tuned to respond to certain ranges of VOTs Perception of voiceless versus voiced stop consonants is categorical and is based on VOT Other dimensions of speech are also categorical ◦ Don’t change with small variations in the timing or frequency content Vision and Speech Perception: McGurk Effect Vision helps reinforce auditory perception McGurk effect: when auditory and visual stimuli conflict, the auditory system tends to compromise on a perception that shares features with both the seen and the heard stimuli ◦ If no good compromise is available, either the conflict is resolved in favour of visual stimulus or there is a conflicting perceptual experience Knowledge and Speech Perception Knowledge takes 3 forms 1. Syntax and Semantics: Knowledge of the grammatical rules of the language and the context in which an utterance is produced 2. Word Segmentation: Knowledge about the probability of various sequences of phonemes within words or across words in the language they're hearing Phoneme transition probabilities: the chances that the sequence occurs at the start, in the middle, at the end of a word of across the boundary between 2 words 3. Perception Completion: Knowledge of the specific words that are expected in a particular situation ◦ Phonemic restoration: perceptual completion in which listeners seem to perceive obscured or missing speech sounds Can be affected by visual context Brain Pathways for Speech Perception and Production Aphasia: impairment in speech production or comprehension (or both) caused by damage to speech centers in the brain (left hemisphere) Broca: speech production Wernicke: speech comprehension ◦ To A1 ◦ Through dorsal and ventral pathways Ventral: WHAT PATHWAY meanings of words and combination of words Dorsal: WHERE/HOW Motor system MUSIC Dimensions of Music: Pitch, Loudness, Timing, and Timbre PITCH Most fundamental dimension of music Clearly distinguishes 1 musical composition from another Octave: sequence of notes where the fundamental frequency of the last note is x2 the fundamental frequency of the 1st note (13 notes) ◦ Tone chroma: differences in pitch within an octave ◦ Tone height: octave in which a tone appears which increases from bottom to the top Semitones: 12 proportionally equivalent intervals between the notes in an octave ◦ Semitone intervals are perceptually equivalent to one another LOUDNESS AND TIMING Dynamics: manner in which loudness varies as a piece of music progresses Rhythm: temporal patterning of events in a musical composition Tempo: overall pace Beat: equally spaced pulses Meter: temporal patterning TIMBRE Timbre: difference between complex sounds that have the same pitch (same fundamental frequency) and same loudness that don’t sound the same Melody Melody: sequence of musical notes arranged in a particular rhythmic pattern, which listeners perceive as a single unit  Transpositions: 2 versions of the same melody with the same intervals but starting at different notes ◦ Perceptually equivalent Scales and Keys: Consonance and Dissonance Scale: particular subset of the notes in an octave Start and end with the note its named after Different pattern of intervals = minor vs major ◦ Different emotional tones Consonance: quality exhibited by a combination of 2+ notes from a scale that sounds pleasant (go together) Dissonance: quality exhibited by a combination of 2+ notes from a scale that sound unpleasant (off) Key: scale that functions as the basis of musical composition Harmonicity: harmonics of notes played in combination coincide with the harmonics of a note with a lower fundamental frequency Knowledge and Music Perception Perception of music depends on accumulated knowledge ◦ Implicit knowledge of how language works + about what to expect Neural Basis of Music Perception Primary auditory cortex (A1) LEFT Representing fine differences in timing RIGHT Representing fine differences in pitch Amusia: impairment in perceiving and remembering melodies and in distinguishing one melody from another Thickening of the right inferior frontal cortex Right auditory cortex Application Automatic speech recognition: accurate perception of human speech by machines