Digital Signal and Image Processing Lecture Notes PDF

Digital Signal and Image Processing Lecture 1 – Introduction © Dr. T. Akilan (Ph.D., P.Eng., SMIEEE) 1 Logistics Meeting time: o Lectures: MW 11:30 AM - 01:00 PM @ AT-2021 o Recitation: M 02:30 PM - 04:00 PM...

Digital Signal and Image Processing Lecture 1 – Introduction © Dr. T. Akilan (Ph.D., P.Eng., SMIEEE) 1 Logistics Meeting time: o Lectures: MW 11:30 AM - 01:00 PM @ AT-2021 o Recitation: M 02:30 PM - 04:00 PM @ AT-4020 TA: Frost, Russell ([email protected]) Course Evaluations: o 3/4 Assignments with reports - 15 o Project - 20 o Mid-term test - 25 o Final examination - 30 o Pop quizzes and attendance - 10 Total 100 © Dr. T. Akilan (Ph.D., P.Eng., SMIEEE) 2 Logistics Cont. Prerequisites: o Computer Science-1431 o Mathematics-1210 o Python OpenCV MATLAB © Dr. T. Akilan (Ph.D., P.Eng., SMIEEE) 3 Logistics Cont. Communication: Emails to the course instructor and the TA/GA must be appropriate. Response time: o Your professors want you to know that they simply cannot always answer a message quickly. o Allow them a day or two, or even more, to respond. o Note: Dr. Akilan’s best response time is < 2 hrs. © Dr. T. Akilan (Ph.D., P.Eng., SMIEEE) 4 Logistics Cont. Project: o Form a group of 2 (max). o Explore the projects listed in the documents. o Work actively to meet all the submission deadlines described in the project description. o Follow the evaluation rubrics. Observed Common Mistakes © Dr. T. Akilan (Ph.D., P.Eng., SMIEEE) 5 Academic Advising: Common Mistakes Imbalance: Work-study priority and schedule Overdependence: Group tasks, degree projects Last-minute.com Image adapted from https://www.dreamstime.com/ Fail to seek help for academic support and advice ahead of time © Dr. T. Akilan (Ph.D., P.Eng., SMIEEE) 6 In This Session Introduction to Signals o Examples of signals o Properties of signals o Analog and digital signals Digital Images o Digital Image Processing MATLAB © Dr. T. Akilan (Ph.D., P.Eng., SMIEEE) 7  Signals Sensory data from the real world: o Sound waves o Seismic vibration a quantity that o Visuals or videos varies in time o Etc. DSP: o The mathematics, the algorithms, and the techniques used to manipulate the signals (after analog-to-digital conversion) for useful applications. o E.g. image enhancement, speech recognition, and generation, data compression for storage and transmission. Signal → Applications Areas of DSP © Dr. T. Akilan (Ph.D., P.Eng., SMIEEE) 8  Few Application Areas of DSP Image and Space Diagnostic sound Oil and mineral imaging (CT, Earthquake photograph compression for Voice and data prospecting MRI, ultrasound, recording & enhancement multimedia compression Radar (Flight, analysis etc.) (WhatsApp) ship target presentation Process (Zoom) detection) Data monitoring & Data acquisition compression Electrocardiogram Echo reduction control analysis (EEG, Movie special Ordnance EOG) effects guidance Spectral analysis Intelligent Signal Nondestructive sensory analysis multiplexing testing by remote space Medical image Video Secure Simulation and storage/retrieval communication probes conference modeling Filtering CAD and tools calling (Skype) Space Medical Commercial Telephone Military Industrial Scientific Applications Areas of DSP → Signal Examples © Dr. T. Akilan (Ph.D., P.Eng., SMIEEE) 9  Examples of Signals # 1 Signals can be mathematically defined. These mathematical functions are signals that vary with the independent variable 𝑥 𝒔𝟏 (𝒙) = 𝟓𝒙 𝒔𝟐 (𝒙) = 𝟐𝒙² − 𝟑𝒙 + 𝟐 𝒔𝟑 (𝒙) = 𝐬𝐢𝐧( 𝟒𝟎𝒙) What is the one commonality of all these signals? Examples of Signals # 2 → © Dr. T. Akilan (Ph.D., P.Eng., SMIEEE) 10  Examples of Signals # 2 Sound is a signal that varies over time s4(t) = “Windows makes it easy for several users to share a computer” s5(t) = The Microsoft “tada” sound Examples of Signals # 3 → © Dr. T. Akilan (Ph.D., P.Eng., SMIEEE) 11  Examples of Signals # 3 Signals can vary with several independent variables − x²− y ² s6 ( x, y ) = xe horizontal plane Examples of Signals # 4 → © Dr. T. Akilan (Ph.D., P.Eng., SMIEEE) 12  Examples of Signals # 4 Altitude (s7) is a signal that varies over longitude and latitude → Type of Signals & Properties © Dr. T. Akilan (Ph.D., P.Eng., SMIEEE) 13  Type of Signals and Properties Continuous-time Signal: a signal that can be defined at every instant of time. o E.g., voltage, velocity, pressure, temperature Discrete-time Signal: can be defined at a discrete instant of time and is called a discrete-time signal. o E.g., pixels, a digitized representation of continuous-time signals, i.e., quantized value X[n] at sampled independent value, n. (discrete in time and quantized in amplitude; e.g., daily average temperature) Digital Signal: The value of the function is discrete (not continuous values, unlike the top two figures. It can take a value from a predefined finite set of amplitudes). o It can be a discrete or continuous-time signal. o That is, all digital signals are not discrete-time signals. o The digital signals are usually indicated by a square wave. o Binary signal – Special case of the digital signal; the signal is represented by binary bits (the amplitude is either 1 or 0) Is this continuous-/discrete-time signal? → Type of Signals & Properties © Dr. T. Akilan (Ph.D., P.Eng., SMIEEE) 14  Useful References Tamal Bose, Digital Signal and Image Processing, Wiley 2004, ISBN: 978-0-471-32727-1 Digital Image Processing, 4th Edition, Richard E. Woods, 2018 , Pearson. Understanding Digital Signal Processing, Third Edition, Richard G. Lyon © Dr. T. Akilan (Ph.D., P.Eng., SMIEEE) 15 Digital Signal and Image Processing Lecture 2 – Introduction Cont. Dr. T. Akilan [email protected] © Dr. T. Akilan (Ph.D., P.Eng., SMIEEE) 16 This Week’s Plan Lab Group formation & Project proposal Pop quiz © Dr. T. Akilan (Ph.D., P.Eng., SMIEEE) 17 This Session Recap o Pop quiz Introduction to Signals Cont. o Examples of signals o Properties of signals: periodic, aperiodic, deterministic, randomness, etc. o Analog and digital signals Digital Images o Digital Image Processing MATLAB © Dr. T. Akilan (Ph.D., P.Eng., SMIEEE) 18 Recap – Pop quiz # 1 What is a signal? Define and give 3 examples. Compare and contrast the following three signals: oContinuous-time signal oDiscreate-time signal oDigital signal © Dr. T. Akilan (Ph.D., P.Eng., SMIEEE) 19  Type of Signals and Properties: Periodic Signal Periodic Signal: A signal is said to be periodic if it Example of a periodic signal repeats itself after some amount of time: 𝒙(𝒕 + 𝒏𝑻) = 𝒙(𝒕) 2𝑇 for some value of 𝑇, where 𝑇 > 0 and 𝑛 is an integer. 𝑡 o 𝑇 - period of the signal (the minimum value of time) for which the signal exactly repeats itself. (Michael D. Adams) o Two quantities closely related to the period are the frequency (f) and angular frequency (ω): 1 2𝜋 𝑓 = , and 𝜔 = 2𝜋𝑓 =. 𝑇 𝑇 𝑦1 𝑡 = 𝐴 ⋅ 𝑠𝑖𝑛 𝜔𝑡 + 𝜑 ; 𝜑 – specifies a phase-shift 𝑦2 𝑡 = 𝐴 ⋅ 𝑠𝑖𝑛(𝜔𝑡 + 𝜋/2) Aperiodic Signal: Signal which does not repeat itself after a certain period is called aperiodic signal. → Periodic Signal Simulation (image: https://undergroundmathematics.org/) © Dr. T. Akilan (Ph.D., P.Eng., SMIEEE) 20  Type of Signals & Properties: Periodic Signal Sim. 𝑠𝑖𝑛 𝑡 𝑡 90o 180o 270o 360o 𝜋/2 𝜋 3𝜋/2 2𝜋 → Type of Signals & Properties (image: https://alex-hhh.github.io/2021/01/plot-animations.html) © Dr. T. Akilan (Ph.D., P.Eng., SMIEEE) 21  Type of Signals and Properties – Periodic Signal Rotation 𝜔 = 2𝜋𝑓 rads/s 90 +𝑨𝒎 60 𝐴 𝑡 = 𝐴𝑚 sin(𝜔𝑡 + 𝜑) 30 𝝎𝒕 0 360 𝑡 330 𝝎𝒕 300 270 180o 270o 360o −𝑨𝒎 𝜋 3𝜋/2 2𝜋 The Signal in the Angular The Sinusoidal Waveform in the Frequency (ω) Domain Time Domain Deterministic & Random Signals → (image: https://alex-hhh.github.io/2021/01/plot-animations.html) © Dr. T. Akilan (Ph.D., P.Eng., SMIEEE) 22  Type of Signals & Properties: Deterministic Signal Deterministic signals: a signal is deterministic if it is exactly predictable for the time span of interest. o Described by mathematical models. o E.g., the signal of the electromotive force (voltage) is described by: Voltage (V) 𝑨 V 𝑡 = 𝐴 × 𝑠𝑖𝑛 𝜔𝑡 , where Time (t) V 𝑡 - the signal over time,  in Sec. 𝐴 – amplitude, 𝜔 - the model parameter, and 𝑡 – a point in time,  - the period o Recall: 𝜔 = 2𝜋𝑓 , where 𝑓 is the A deterministic signal is one frequency of the signal. that can be described So, V 𝑡 = 𝐴 × 𝑠𝑖𝑛 2𝜋𝑓𝑡 mathematically Deterministic Signal E.g. → © Dr. T. Akilan (Ph.D., P.Eng., SMIEEE) 23  Type of Signals & Properties: Deterministic Signal E.g. Deterministic signals E.g.: o s1, s2, s3, and s6 are functions of independent variables that follow known formulae 𝒙 𝒚 𝒔𝟏 (𝒙) = 𝟓𝒙 𝒔𝟐 (𝒙) = 𝟐𝒙² − 𝟑𝒙 + 𝟐 𝒔𝟑 (𝒙) = 𝐬𝐢𝐧( 𝟒𝟎𝒙) 𝒔𝟔 𝒙, 𝒚 = 𝒙. 𝒆−𝒙²−𝒚² Random Signals → © Dr. T. Akilan (Ph.D., P.Eng., SMIEEE) 24  Type of Signals & Properties: Random Signals Random Signal cannot be described by any mathematical function. o Sometimes the mathematical function is too complex, unknown, or does not exist o E.g.: different realizations of discreate and continuous time random signals Statical measures: Amplitude Mean Median Time (t) Mode Range Amplitude Sample ID (n) Random Signals Example → © Dr. T. Akilan (Ph.D., P.Eng., SMIEEE) 25  Type of Signals & Properties: Random Signal e.g. Random (stochastic) signals E.g.: o s4, s5, and s7 are functions of independent variables, but there is no exact mathematical formulae modelling them. s5(t) = The Microsoft “tada” sound Image Source: http://www.bramboroson.com/astro/jan30.html 𝑠7 𝑙𝑜𝑛, 𝑙𝑎𝑡 = Altitude is a signal of a landscape wrt longitude and latitude Causal and Anti-causal Signal → © Dr. T. Akilan (Ph.D., P.Eng., SMIEEE) 26 Type of Signals & Properties: Causality Causal and Anti-causal Signal: Signals that have a zero value for all negative times are called causal signals, while the signals that are zero for all positive values of time are called anti-causal signals. 𝑥(𝑡) 𝑥(𝑡) 𝑡 𝑡 Causal signal Anti-causal signal Non-causal: a signal that has nonzero values in both positive and negative time. 𝑥(𝑡) 𝑡 → Odd & Even Signals © Dr. T. Akilan (Ph.D., P.Eng., SMIEEE) 27 Type of Signals & Properties: Even and Odd Signal Even and Odd Signal: Even signal 𝑥(𝑡) Odd signal 𝑥(𝑡) 𝑡 𝑡 o Even signal is any signal '𝑥' such that 𝑥(𝑡) = 𝑥(−𝑡). o Odd signal is a signal '𝑥' for which 𝑥(𝑡) = −𝑥(−𝑡). o Even signals are symmetric around the vertical axis. o Even signal is one that is invariant under the time scaling: 𝑡 → − 𝑡 o Odd signal is one that is invariant under the amplitude and time scaling: 𝑥(𝑡) → −𝑥(−𝑡). Recall: Analog and Digital Signals → © Dr. T. Akilan (Ph.D., P.Eng., SMIEEE) 28  Quick Recap Independent variables: Signals can vary over any number of independent variables o E.g.,: s1 to s5 varied over a single variable, s6 and s7 varied over two variables o The signals s1, s2, and s3 are function of 𝑥 o The signal s6 is the function of (𝑥, 𝑦) o Most commonly, they vary over time, as was the case for the sound signals s4 and s5 o Signals can also vary over spatial position, such as s7 Information: Signals can represent any information o s4 and s5 represent sound o s7 represents geographical altitude o s1, s2, s3, and s6 represent the value of a function → Analog & Digital © Dr. T. Akilan (Ph.D., P.Eng., SMIEEE) 29 Quick Recap: Analog & Digital All the signals we’ve seen mostly are analog Signal (A) o They are functions of continuous variables: 𝑓(𝑥, 𝑦) o E.g., 𝑓(𝑥 = 𝑡𝑖𝑚𝑒, 𝑦 = 𝑝ℎ𝑦𝑠𝑖𝑐𝑎𝑙 𝑝𝑜𝑠𝑖𝑡𝑖𝑜𝑛) What could be a signal that is a function of time and position? Signal (B) Alternative is a digital signal o Function of discrete variables o A variable that takes its value from a finite or countably infinite set Analog to Digital → © Dr. T. Akilan (Ph.D., P.Eng., SMIEEE) 30  Analog to Digital Signals What we know: Most signals of practical interest are analog in nature o E.g. Voice, Video, RADAR signals, Transducer/Sensor output, Biological signals, etc. o They carry lots of useful information about a (physical) phenomenon o So, to utilize those benefits, we need to convert analog signals into digital o This process is called A/D conversion A/D Conversion can be elaborated with a three-step process: Sampler Quantizer Coder o Sampling, o Quantization, and o Coding. Sampler → © Dr. T. Akilan (Ph.D., P.Eng., SMIEEE) 31  Analog to Digital Signals – Step 1: Sampling Conversion from continuous-time, continuous-valued signal to discrete-time, continuous-valued signal The simplified sampling process E.g. → © Dr. T. Akilan (Ph.D., P.Eng., SMIEEE) 32  Analog to Digital Signals – Step 1: Sampling Cont. Analog signal Discrete-time signal Sampler Ts Ts – Step size What is the step size in this case? 𝑠2 (𝑥) = 2𝑥² − 3𝑥 + 2 𝑠2 (𝑥𝑠 ) = 2𝑥𝑠 ² − 3𝑥𝑠 + 2 𝑥 ∈ [0,1] 𝑥𝑠 ∈ {0,0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9,1} Conversion from continuous-time, continuous-valued signal to discrete-time, continuous-valued signal The simplified sampling process © Dr. T. Akilan (Ph.D., P.Eng., SMIEEE) 33  Analog to Digital Signals – Sampling Cont. Why do we need to sample? o Recall: Analog signal is continuous in time and continuous in amplitude, i.e., it has some value defined at every time instant, so it has an infinite number of sample points. o Issue: It means that it carries infinite information of time and infinite information of amplitude. ▪ It is impossible to digitize an infinite number of points: The infinite points cannot be processed by the digital signal processor or computer, since they require an infinite amount of memory and an infinite amount of processing power for computations. o Solution: Sampling is the process of reducing the time information or sample points. © Dr. T. Akilan (Ph.D., P.Eng., SMIEEE) 34  Analog to Digital Signals – Sampling Cont. Sampling can take samples at a nth sample = x[n] = 𝑥(𝑛𝑇) fixed time interval. Continuous time analog signal 𝑥(𝑡) o The length of the sampling Sampling interval T interval is called the sampling period (Ts) o The reciprocal of the sampling period is the sampling frequency 𝟏 fs: 𝒇𝒔 =. 𝑻𝒔 If 𝒙(𝒕) is the input to the sampler, the output is 𝒙(𝒏𝑻𝒔 ), where 𝑻𝒔 is called the sampling period, and 𝑛 is the sample ID Sample & hold: Once a sample is acquired, the sampler holds the sampled value steady for the remainder of the sampling interval. o The hold time is needed to allow time for an A/D converter to generate a digital code that best corresponds to the analog sample. E.g. of various sampling intervals → © Dr. T. Akilan (Ph.D., P.Eng., SMIEEE) 35  Analog to Digital Signals: Sampling Cont. (a) continuous-time, and continuous- (b) valued signal Ts = 1 sec (c) (d) Ts = 0.1 sec Ts = 1 sec → E.g. of Sample and Hold (S/H) © Dr. T. Akilan (Ph.D., P.Eng., SMIEEE) 36  Analog to Digital Signals: Sample and Hold Sample and hold e.g.: Analog voltage to Digital conversion o Each sample maintains its voltage level during the sampling interval 𝑻 to give the ADC enough time to convert it. Simplest option: Hold the 𝑉(𝑡) Voltage for ADC current value for one sample Analog voltage period Ts o Good enough approximation for many applications Better idea: o Interpolate based on past 𝑛𝑇 few samples What is the optimal sampling period or sampling frequency? → Optimal Sampling Period © Dr. T. Akilan (Ph.D., P.Eng., SMIEEE) 37 Analog to Digital Signals Often, we have to reconstruct the analog signal after the required processes are completed in the digital domain. The quality of the reconstruction largely depends on the sampling frequency. Typical A/D and D/A process flow: Input Analog- Input Digital Output Digital-to- Output analo to-digital digital signal digital analog analog signal g converter signal processor signal converter signal E.g.: Digital Edit, Computer Live Digital Modified video upload, screen & Play-back show copy copy camera download speakers → Reconstruction © Dr. T. Akilan (Ph.D., P.Eng., SMIEEE) 38 Digital to Analog Conversion: Signal reconstruction Signal reconstruction: The opposite of sampling Multiple options: Interpolate between (subset of) samples Example: straight line between pairs of samples Curve-fitting if the original signal followed a mathematical function → Reconstruction E.g. © Dr. T. Akilan (Ph.D., P.Eng., SMIEEE) 39 Sampling and Reconstruction It was easy to perform the signal reconstruction with s2 What about s3? s3 ( x) = sin( 40 x) s3 (nTs ), Ts = 0.1 Reconstructed s3 What went wrong? © Dr. T. Akilan (Ph.D., P.Eng., SMIEEE) 40 Sampling and Reconstruction Cont. We did not take enough samples (10 samples/Sec.) Without sufficient samples, we could not accurately reconstruct the signal How do we know how many samples are required? → Sampling theorem © Dr. T. Akilan (Ph.D., P.Eng., SMIEEE) 41  Useful References Tamal Bose, Digital Signal and Image Processing, Wiley 2004, ISBN: 978-0-471-32727-1 Digital Image Processing, 4th Edition, Richard E. Woods, 2018 , Pearson. © Dr. T. Akilan (Ph.D., P.Eng., SMIEEE) 42 Next up Nyquist–Shannon Sampling Theorem © Dr. T. Akilan (Ph.D., P.Eng., SMIEEE) 43