Lecture 6: ELEC 421 Digital Signal and Image Processing PDF

Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing ELEC 421 Digital Signal and Image Processing Siamak Najarian, Ph.D., P.Eng.,...

Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing ELEC 421 Digital Signal and Image Processing Siamak Najarian, Ph.D., P.Eng., Professor of Biomedical Engineering (retired), Electrical and Computer Engineering Department, University of British Columbia Siamak Najarian 2024 1  Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing Course Roadmap for DSP Lecture Title Lecture 0 Introduction to DSP and DIP Lecture 1 Signals Lecture 2 Linear Time-Invariant System Lecture 3 Convolution and its Properties Lecture 4 The Fourier Series Lecture 5 The Fourier Transform Lecture 6 Frequency Response Lecture 7 Discrete-Time Fourier Transform Lecture 8 Introduction to the z-Transform Lecture 9 Inverse z-Transform; Poles and Zeros Lecture 10 The Discrete Fourier Transform Lecture 11 Radix-2 Fast Fourier Transforms Lecture 12 The Cooley-Tukey and Good-Thomas FFTs Lecture 13 The Sampling Theorem Lecture 14 Continuous-Time Filtering with Digital Systems; Upsampling and Downsampling Lecture 15 MATLAB Implementation of Filter Design Siamak Najarian 2024 2  Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing Lecture 6: Frequency Response Siamak Najarian 2024 3  Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing Table of Contents Proving the convolution property of the Fourier Transform The frequency response: the Fourier Transform of the impulse response Series of systems in the frequency domain Interpreting the frequency response: the action of the system on each complex sinusoid A real LTI system only changes the magnitude and phase of a real cosine input An LTI system cannot introduce new frequencies Introduction to filters Example: frequency response for a one-sided exponential impulse response Computing outputs for arbitrary inputs using the frequency response Partial fractions A more complicated example Using the Fourier Transform to solve differential equations Convolution in the frequency domain is multiplication in the time domain Siamak Najarian 2024 4  Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing Proving the convolution property of the Fourier Transform If we have a signal, y(t), that is the convolution of two other signals, x(t) and h(t), then there (1) is an FT property that tells us that the FT of y(t), Y(ω), is simply the product of the two FTs, as shown in (1). Let us prove this. In (2), we are going to do the standard trick of moving around the two integrals. So, we are going to interchange. This means that we can take out the terms that do not depend on t. Now, we see that (*) looks like an FT itself, i.e., (*) the FT of h(t - τ). This FT is abbreviated as curly F of the time-domain signal of h(t - τ). (2) sub in (3) (3) Siamak Najarian 2024 5  Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing The frequency response: the Fourier Transform of the impulse response The reason that we care about the convolutions like (1) this in the first place, i.e., (1), is that we will be dealing with systems in which x(t) is the input, h(t) is the impulse response of the system, and y(t) is the LTI System output. In that case, we give this H(ω) a special name. If we put the delta function into the system (and again this only works for an LTI system), what comes out is h(t), which we call the impulse response. Now, we define this new thing, which is the FT of h(t) or H(ω), and we call that the frequency response. This is h(t) (2) a very important concept. Because that is what lets us take things that were complicated in the time domain and make them easier in the frequency domain. This H(ω) (3) is why sometimes we will interchangeably use either of the block diagrams shown in (2) or (3). In diagram (3), in some references, the input and output are still shown in time-domain, i.e., x(t) and y(t), instead of X(ω) and Y(ω). We will use both methods of representation. Here, using either notation, it should be clear that h(t) is the impulse response of the system and H(ω) is the frequency response of the system. Siamak Najarian 2024 6  Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing Series of systems in the frequency domain We can now put it all together. Let us h1(t) h2(t) (1) suppose we put the signal, x(t), through two impulse responses, (1). We can equivalently H1(ω) H2(ω) (2) talk about putting that signal through two frequency responses, (2). In the frequency (3) domain, the output, Y(ω), is going to be the input X(ω) times one frequency response, H1(ω), times the other frequency response, H2(ω) H1(ω) H2(ω), as represented by (3). (4) In (3), we know that the order does not h2(t) h1(t) matter, since this is just a multiplication. There is no problem interchanging these two things. This is another way of saying that if we interchange the two impulse responses, we get the same result. In (4), the two block diagrams are equivalent. Siamak Najarian 2024 7  Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing Interpreting the frequency response: the action of the system on each complex sinusoid What does the frequency response mean? Let us put a complex exponential into our system. This exponential has a fixed frequency ω0. One way to think about this x(t) is like a constant amplitude signal, 1, times the exponential, or (1) x(t) = 1.exp(jω0t) = exp(jω0t). * What is X(ω)? That is, what would be the FT of P x(t)? It is just going to be a delta function. We We used the following from earlier:(2) can find that the FT of x(t) or X(ω) is just a ** shifted delta function, shown in (1). (3) When k = 1, we get (1). (4) Siamak Najarian 2024 8  Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing Interpreting the frequency response: the action of the system on each complex sinusoid Let us find out how the system responds to this shifted delta function, i.e., how does the system respond to X(ω), and how to get Y(ω), shown in (2)? In (3), we are multiplying the delta function, (1) 2π.𝛅(ω-ω0), shown in Graph **, by whatever the * complex number of H(ω0) is (shown by point P in P Graph *). (2) As shown in Graphs * and **, this is like we have ** some kind of arbitrary frequency response H(ω0) and we are multiplying it by a delta function that is shifted to fire at ω0. This leads us to the equation of Y(ω) = H(ω0).2π.𝛅(ω - ω0), (3). inverse FT (3) In (4), we undo the frequency domain to get back into time domain and to find the (4) corresponding output of the system, y(t). Here, we did an inverse FT on (3). Siamak Najarian 2024 9  Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing Interpreting the frequency response: the action of the system on each complex sinusoid What is the meaning of equation (4)? This mean that if we put a complex exponential in, with a certain frequency ω0, then what comes out is the same complex exponential just multiplied by some complex number. So, we (1) are not changing anything about the * frequencies in this signal. The only thing that P is happening is we are taking that complex (2) exponential, and then multiplying it by some ** scaler. That scaler, H(ω0), has a magnitude and an angle. Here, we are not changing the fundamental frequency content of the signal. inverse FT Conclusion: The key idea is that the system (3) cannot introduce new frequencies into the output that were not present in the input. (4) Siamak Najarian 2024 10  Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing A real LTI system only changes the magnitude and phase of a real cosine input In the complex world, concepts are a little bit abstract. So, let us make things little more (1) concrete by assuming that h(t), the impulse response, is real. What happens now is that (2) since the impulse response is real, we can use the symmetry properties of the Fourier transform that we disused before. Here, if we put a cosine, shown by (1), into this real-valued impulse response system, what comes out is the very same cosine multiplied by the magnitude of the frequency response and phase-shifted by the angle, shown by (3). Here, we used equation (2), which is complex IFT conjugate property. So, the same cosine comes (3) out perhaps, amplified or attenuated and shifted around. magnitude of the frequency response So, this is a different way of thinking about if the signal is made up of a real decomposition For a real-valued signal h(t), the Fourier transform H(ω) of cosines and sines, then we cannot get any satisfies the conjugate symmetry property: other cosines and sines out that were not present originally. Siamak Najarian 2024 11  Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing An LTI system cannot introduce new frequencies Example: As an example, if cos(3t) goes in and cos(5t) comes out, we can immediately conclude this system is not an LTI system. Because, we have introduced a different frequency that was not present in the original signal. Here, there was no cosine or sine of 5t in the original part, so this cannot be LTI. Siamak Najarian 2024 12  Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing Introduction to filters What is usually happening is that we are taking frequencies of the input and then we are damping down or fully removing certain frequencies. That is why we call it a filter because only certain things are passing through. The most important of these filter is the lowpass filter. Frequency Response: As pointed out before, the frequency response H(ω) of a system is a function that describes how the system The equation describing a lowpass filter in the frequency processes different frequency components domain is the frequency response of the filter. It tells us how of an input signal. It provides information different frequencies are modified by the filter, either being about how the system modifies both the passed (retained) or attenuated. In this context, the frequency amplitude and phase of each frequency response of the filter provides the relationship between the component of the input. Specifically, the input and output for different frequency components. frequency response represents the Fourier transform of the system's impulse response h(t). In other words, if h(t) is the impulse response of a system, then its Fourier transform H(ω) is the frequency response. Siamak Najarian 2024 13  Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing Introduction to filters Let us sketch a lowpass filter in the frequency (1) domain (1). We have a frequency response that looks like H(ω). The height is 1 in the range between -ωc and +ωc and 0 elsewhere. The result of applying H(ω) to some input signal that has some arbitrary FT of X(ω) (shown in (2)) (2) (*) is that it cuts out all the frequencies that are higher than ωc. So, what we get at the end is something like (3). (4) In terms of terminology, the passband and the stopband are shown in (1). What is under the (3) P1 P2 P 3 filter is the passband and what is outside of it is a stopband. What would be the corresponding impulse response of the system? That is, how H(ω) and h(t) are related? We know that a pulse in one world corresponds to a sinc function in the other world. So, we would expect that the corresponding impulse response is a sinc function (*) related to the cut-off frequency, shown in (4). In h(t) graph, we have an evenly-spaced set of zero crossings and these zero-crossings occur at multiples of π/ωc, shown by P1, P2, and P3. Siamak Najarian 2024 14  Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing Introduction to filters (1) More on h(t) graph: Graph (4) illustrates an important tradeoff in filter design. Graph (1) is often exactly what we want in the frequency domain, something that cuts out only what we (2) (*) want and throws away exactly all the rest. But if we think about how we are going to actually implement the filter in Graph (1) in the time (4) domain, then Graph (4) is not so great. Why? (3) P1 P2 P 3 Siamak Najarian 2024 15  Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing Introduction to filters (1) Reasons why (4) is not great: (a) It is not causal. It goes in both directions. That means we are looking into the future in order to filter the signal corresponding to the pieces of the impulse response that are to the left of the t-axis (2) (*) (shown by blue line). (b) The other thing is the wiggliness (oscillations). Even if we were to somehow truncate things, Graph (4) goes out infinitely far in both directions and so we would (4) have to eventually stop at some point. We do P1 P2 P 3 not want to be using a huge delay looking back (3) into time for thousands of samples to be able to do our filtering. Also, it turns out that this wiggliness in the time domain causes some problems in the output. That is, if we were to use an approximation of this in order to filter a signal, this wiggliness in the time domain will make some rippliness in the output signal that we may not want. Note: Everything we discussed here is still in the continuous world, whereas in real life, we have to design digital filters that approximate H(ω). Siamak Najarian 2024 16  Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing Example: frequency response for a one-sided exponential impulse response Example: What would be a good approximation to an ideal lowpass filter? If we compare (A) with (B), there is a similarity. There is no negative Decaying exponential part in (A), but there is this trend that the filter (A) (B) values get smaller as we go out in time. The FT graph of (A) is shown in (C). Let us see whether or not (C) would make a good lowpass filter. For the moment, let us not worry about the phase. Typically, for filtering, the first thing we care about is the magnitude of the frequency response. We usually look at the (C) absolute value of things, i.e., 𝐇(𝛚). Also, we care about the phase. We are going to talk about phase when we discuss filter design. Siamak Najarian 2024 17  Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing Example: frequency response for a one-sided exponential impulse response In (C), as ω increases, we are falling off. So, the magnitude of this filter goes to zero as we go (A) Decaying exponential (B) further up. Hence, this is a fairly crude lowpass filter. Now, how could we make the fall off of this filter wider or thinner? That depends on the value of “a” that we choose. The value of “a” is related to the width of the filter in (C). Larger a: The decay is slower, causing the curve (C) to stretch horizontally (making it wider). Smaller a: The decay is faster, resulting in a narrower curve. Width Effect on Width: The width of the graph refers to how far along the ω-axis the significant part of the curve extends. When a is larger, the response decays more gradually, making the graph broader and extending further along the ω-axis. When a is smaller, the response decays more sharply, leading to a narrower width. Siamak Najarian 2024 18  Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing Computing outputs for arbitrary inputs using the frequency response What is the general setup to solving LTI systems? Usually, step 3 is the hardest part where we have got to take the inverse FT. Siamak Najarian 2024 19  Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing Partial fractions Example 1: Let us go over some examples on how we can actually solve LTI systems using the setup we just discussed. Here, we use the method of partial fractions, where we hypothesize that we could (1) decompose Y(ω), (1), in terms of the two simple fractions. IFT Siamak Najarian 2024 20  Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing A more complicated example Example 2: (3) * *** (2) ** (1) Note: When we compare (1) with (2) to obtain the set of equation shown by (3), we perform the process of term matching. Term * corresponds to the constant term, which is 2 (shown by **), and *** corresponds to 1, since we have (1)jω in (1). The term (B + C) in (2) is equal to 0, because there is no jω2 term up there in the numerator of (1). Siamak Najarian 2024 21  Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing Using the Fourier Transform to solve differential equations Techniques to make solving differential equations easier: 1) Example 3: Siamak Najarian 2024 22  Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing Convolution in the frequency domain is multiplication in the time domain (*) 2) A B Example 4: We showed that convolution in the time domain is the same thing as multiplication in the frequency domain. The same thing applies if we flip it around because of the Principle of Duality. So, we could say that multiplication in the time domain is the same as convolution in the frequency domain. Following this and as shown in z(t) = x(t)y(t), there is a property that says that if we have a product in the time domain, then in the frequency domain, we have a convolution. The place where this comes up the most is when we are doing operations like amplitude modulation. In this example, let us say we have a cosine function in time domain that represents y(t). Then, in the frequency domain, where we have the Z(ω), what we get is the convolution of the original signal, X(ω), (Graph A) convolved with a cosine (Graph B). So, the result of this operation is two half-height copies of X(ω) that are centered at the modulation frequency of ω0. Note that equation (*) is a general equation and is valid for any two signals x(t) and y(t). Siamak Najarian 2024 23  Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing End of Lecture 6 Siamak Najarian 2024 24