Lecture 6 ELEC 421 DSP LV-merged.pdf

Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing ELEC 421 Digital Signal and Image Processing Siamak Najarian, Ph.D., P.Eng.,...

Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing ELEC 421 Digital Signal and Image Processing Siamak Najarian, Ph.D., P.Eng., Professor of Biomedical Engineering (retired), Electrical and Computer Engineering Department, University of British Columbia Siamak Najarian 2024 1  Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing Course Roadmap for DSP Lecture Title Lecture 0 Introduction to DSP and DIP Lecture 1 Signals Lecture 2 Linear Time-Invariant System Lecture 3 Convolution and its Properties Lecture 4 The Fourier Series Lecture 5 The Fourier Transform Lecture 6 Frequency Response Lecture 7 Discrete-Time Fourier Transform Lecture 8 Introduction to the z-Transform Lecture 9 Inverse z-Transform; Poles and Zeros Lecture 10 The Discrete Fourier Transform Lecture 11 Radix-2 Fast Fourier Transforms Lecture 12 The Cooley-Tukey and Good-Thomas FFTs Lecture 13 The Sampling Theorem Lecture 14 Continuous-Time Filtering with Digital Systems; Upsampling and Downsampling Lecture 15 MATLAB Implementation of Filter Design Siamak Najarian 2024 2  Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing Lecture 6: Frequency Response Siamak Najarian 2024 3  Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing Table of Contents Proving the convolution property of the Fourier Transform The frequency response: the Fourier Transform of the impulse response Series of systems in the frequency domain Interpreting the frequency response: the action of the system on each complex sinusoid A real LTI system only changes the magnitude and phase of a real cosine input An LTI system cannot introduce new frequencies Introduction to filters Example: frequency response for a one-sided exponential impulse response Computing outputs for arbitrary inputs using the frequency response Partial fractions A more complicated example Using the Fourier Transform to solve differential equations Convolution in the frequency domain is multiplication in the time domain Siamak Najarian 2024 4  Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing Proving the convolution property of the Fourier Transform If we have a signal, y(t), that is the convolution of two other signals, x(t) and h(t), then there (1) is an FT property that tells us that the FT of y(t), Y(ω), is simply the product of the two FTs, as shown in (1). Let us prove this. In (2), we are going to do the standard trick of moving around the two integrals. So, we are going to interchange. This means that we can take out the terms that do not depend on t. Now, we see that (*) looks like an FT itself, i.e., (*) the FT of h(t - τ). This FT is abbreviated as curly F of the time-domain signal of h(t - τ). (2) sub in (3) (3) Siamak Najarian 2024 5  Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing The frequency response: the Fourier Transform of the impulse response The reason that we care about the convolutions like (1) this in the first place, i.e., (1), is that we will be dealing with systems in which x(t) is the input, h(t) is the impulse response of the system, and y(t) is the LTI System output. In that case, we give this H(ω) a special name. If we put the delta function into the system (and again this only works for an LTI system), what comes out is h(t), which we call the impulse response. Now, we define this new thing, which is the FT of h(t) or H(ω), and we call that the frequency response. This is h(t) (2) a very important concept. Because that is what lets us take things that were complicated in the time domain and make them easier in the frequency domain. This H(ω) (3) is why sometimes we will interchangeably use either of the block diagrams shown in (2) or (3). In diagram (3), in some references, the input and output are still shown in time-domain, i.e., x(t) and y(t), instead of X(ω) and Y(ω). We will use both methods of representation. Here, using either notation, it should be clear that h(t) is the impulse response of the system and H(ω) is the frequency response of the system. Siamak Najarian 2024 6  Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing Series of systems in the frequency domain We can now put it all together. Let us h1(t) h2(t) (1) suppose we put the signal, x(t), through two impulse responses, (1). We can equivalently H1(ω) H2(ω) (2) talk about putting that signal through two frequency responses, (2). In the frequency (3) domain, the output, Y(ω), is going to be the input X(ω) times one frequency response, H1(ω), times the other frequency response, H2(ω) H1(ω) H2(ω), as represented by (3). (4) In (3), we know that the order does not h2(t) h1(t) matter, since this is just a multiplication. There is no problem interchanging these two things. This is another way of saying that if we interchange the two impulse responses, we get the same result. In (4), the two block diagrams are equivalent. Siamak Najarian 2024 7  Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing Interpreting the frequency response: the action of the system on each complex sinusoid What does the frequency response mean? Let us put a complex exponential into our system. This exponential has a fixed frequency ω0. One way to think about this x(t) is like a constant amplitude signal, 1, times the exponential, or (1) x(t) = 1.exp(jω0t) = exp(jω0t). * What is X(ω)? That is, what would be the FT of P x(t)? It is just going to be a delta function. We We used the following from earlier:(2) can find that the FT of x(t) or X(ω) is just a ** shifted delta function, shown in (1). (3) When k = 1, we get (1). (4) Siamak Najarian 2024 8  Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing Interpreting the frequency response: the action of the system on each complex sinusoid Let us find out how the system responds to this shifted delta function, i.e., how does the system respond to X(ω), and how to get Y(ω), shown in (2)? In (3), we are multiplying the delta function, (1) 2π.𝛅(ω-ω0), shown in Graph **, by whatever the * complex number of H(ω0) is (shown by point P in P Graph *). (2) As shown in Graphs * and **, this is like we have ** some kind of arbitrary frequency response H(ω0) and we are multiplying it by a delta function that is shifted to fire at ω0. This leads us to the equation of Y(ω) = H(ω0).2π.𝛅(ω - ω0), (3). inverse FT (3) In (4), we undo the frequency domain to get back into time domain and to find the (4) corresponding output of the system, y(t). Here, we did an inverse FT on (3). Siamak Najarian 2024 9  Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing Interpreting the frequency response: the action of the system on each complex sinusoid What is the meaning of equation (4)? This mean that if we put a complex exponential in, with a certain frequency ω0, then what comes out is the same complex exponential just multiplied by some complex number. So, we (1) are not changing anything about the * frequencies in this signal. The only thing that P is happening is we are taking that complex (2) exponential, and then multiplying it by some ** scaler. That scaler, H(ω0), has a magnitude and an angle. Here, we are not changing the fundamental frequency content of the signal. inverse FT Conclusion: The key idea is that the system (3) cannot introduce new frequencies into the output that were not present in the input. (4) Siamak Najarian 2024 10  Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing A real LTI system only changes the magnitude and phase of a real cosine input In the complex world, concepts are a little bit abstract. So, let us make things little more (1) concrete by assuming that h(t), the impulse response, is real. What happens now is that (2) since the impulse response is real, we can use the symmetry properties of the Fourier transform that we disused before. Here, if we put a cosine, shown by (1), into this real-valued impulse response system, what comes out is the very same cosine multiplied by the magnitude of the frequency response and phase-shifted by the angle, shown by (3). Here, we used equation (2), which is complex IFT conjugate property. So, the same cosine comes (3) out perhaps, amplified or attenuated and shifted around. magnitude of the frequency response So, this is a different way of thinking about if the signal is made up of a real decomposition For a real-valued signal h(t), the Fourier transform H(ω) of cosines and sines, then we cannot get any satisfies the conjugate symmetry property: other cosines and sines out that were not present originally. Siamak Najarian 2024 11  Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing An LTI system cannot introduce new frequencies Example: As an example, if cos(3t) goes in and cos(5t) comes out, we can immediately conclude this system is not an LTI system. Because, we have introduced a different frequency that was not present in the original signal. Here, there was no cosine or sine of 5t in the original part, so this cannot be LTI. Siamak Najarian 2024 12  Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing Introduction to filters What is usually happening is that we are taking frequencies of the input and then we are damping down or fully removing certain frequencies. That is why we call it a filter because only certain things are passing through. The most important of these filter is the lowpass filter. Frequency Response: As pointed out before, the frequency response H(ω) of a system is a function that describes how the system The equation describing a lowpass filter in the frequency processes different frequency components domain is the frequency response of the filter. It tells us how of an input signal. It provides information different frequencies are modified by the filter, either being about how the system modifies both the passed (retained) or attenuated. In this context, the frequency amplitude and phase of each frequency response of the filter provides the relationship between the component of the input. Specifically, the input and output for different frequency components. frequency response represents the Fourier transform of the system's impulse response h(t). In other words, if h(t) is the impulse response of a system, then its Fourier transform H(ω) is the frequency response. Siamak Najarian 2024 13  Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing Introduction to filters Let us sketch a lowpass filter in the frequency (1) domain (1). We have a frequency response that looks like H(ω). The height is 1 in the range between -ωc and +ωc and 0 elsewhere. The result of applying H(ω) to some input signal that has some arbitrary FT of X(ω) (shown in (2)) (2) (*) is that it cuts out all the frequencies that are higher than ωc. So, what we get at the end is something like (3). (4) In terms of terminology, the passband and the stopband are shown in (1). What is under the (3) P1 P2 P 3 filter is the passband and what is outside of it is a stopband. What would be the corresponding impulse response of the system? That is, how H(ω) and h(t) are related? We know that a pulse in one world corresponds to a sinc function in the other world. So, we would expect that the corresponding impulse response is a sinc function (*) related to the cut-off frequency, shown in (4). In h(t) graph, we have an evenly-spaced set of zero crossings and these zero-crossings occur at multiples of π/ωc, shown by P1, P2, and P3. Siamak Najarian 2024 14  Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing Introduction to filters (1) More on h(t) graph: Graph (4) illustrates an important tradeoff in filter design. Graph (1) is often exactly what we want in the frequency domain, something that cuts out only what we (2) (*) want and throws away exactly all the rest. But if we think about how we are going to actually implement the filter in Graph (1) in the time (4) domain, then Graph (4) is not so great. Why? (3) P1 P2 P 3 Siamak Najarian 2024 15  Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing Introduction to filters (1) Reasons why (4) is not great: (a) It is not causal. It goes in both directions. That means we are looking into the future in order to filter the signal corresponding to the pieces of the impulse response that are to the left of the t-axis (2) (*) (shown by blue line). (b) The other thing is the wiggliness (oscillations). Even if we were to somehow truncate things, Graph (4) goes out infinitely far in both directions and so we would (4) have to eventually stop at some point. We do P1 P2 P 3 not want to be using a huge delay looking back (3) into time for thousands of samples to be able to do our filtering. Also, it turns out that this wiggliness in the time domain causes some problems in the output. That is, if we were to use an approximation of this in order to filter a signal, this wiggliness in the time domain will make some rippliness in the output signal that we may not want. Note: Everything we discussed here is still in the continuous world, whereas in real life, we have to design digital filters that approximate H(ω). Siamak Najarian 2024 16  Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing Example: frequency response for a one-sided exponential impulse response Example: What would be a good approximation to an ideal lowpass filter? If we compare (A) with (B), there is a similarity. There is no negative Decaying exponential part in (A), but there is this trend that the filter (A) (B) values get smaller as we go out in time. The FT graph of (A) is shown in (C). Let us see whether or not (C) would make a good lowpass filter. For the moment, let us not worry about the phase. Typically, for filtering, the first thing we care about is the magnitude of the frequency response. We usually look at the (C) absolute value of things, i.e., 𝐇(𝛚). Also, we care about the phase. We are going to talk about phase when we discuss filter design. Siamak Najarian 2024 17  Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing Example: frequency response for a one-sided exponential impulse response In (C), as ω increases, we are falling off. So, the magnitude of this filter goes to zero as we go (A) Decaying exponential (B) further up. Hence, this is a fairly crude lowpass filter. Now, how could we make the fall off of this filter wider or thinner? That depends on the value of “a” that we choose. The value of “a” is related to the width of the filter in (C). Larger a: The decay is slower, causing the curve (C) to stretch horizontally (making it wider). Smaller a: The decay is faster, resulting in a narrower curve. Width Effect on Width: The width of the graph refers to how far along the ω-axis the significant part of the curve extends. When a is larger, the response decays more gradually, making the graph broader and extending further along the ω-axis. When a is smaller, the response decays more sharply, leading to a narrower width. Siamak Najarian 2024 18  Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing Computing outputs for arbitrary inputs using the frequency response What is the general setup to solving LTI systems? Usually, step 3 is the hardest part where we have got to take the inverse FT. Siamak Najarian 2024 19  Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing Partial fractions Example 1: Let us go over some examples on how we can actually solve LTI systems using the setup we just discussed. Here, we use the method of partial fractions, where we hypothesize that we could (1) decompose Y(ω), (1), in terms of the two simple fractions. IFT Siamak Najarian 2024 20  Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing A more complicated example Example 2: (3) * *** (2) ** (1) Note: When we compare (1) with (2) to obtain the set of equation shown by (3), we perform the process of term matching. Term * corresponds to the constant term, which is 2 (shown by **), and *** corresponds to 1, since we have (1)jω in (1). The term (B + C) in (2) is equal to 0, because there is no jω2 term up there in the numerator of (1). Siamak Najarian 2024 21  Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing Using the Fourier Transform to solve differential equations Techniques to make solving differential equations easier: 1) Example 3: Siamak Najarian 2024 22  Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing Convolution in the frequency domain is multiplication in the time domain (*) 2) A B Example 4: We showed that convolution in the time domain is the same thing as multiplication in the frequency domain. The same thing applies if we flip it around because of the Principle of Duality. So, we could say that multiplication in the time domain is the same as convolution in the frequency domain. Following this and as shown in z(t) = x(t)y(t), there is a property that says that if we have a product in the time domain, then in the frequency domain, we have a convolution. The place where this comes up the most is when we are doing operations like amplitude modulation. In this example, let us say we have a cosine function in time domain that represents y(t). Then, in the frequency domain, where we have the Z(ω), what we get is the convolution of the original signal, X(ω), (Graph A) convolved with a cosine (Graph B). So, the result of this operation is two half-height copies of X(ω) that are centered at the modulation frequency of ω0. Note that equation (*) is a general equation and is valid for any two signals x(t) and y(t). Siamak Najarian 2024 23  Lecture 6: Frequency Response ELEC 421: Digital Signal and Image Processing End of Lecture 6 Siamak Najarian 2024 24  Lecture 5: The Fourier Transform ELEC 421: Digital Signal and Image Processing ELEC 421 Digital Signal and Image Processing Siamak Najarian, Ph.D., P.Eng., Professor of Biomedical Engineering (retired), Electrical and Computer Engineering Department, University of British Columbia Siamak Najarian 2024 1  Lecture 5: The Fourier Transform ELEC 421: Digital Signal and Image Processing Course Roadmap for DSP Lecture Title Lecture 0 Introduction to DSP and DIP Lecture 1 Signals Lecture 2 Linear Time-Invariant System Lecture 3 Convolution and its Properties Lecture 4 The Fourier Series Lecture 5 The Fourier Transform Lecture 6 Frequency Response Lecture 7 Discrete-Time Fourier Transform Lecture 8 Introduction to the z-Transform Lecture 9 Inverse z-Transform; Poles and Zeros Lecture 10 The Discrete Fourier Transform Lecture 11 Radix-2 Fast Fourier Transforms Lecture 12 The Cooley-Tukey and Good-Thomas FFTs Lecture 13 The Sampling Theorem Lecture 14 Continuous-Time Filtering with Digital Systems; Upsampling and Downsampling Lecture 15 MATLAB Implementation of Filter Design Siamak Najarian 2024 2  Lecture 5: The Fourier Transform ELEC 421: Digital Signal and Image Processing Lecture 5: The Fourier Transform Siamak Najarian 2024 3  Lecture 5: The Fourier Transform ELEC 421: Digital Signal and Image Processing Table of Contents The Fourier Transform Thinking of the Fourier series coefficients as samples of a function Deriving the Fourier Transform from a Fourier Series whose period goes to infinity The result: formulas for the forward and inverse Fourier Transform When can we compute the Fourier Transform? Fourier Transform examples Delta function Shifted delta function Delta functions at ± t0 One-sided exponential Pulse Comments on the sinc function A pulse in the frequency domain Periodic signals The relationship between the Fourier Series coefficients and Fourier Transform for a periodic signal Reading off time domain signals from delta functions in the frequency domain Fourier Transform of an impulse train Fourier Transform properties Symmetry properties Differentiation/integration Convolution and preview of frequency response Siamak Najarian 2024 4  Lecture 5: The Fourier Transform ELEC 421: Digital Signal and Image Processing The Fourier Transform Review of Fourier series: We talked about the Fourier series, which only applies to periodic signals. (1) * ** But a lot of times, the signals we care about in real life are not periodic (i.e., aperiodic) and here we will investigate how to handle them using Fourier transform. × Fourier series × (2) × Just as a refresher, the idea of the Fourier series was that we have some of periodic signal with period T. The definition of periodic means it repeats every T units as shown in (1). What we did in Fourie series was to write the signal as the combination of a bunch of other simpler periodic signals, which turned out to be sinusoids as shown in (2). Now, what do we do when our signal is not periodic? An aperiodic signal would look like (1) just without any copies (such as * or **). Siamak Najarian 2024 5  Lecture 5: The Fourier Transform ELEC 421: Digital Signal and Image Processing The Fourier Transform Let us look at an aperiodic or a non-periodic signal. This is just like we have one lonely thing in the middle (see *) and no copies. For the Fourier transform, we can imagine that we have copies that are infinitely far away. The idea is that in the limit, the Fourier series coefficients somehow * become what is called the Fourier transform. For the Fourier series representation, we used (1). What happens when we have a signal where T is very large? When T is very large, that means ω0 is very small and that, (1) in turn, means that the sinusoids that we are adding together are actually very close together in frequency. As T goes to infinity and ω0 goes to 0, the sum will turn into an integral. Eventually, the terms are so finely spaced that we will have a continuous function of frequency shown by the (2) integral (2). What goes in the red box is going to be the Fourier transform. We will derive how that works. Siamak Najarian 2024 6  Lecture 5: The Fourier Transform ELEC 421: Digital Signal and Image Processing Thinking of the Fourier series coefficients as samples of a function We have a pulse train (square wave) the width of which is ±T1. The Fourier series coefficients turned out to be the sinc functions. The right-hand side of (1) says that we are evaluating 2sin(ωT1)/ω at ω = kω0. It means that in order to get the ak’s, we need to take this continuous function of omega, i.e., 2sin(ωT1)/ω and sample it every ω0 units. ( ) (1) Siamak Najarian 2024 7  Lecture 5: The Fourier Transform ELEC 421: Digital Signal and Image Processing Thinking of the Fourier series coefficients as samples of a function We just showed that in order to get the ak’s, we would be sampling 2sin(ωT1)/ω every ω0 units, as shown in graph A. That is a nice way to think about how the Fourier (1) series and the Fourier transform are related. Equation (1) is basically saying that the underlying concept is the Fourier transform and to get the Fourier series, i.e., ak’s, we sample the Fourier A transform at these equally spaced values. Siamak Najarian 2024 8  Lecture 5: The Fourier Transform ELEC 421: Digital Signal and Image Processing Deriving the Fourier Transform from a Fourier Series whose period goes to infinity As T goes to infinity, ω0 goes to zero and samples get infinitely close. So, if we wanted to push the square wave off to infinity, eventually, what we would have would be the samples shown by vertical lines in region **. That is, instead of looking like *, they would look more like **. In the limit, we will be getting every point along this continuous function. Siamak Najarian 2024 9  Lecture 5: The Fourier Transform ELEC 421: Digital Signal and Image Processing Deriving the Fourier Transform from a Fourier Series whose period goes to infinity * Let us mathematically derive how this works. Assume that we begin with a non-periodic signal *. We want the Fourier transform of this signal. What we can do is make a related signal. Let us assume that the signal has finite extent for the moment. This means that there is some T, large enough, so that we can make copies of the signal way out on the t-axis. Now, we can take this x(t) and create periodic copies of it for some T. Let us call this signal 𝐱 𝐭 , x tilde of t. It is related to x(t) but not the same as x(t). The copies do not overlap. Now, we have a signal that we can take the Fourier series of. This is a periodic signal that can (1) be shown by equation (1). Put differently, the signal 𝐱 𝐭 is going to have some Fourier series. What are the ak’s? We need to use the other formula, the analysis formula, to figure out what the ak’s are. Siamak Najarian 2024 10  Lecture 5: The Fourier Transform ELEC 421: Digital Signal and Image Processing Deriving the Fourier Transform from a Fourier Series whose period goes to infinity We can just replace 𝐱 𝐭 with the actual x(t) in the range between –T/2 and +T/2 since these two signals are exactly the same in this range. Now, we can say if we are going to integrate from –T/2 and +T/2, we are assuming the signal is zero outside this range (shown by red oval). So, we do not lose anything by turning the integral from −∞ to +∞. Nothing is lost because all we are doing is just adding zero. Siamak Najarian 2024 11  Lecture 5: The Fourier Transform ELEC 421: Digital Signal and Image Processing Deriving the Fourier Transform from a Fourier Series whose period goes to infinity As pointed out before, we do not lose anything by turning the integral from −∞ to +∞. This is shown in (1). The continuous function of X(kω0) is, in fact, X(ω) that is being evaluated at kω0. So, we see now that the ak’s are related to (1) X(ω). This continuous function acts like an envelope. What we get for the ak’s are samples of this envelope. This is shown in Graph A. Graph A The ak’s are sampling inside this envelope. The sinc signal is like the envelope. Siamak Najarian 2024 12  Lecture 5: The Fourier Transform ELEC 421: Digital Signal and Image Processing Deriving the Fourier Transform from a Fourier Series whose period goes to infinity Now that we have ak’s, (1), we are going to plug them into 𝐱 𝐭 equation, (2), and reconstruct our signal. (1) (2) Let us think about a function that is defined as X(ω) (shown in Graph A). Let us see what * is (i.e., the terms inside the box of sigma). Here, we are summing up the area of the shaded rectangles. The width of the rectangle is ω0 and the height of the rectangle is the continuous function, X(ω).ejωt, sampled at kω0 * (i.e., the term in the square bracket in *). As we make the shaded boxes narrower and narrower, eventually, we will get things that add up to exactly the integral. So, the sum converges to the Graph A integral. The sum of these boxes (or rectangles) is called the Riemann sum. We want to make ω0 very small (or T very large). So, we take the limit of both sides. That is, we find the limit of 𝐱 𝐭 and the limit of the sigma and we let ω0 approach 0. So, the left hand side, the limit of 𝐱 𝐭 , is going to become x(t), which corresponds to pushing the copies infinitely far away, and the right hand side is going to become the integral. Siamak Najarian 2024 13  Lecture 5: The Fourier Transform ELEC 421: Digital Signal and Image Processing The result: formulas for the forward and inverse Fourier Transform Fourier transform (FT) tells us how to go from the time domain to the omega domain and the inverse Fourier transform (IFT) tells us how to go backwards. Now, we can go both backwards and forwards for (1) any finite-length aperiodic signal using these two equations, (1) and (2). The Fourier Transform is not limited to finite-length signals; it applies to (2) infinite-length aperiodic signals as well, provided the signal is absolutely integrable. This concludes the derivation of the Fourier transform. In summary, let us say we have x(t) being a continuous function. If it is periodic, what we get is the Fourier series, which has a discrete set of ak. And if it is not periodic (aperiodic), we get the Fourier transform, which has a continuous X(ω). Later on, we are going to generalize the above formulation. We will discuss about what happens when x(t) is discrete or digital, i.e., when we have x[n] instead of x(t). In DSP context, it turns out that there are also digital equivalents to both of the above equations. We will later learn about what the other two formulations are. Siamak Najarian 2024 14  Lecture 5: The Fourier Transform ELEC 421: Digital Signal and Image Processing When can we compute the Fourier Transform? Property #1: We should have an integral where the energy converges. We have to have finite energy signal. For example, we cannot have a signal like * that is aperiodic and infinite area. Property #2: It is about having a finite number of * extrema (maxima and minima). This is the equivalent of saying not wiggling infinitely much. Property #3: It has to have a finite number of discontinuities. These are basically the same things we discussed when dealing with Fourier series. The Fourier series and the Fourier transform are really the same thing. The Fourier series is like a special case of the Fourier transform in some sense. We do allow the Fourier transform of periodic signals. Siamak Najarian 2024 15  Lecture 5: The Fourier Transform ELEC 421: Digital Signal and Image Processing Fourier Transform examples; Delta function; Shifted delta function Examples for FT: Example 1: It is just saying what happens when we integrate with the delta function. The integral basically corresponds to looking at the places where the delta function is firing. So, the FT of the delta function is a constant. Example 2: What happens if we shift the delta function? It is like shifting where the delta function fires. Now, all that is going to happen is that the delta function, instead of firing at t = 0, is firing at t = t0. Conclusion: Both in Examples 1 and 2, the functions have the same magnitude. The magnitudes are just some point on the unit circle. But, they have different phase. When we time shift the signal, we phase shift the FT, the same way we phase shift the FS coefficients. We are not really changing the contribution of things. All we are doing is changing when these Note: cosines and sines start. Siamak Najarian 2024 16  Lecture 5: The Fourier Transform ELEC 421: Digital Signal and Image Processing Delta functions at ± t0; One-sided exponential x(t) (A) Example 3: In (A), we want to see what happens if we have sums of delta functions. So, when we have two t impulses like the ones shown, we get a cosine. ( ) Example 4: In (B), we have an exponential function that turns on at zero and is decreasing. The equation for X(ω) is similar to that of the Laplace transform of the signal. Just a note that as the Fourier transform is x(t) (B) more general than the Fourier series, the Laplace transform is more general than the Fourier transform. t Siamak Najarian 2024 17  Lecture 5: The Fourier Transform ELEC 421: Digital Signal and Image Processing Pulse Example 5: One signal that is really important is the pulse (not repeated), just a regular pulse. The result tells us that a pulse in the time- domain corresponds to a sinc function in the frequency-domain and vice versa. Siamak Najarian 2024 18  Lecture 5: The Fourier Transform ELEC 421: Digital Signal and Image Processing Comments on the sinc function What does sinc(ω) look like? The function sinc(ω) is like sin(ω)/ω. Here, if we draw them separately, 1/ω is looks like (A) and (A) sin(ω) looks like (B). When we multiply these (B) two things together, 1/ω will act like an envelope that modulates (or multiplies) the sin(ω) that is oscillating inside it. The sinc sin function has a central peak at ω = 0 and 2π sidelobes that decay on either side. π (C) 0 The function sin(ω)/ω crosses zero (i.e., the x- axis) in the same places that sin(ω) crosses the x-axis. This occurs when we have π, 2π, 3π and so on. Basically, these are multiples of π. As we approach ω = 0, the function sin(ω)/ω approaches 1. The result is shown in (C). Finally, because our target is to sketch sinc(ωT1), X(ω) = 2.T1.sinc(ωT1) 2.T1 (D) we need to scale the x-axis by 1/T1, as shown in (D). Here, the zeros are function of T1. ω Siamak Najarian 2024 19  Lecture 5: The Fourier Transform ELEC 421: Digital Signal and Image Processing A pulse in the frequency domain An example of inverse Fourier transform: Duality Property: We showed that if we Example: have a pulse in the time domain, we get a sinc in the frequency domain. In this example, we have a pulse in the frequency domain, and as shown, we get a sinc in the time domain. This is not a coincidence and is called Duality. In table (A), if we have a delta function in the time domain, then in the frequency domain, we get a constant. In a similar way, if we have a delta function in the frequency domain, we get a constant in the time domain. A summary of two examples are shown in tables (A) and (B). T: Time domain F: Frequency domain Another example, that we had before, is that a pair of impulses in the time domain (B) corresponds to a cosine in the frequency (A) domain. And, in the same way, a cosine in This is the case shown above. the time domain corresponds to a pair of impulses in the frequency domain. Siamak Najarian 2024 20  Lecture 5: The Fourier Transform ELEC 421: Digital Signal and Image Processing Periodic signals So far, none of the functions for which we did the FT has been periodic. But, we mentioned that we are also allowed to do the FT of signals that are Example: x(t) periodic. So, let us see what happens if we have a periodic signal. Here, we can take either the FS or the FT. FT and FS are closely related. t The impulse train is an important signal used in the sampling theorem. When we do the FS, in the integral, in the range of –T/2 to +T/2, the only place that delta function fires is at 0, where its value is 1. So, we just plug in 0 for t in exp(-jk2πt/T). Eventually, for all the ak’s, the impulse train gives us a constant for these FS coefficients, i.e., 1/T. Later on, we will discuss how to find the FT of this impulse train. That is, we will finish this example at a later time and after we fully cover some additional concepts. Siamak Najarian 2024 21  Lecture 5: The Fourier Transform ELEC 421: Digital Signal and Image Processing The relationship between the Fourier Series coefficients and Fourier Transform for a periodic signal What is the general equation for the computation of the Fourier transform of a periodic signal? That is, if we have x(t), (1), in time-domain, how can we get X(ω), which is in the frequency-domain? To do this, we can use equation (2). Conclusion: The coefficients in FS, i.e., ak, are FS related to the coefficients in FT, i.e., 2πak. That is, to (1) get the coefficients in FT, we just multiply the coefficients in FS by 2π. So, the Fourier transform of a periodic function results in a discrete spectrum of (2) delta functions at harmonics of the fundamental frequency ω0. In other words, the Fourier transform FT of a periodic function does not yield a continuous function but rather a sum of delta functions (spikes) at these harmonic frequencies. Siamak Najarian 2024 22  Lecture 5: The Fourier Transform ELEC 421: Digital Signal and Image Processing The relationship between the Fourier Series coefficients and Fourier Transform for a periodic signal Let us suppose we have a signal whose FT looks like (A). What is the corresponding x(t)? The impulses are spaced apart by ω0. So, X(ω) is the sum of whatever the height of the (A) impulse is, i.e., 2π.ak, times the delta function spaced out at multiples of ω0. The key thing is understanding what the inverse FT of the shifted delta function is, shown by *. After that, we can find x(t). * Siamak Najarian 2024 23  Lecture 5: The Fourier Transform ELEC 421: Digital Signal and Image Processing The relationship between the Fourier Series coefficients and Fourier Transform for a periodic signal (1) Let us just take one of the delta functions, i.e., just 𝛅 𝛚 − 𝐤𝛚𝟎 , shown in (1) and find the IFT corresponding x(t). (2) Equation (2) is the IFT or the inverse Fourier transform of the shifted delta function. As shown in *, if we have a series of impulses (which may or may not be the same height in the frequency domain), that are also equally * spaced, when we take that back in the time domain, what we get is a periodic signal. We can interpret the heights of these impulses (2πak) as being read off from our FS coefficients. That is, the FS coefficients are the heights of those impulses that have been divided by 2π. An example is the height 2πa0 in Graph A. The Graph A corresponding coefficient in x(t) will be a0 , i.e., a0 = (2πa0)/(2π). Siamak Najarian 2024 24  Lecture 5: The Fourier Transform ELEC 421: Digital Signal and Image Processing Reading off time domain signals from delta functions in the frequency domain Example: Given X(ω) find A x(t). pulse train in frequency domain Sample calculation for a1: From X(ω) graph, at ω0, the height of the impulse is 2πa1. Solving for a1, 2πa1 = π, we get a1 = 1/2. The above reconfirms what we already knew. We knew that when we had a cosine in one domain (here, cos(ω0t)), we got the two impulses in the other domain. So, moving backwards, if we start from x(t) = cos(ω0t), this is like saying we can take the cosine, compute the Fourier series of it, put those Fourier coefficients on top of the impulses in X(ω), and eventually, we can get the Fourier transform, X(ω). Siamak Najarian 2024 25  Lecture 5: The Fourier Transform ELEC 421: Digital Signal and Image Processing Fourier Transform of an impulse train Example: Here, we revisit the impulse train and will find X(ω). Given that ak = 1/T, the heights of the impulses in X(ω) are going to be 2π.ak or 2π/T. We showed this before. Siamak Najarian 2024 26  Lecture 5: The Fourier Transform ELEC 421: Digital Signal and Image Processing Fourier Transform of an impulse train Graph A Graph B In summary, if we have an impulse train in the time domain (Graph A), where the impulses are spaced apart by T, what we have in the frequency domain is also an impulse train where the impulses are spaced apart by 2π/T (Graph B). The wider the impulses are in Graph A, the closer they are together in Graph B. This makes sense because the wider the impulses in Graph A are (i.e., if T is getting really large), the more it is pushing it out. So, it is not periodic at all anymore. So, eventually, we just have one impulse in the middle and nothing else in Graph A. The corresponding thing that would happen in Graph B would be that the impulses would all push together so closely that they become a constant. We already knew that the delta function in the time domain (i.e., a single delta function and not an impulse train) corresponds to a constant in the frequency domain. Siamak Najarian 2024 27  Lecture 5: The Fourier Transform ELEC 421: Digital Signal and Image Processing Fourier Transform properties In some references instead of X(ω), they use X(jω). We prefer X(ω) since we usually Inverse Fourier Transform prefer to plot X(ω) vs. ω. Forward Fourier Transform Property #1 (Linearity): The FT exhibits linearity. This means that if we take the FT of a sum of signals, ax(t) + by(t), it is equivalent to taking the FT of each signal individually and then summing the results: FT [ax(t) + by(t)] = FT [ax(t)] + FT [by(t)] = aX(ω) + bY(ω). So, the FT is a linear operator. Siamak Najarian 2024 28  Lecture 5: The Fourier Transform ELEC 421: Digital Signal and Image Processing Symmetry properties Property #2 (Time Shift): If we take a signal and we shift it, the thing that happens to its FT is a phase shift. The magnitude of (1) is the same as whatever the (1) magnitude was before, as shown in (2). So, all we are (2) doing is shifting the phase of the sinusoids in the integral that we add up. We are not really shifting any : of the important content. Property #3 (Symmetry Properties): Let us say we have X(ω) and it is equal to a real part plus an imaginary part. If x(t) is real, then we have some useful properties that tell us the real part is even and the imaginary part is odd. Additionally, the magnitude of X(ω) is even and the angle of X(ω) is odd. So, for real signals, given that Graph A |X(ω)| = |X(-ω)| for all frequencies ω, we do not have to plot the FT for all values of ω. Since the magnitude is even, we usually do not bother plotting the stuff on the left hand side because we know it is just the mirror image across the y-axis, as shown in Graph A. Siamak Najarian 2024 29  Lecture 5: The Fourier Transform ELEC 421: Digital Signal and Image Processing Symmetry properties Property #4 (Even and Odd Parts): We can show that the even part of x(t), or Ev(x(t)), corresponds to the real 4) part of X(ω), or Re(X(ω)). Also, the odd part of x(t), or Od((x)t)), corresponds to the imaginary part of X(ω), jIm(X(ω)). Additionally, if x(t) is real and even, then X(ω) is also real and even. And, if x(t) is real and odd, then X(ω) is purely imaginary and odd. The applications of the above is that we can predict some simple things about a signal without actually having to compute the FT if we recognize the relationship between the even parts and the odd parts. Siamak Najarian 2024 30  Lecture 5: The Fourier Transform ELEC 421: Digital Signal and Image Processing Differentiation/integration Property #5 (Differentiation and Integration): In (1), there is just a little extra bit (*) that could 5) happen at ω = 0. This could be a DC offset that we get when we do the integration. similar to. Property #6 (Time Scaling): If we scale the signal, x(at), what we get is a scaled version of the FT, (1) going the other way, i.e., 1/a. So, if we shrink the time domain signal, we spread out the frequency (*) domain signal, and vice versa. 6) Property #7 (Duality): Duality says when we have a pulse in one world, (A), it corresponds to a sinc in the other world, (B). Then if something (A) happens in one domain, we can relate it to what (B) 7) FT pair happens in the other domain. That is, we can switch them and be sure that it will work out. It is FT pair like a simple change of variable. Duality implies that all the properties we just wrote down also applies to the other variables (relating t to ω). The duality property highlights the inherent connection between a function's behavior in the time and frequency domains. It allows us to predict the frequency content of a signal based on its time-domain shape (and vice versa) for certain functions. Siamak Najarian 2024 31  Lecture 5: The Fourier Transform ELEC 421: Digital Signal and Image Processing Convolution and preview of frequency response (1) Property #8 (Parseval Theorem): This says that the integral (1) in the time domain (that gives us 8) the total energy) is the same as the total energy in the frequency domain (except for this 2π factor). So, basically, it is saying that we have the same total energy either way and that we do not 9) lose anything. Property #9 (Convolution): Convolution says that if y(t) is x(t) convolve with h(t), then Y(ω) is X(ω) times H(ω). This basically says that if we have an LTI system and we put a signal through it that has a certain impulse response, we can have an h(t) equivalent way of representing this system with the frequency response, H(ω), shown in block diagram (2). Multiplying things together in the H(ω) (2) frequency domain and taking the inverse for a transform is generally a lot easier than manually doing the time domain convolution. Given that our system is assumed to be LTI, this suddenly makes our life much easier! Siamak Najarian 2024 32  Lecture 5: The Fourier Transform ELEC 421: Digital Signal and Image Processing End of Lecture 5 Siamak Najarian 2024 33  Lecture 4: The Fourier Series ELEC 421: Digital Signal and Image Processing ELEC 421 Digital Signal and Image Processing Siamak Najarian, Ph.D., P.Eng., Professor of Biomedical Engineering (retired), Electrical and Computer Engineering Department, University of British Columbia Siamak Najarian 2024 1  Lecture 4: The Fourier Series ELEC 421: Digital Signal and Image Processing Course Roadmap for DSP Lecture Title Lecture 0 Introduction to DSP and DIP Lecture 1 Signals Lecture 2 Linear Time-Invariant System Lecture 3 Convolution and its Properties Lecture 4 The Fourier Series Lecture 5 The Fourier Transform Lecture 6 Frequency Response Lecture 7 Discrete-Time Fourier Transform Lecture 8 Introduction to the z-Transform Lecture 9 Inverse z-Transform; Poles and Zeros Lecture 10 The Discrete Fourier Transform Lecture 11 Radix-2 Fast Fourier Transforms Lecture 12 The Cooley-Tukey and Good-Thomas FFTs Lecture 13 The Sampling Theorem Lecture 14 Continuous-Time Filtering with Digital Systems; Upsampling and Downsampling Lecture 15 MATLAB Implementation of Filter Design Siamak Najarian 2024 2  Lecture 4: The Fourier Series ELEC 421: Digital Signal and Image Processing Lecture 4: The Fourier Series Siamak Najarian 2024 3  Lecture 4: The Fourier Series ELEC 421: Digital Signal and Image Processing Table of Contents The Fourier Series Assumption: x(t) is periodic with period T Complex exponentials with period T Interpreting the Fourier Series sum The Fourier Series definition Deriving the formula for the {ak} The result of the derivation Symmetries in {ak} for real x(t) Different forms of the Fourier Series for real signals Fourier Series examples Fourier Series for a pulse train The sinc function Fourier Series applet When can we not compute the Fourier Series? Discontinuities and the Gibbs phenomenon Properties of the Fourier Series (time shift, differentiation, Parseval, convolution) Siamak Najarian 2024 4  Lecture 4: The Fourier Series ELEC 421: Digital Signal and Image Processing The Fourier Series We noticed that convolution process can be tedious. The great strength of LTI signals is the ability to use what are called transform methods to make solving LTI systems easier. That means we bring the input, the system, and the output into a new domain which is going to be either the Fourier domain or the z-transform domain. In Fourier analysis we do the decomposition of a signal into sines and cosines. Sinusoids naturally occur in a lot of situations, such as swinging pendulums, spinning wheels, and communications theory. Cellphones, AM/FM radio are built on things like carrier waves that are high- frequency sinusoids. LTI systems respond to sinusoidal inputs in a particularly special way. That It is very important to know what the Fourier Series is and is why we use Fourier Transforms in the first place. how it works. This is because the DFT (Digital Fourier Transform) is like a discrete version of the Fourier Series. Every periodic continuous-time signal can be written as a sum of sinusoids. We want to see how this works. Siamak Najarian 2024 5  Lecture 4: The Fourier Series ELEC 421: Digital Signal and Image Processing Assumption: x(t) is periodic with period T The first question is what other signals are periodic. x(t) The most natural signal that has the period T is a sinusoid (Graph A). Graph A Fundamental Frequency: Here, ω0 is called the fundamental frequency. This is because if we plug in t t = 1.T in (1), we get cos(2π), which is 1. If we plug in t = 2.T, we get cos(4π), which is also 1, and so on. (1) Put differently, the fundamental frequency of a sinusoid is the lowest frequency component Graph B present in the signal. In Graph B, the cosine wiggles twice as fast. We can also have sin(ω0t), since it is basically a cosine function that has shifted over to the right. It also has a period of T. Both cos(kω0t) and sin(kω0t) are periodic and have the same period of T. In cos(kω0t) = cos[k(2π/T)t], the angular frequency is scaled by a factor of k (k being an integer greater than 1). This effectively increases the frequency, causing the function to oscillate faster. Siamak Najarian 2024 6  Lecture 4: The Fourier Series ELEC 421: Digital Signal and Image Processing Complex exponentials with period T Siamak Najarian 2024 7  Lecture 4: The Fourier Series ELEC 421: Digital Signal and Image Processing Interpreting the Fourier Series sum Interpretation of ak’s: Coefficients (ak’s) could be any complex number. Multiplying the signal by these coefficients, such as by 2 or 3 or by 1 + j, does not fundamentally change its period; it just changes its amplitude and phase. The interpretation is that as we increase k, the signal wiggles more and more. ( ) ( ) Siamak Najarian 2024 8  Lecture 4: The Fourier Series ELEC 421: Digital Signal and Image Processing Deriving the formula for the {ak} Our goal is to represent x(t) in the form of sum of terms. This is called the Fourier series. Synthesis equation: It means how we synthesize the signal from the ak’s. So, if we are (Synthesis equation) given the ak’s, the synthesis equation shows us how we add up the ak’s to get back to the x(t). Next, we need formulas for ak’s to get x(t). The synthesis equation in Fourier series is used to reconstruct a periodic function from its Fourier