intro.pptx

Digital Image Processing  Digital image processing deals with the manipulation of digital images through a digital computer.  It is a subfield of signals and systems but focuses particularly on images.  DIP focuses on developing a computer system that can perform processing on an image.  The inp...

Digital Image Processing  Digital image processing deals with the manipulation of digital images through a digital computer.  It is a subfield of signals and systems but focuses particularly on images.  DIP focuses on developing a computer system that can perform processing on an image.  The input of that system is a digital image and the system processes that image using efficient algorithms and gives an image as an output.  The most common example is Adobe Photoshop.  It is one of the widely used applications for In the figure, an image has been captured by a camera and has been sent to a digital system to remove all the other details, and just focus on the water drop by zooming it in such a way that the quality of the image remains the same.  The course teaches widely used methods and procedures for interpreting digital images for image enhancement and restoration and performing operations on images such as blurring, zooming, sharpening, edge detection, etc.  It also focuses on the understanding of how human vision works. How does the human eye visualize so many things, and how does the brain interpret those images?  This chapter slides also cover some of the important concepts of signals and systems such as Sampling, Quantization, Convolution, Frequency domain analysis, etc.  Signal processing is a discipline in electrical engineering and mathematics that deals with the analysis and processing of analog and digital signals, and deals with storing, filtering, and other operations on signals. These signals include transmission signals, sound or voice signals, image signals, other signals, etc.  Out of all these signals, the field that deals with the type of signals for which the input is an image, and the output is also an image is done in image processing. As its name suggests, it deals with the processing of images.  It can be further divided into analog image processing and digital image processing. Analog image processing Analog image processing is done on analog signals. It includes processing twodimensional analog signals. In this type of processing, the images are manipulated by electrical means by varying the electrical signal. A common example includes is the television image. Digital image processing has dominated analog image processing over time due to its wider range of applications. Digital image processing Digital image processing deals with developing a digital system that performs What is an Image An image is nothing more than a twodimensional signal. It is defined by the mathematical function f(x, y) where x and y are the two coordinates horizontally and vertically. The value of f(x, y) at any point gives the pixel value at that point of an image.of a digital image which is The figure is an example nothing but a two-dimensional array of numbers ranging between 0 and 255. 128 30 123 232 123 321 123 77 89 80 255 255 Each number represents the value of the function f(x, y) at any point. In this case, the values 128, 230, and 123 each represent an individual pixel value. The dimensions of the picture are the dimensions of this two-dimensional array. Relationship between a digital image and a signal If the image is a two-dimensional array, then what does it have to do with a signal? To understand this a signal is discussed in detail. Signal In the physical world, any quantity measurable through time over space or any higher dimension can be taken as a signal. A signal is a mathematical function, and it conveys some information. A signal can be a dimensional, two-dimensional, or higher-dimensional signal. One dimensional signal is a signal that is measured over time. A common example is a voice signal. The two-dimensional signals are those that are measured over some other physical quantities. An example of a two-dimensional signal is a digital image. The next slides detail how one-dimensional or two- Relationship Anything that conveys information or broadcasts a message in the physical world between two observers is a signal. That includes speech (human voice) or an image as a signal. When someone speaks, their voice is converted to a sound wave/signal and transformed concerning the time the person is speaking. Not only this, but the way a digital camera works, as acquiring an image from a digital camera involves the transfer of a signal from one part of the system to How a digital image is formed Since capturing an image from a camera is a physical process. The sunlight is used as a source of energy. A sensor array is used for the acquisition of the image. So, when the sunlight falls upon the object, then the amount of light reflected by that object is sensed by the sensors, and a continuous voltage signal is generated by the amount of sensed data. To create a digital image, we need to convert this data into a digital form. This involves sampling and quantization. (discussed later on). The result of sampling and quantization results in a two-dimensional array or matrix of numbers Overlapping fields Machine/Computer vision Machine vision or computer vision deals with developing a system in which the input is an image, and the output is some information. For example: Developing a system that scans the human face and opens any kind of lock. This system would look something like this. Computer graphics Computer graphics deals with the formation of images from object models, rather than the image being captured by some device. For example, Object rendering. Generating an image from an object model. Such a system would look something like this. Artificial intelligence Artificial intelligence is the study of putting human intelligence into machines. Artificial intelligence has many applications in image processing. For example: developing computer-aided diagnosis systems that help doctors in interpreting images of X-rays, MRIs, etc., and then highlighting conspicuous sections to be examined by the doctor. Signal processing Signal processing is an umbrella and image processing lies under it. The amount of light reflected by an object in the physical world (3d world) is passed through the lens of the camera and it becomes a 2D signal and hence results in image formation. This image is then digitized using Signals & Systems Introduction These slides cover the basics of signals and systems necessary for understanding the concepts of digital image processing. Before going into the detailed concepts, let’s first define the simple terms. Signals In electrical engineering, the fundamental quantity representing some information is called a signal. It does not matter what the information is i.e.: Analog or digital information. In mathematics, a signal is a function that conveys some information. Any quantity measurable through time over space or any higher dimension can be taken as a signal. A signal could be of any dimension Analog signals A signal could be an analog quantity which means it is defined concerning the time. It is a continuous signal. These signals are defined over continuous independent variables. They are difficult to analyze, as they carry a huge number of values. They are very accurate due to a large sample of values.  To store these signals, you require an infinite memory because it can achieve infinite values on a real line. Human voice The human voice is an example of an analog signal. When you speak, the voice that is produced travels through the air in the form of pressure waves and thus belongs to a mathematical function, having independent variables of space and time and a value corresponding to air pressure. Another example of a sin wave is shown in the figure below. Y = Sin(x) where x is independent. Digital signals As compared to analog signals, digital signals are very easy to analyze. They are discontinuous signals. They are the appropriation of analog signals. The word digital stands for discrete values and hence it means that they use specific values to represent any information. In a digital signal, only two values are used to represent something i.e.: 0 and 1 (binary values). Digital signals are less accurate than analog signals because they are the discrete samples of an analog signal taken over some time. However Computer keyboard digital signals are not subject to noise. So, they Whenever a key is pressed from the keyboard, the appropriate electrical signaland is sent to the keyboard controller containing the ASCII last longer are easy to interpret. Digital signals value of that particular key. For example, the electrical signal that is are denoted by square waves. generated when keyboard key a is pressed carries information about For example: Computer keyboard Whenever a key is pressed from the keyboard, the appropriate electrical signal is sent to the keyboard controller containing the ASCII value of that particular key. For example, the electrical signal that is generated when keyboard key a is pressed carries information about digit 97 (hexadecimal 61) in the form of 0 and 1, which is the ASCII value of character a. Difference between analog and digital signals Comparison element Analog signal Digital signal Analysis Difficult Possible to analyze Representation Continuous Discontinuous Accuracy More accurate Less accurate Storage Infinite memory Easily stored Subject to Noise Yes No Recording Technique Original signal is preserved Samples of the signal are taken and preserved Examples Human voice, Thermometer, Analog phones etc. Computers, Digital Phones, Digital pens, etc. Systems A system is defined by the type of input and output it deals with. Since we are dealing with signals, in our case, our system would be: a mathematical model, a piece of code/software, a physical device, or a black box whose input is a signal, and it performs some processing on that signal, and the output is a signal. The input is known as excitation and the output is known as response. In the figure, a system has been shown whose input and output both are signals but the input is an analog signal. The output is a digital signal. This is a conversion system that converts analog signals to digital signals. What is in the black box? Conversion of analog to digital signals Since there are many concepts related to this analog to digital conversion and vice-versa. Only related to digital image processing are discussed here. Two main concepts are involved in the conversion. Sampling Quantization Sampling Sampling as its name suggests can be defined as taking samples. Take samples of a digital signal over the x-axis. Sampling is done on an independent variable. In the case of this mathematical equation: Sampling is done on the x variable. The conversion of the x-axis (infinite values) to digital is done undersampling. Sampling is further divided into up-sampling and down-sampling. If the range of values on the x-axis is less, then we will increase the sample of values. This is known as up-sampling and vice versa is known as Quantization Quantization as its name suggests can be defined as dividing into quanta (partitions). Quantization is done on the dependent variable. It is the opposite of sampling. In the case of this mathematical equation y = sin(x) Quantization is done on the Y variable. It is done on the y-axis. The conversion of y-axis infinite values to 1, 0, -1 (or any other level) is known as Quantization. These are the two basic steps that are involved in converting an analog signal to a digital signal. The quantization of a signal has been shown in the figure below. Analog-to-digital converter (ADC) An ADC can be modeled as two processes: sampling and quantization. Sampling converts a voltage signal (function of time) into a discrete-time signal (sequence of real numbers). Quantization replaces each real number with an approximation from a finite set of discrete values (levels), which is necessary for storage and processing by numerical methods. Most commonly, these discrete values are represented as fixed-point words or floating-point words. Common word lengths are 8-bit (256 levels), 16-bit (65,536 levels), 32-bit (4.3 billion levels), and so on, though any number of quantization levels is possible (not just powers of two). Sampling converts the analog signal into a discrete value of samples. The values of these samples depend on the sampling instants. We need to encode each sample value to store it in b bits memory location. But as b is limited, we have to consider finite values of samples. For example If b = 2, we can have 2b=4 different possible sample values. Why do we need to convert an analog signal to digital signal? The first and obvious reason is that digital image processing deals with digital images, that are digital signals. So whenever the image is captured, it is converted into digital format and then it is processed. The second and important reason is, that to perform operations on an analog signal with a digital computer, you have to store that analog signal in the computer. To store an analog signal, infinite memory is required to store it. Since that’s not possible, so that’s why we convert that signal into digital format and then store it in a digital computer, and then perform operations on Continuous systems v/s discrete systems Continuous systems The type of systems whose input and output both are continuous signals or analog signals are called continuous systems. Discrete systems The type of systems whose input and output both are discrete signals or digital signals are called digital systems. History of Photography Origin of camera The history of camera and photography is not the same. The concepts of the camera were introduced a lot before the concept of photography Camera Obscura The history of the camera lies in ASIA. The principles of the camera were first introduced by the Chinese philosopher MOZI. It is known as camera Obscura. The cameras evolved from this principle. The word camera Obscura evolved from two different words. Camera and Obscura. The meaning of the word camera is a room or some kind of vault and Obscura stands for dark. The concept which was introduced by the Chinese philosopher consists of a device, that projects an image of its surroundings on the wall. However, it was not built by the Chinese. The creation of Camera Obscura The concept of Chinese was brought into reality by a Muslim scientist Abu Ali Al-Hassan Ibn alHaitham commonly known as Ibn al-Haitham. He built the first camera Obscura. His camera follows the principles of the pinhole camera. He built this device somewhere around 1000. Portable camera In 1685, the first portable camera was built by Johann Zahn. Before the advent of this device, the camera consisted of a size of room and was not portable. Although a device was made by an Irish scientist Robert Boyle and Robert Hooke that was a transportable camera, still that device was very huge to carry it from one place to the other. Origin of photography Although the camera Obscura was built in 1000 by a Muslim scientist. But its first actual use was described in the 13th century by an English philosopher Roger Bacon. Roger suggested the use of the camera for the observation of solar eclipses. Da Vinci His camera follows the principle of a pinhole camera which can be described as: When images of illuminated objects penetrate through a small hole into a very dark room you will see [on the opposite wall] these objects in their proper form and color, reduced in size in a reversed First photograph The first photograph was taken in 1814 by a French inventor Joseph Nicephore Niepce. He captures the first photograph of a view from the window at Le Gras, by coating the pewter plate with bitumen and after that exposing that plate to light. First underwater photograph The first underwater photograph was taken by an English mathematician William Thomson using a water-tight box. This was done in 1856. The origin of film The origin of film was introduced by an American inventor known as George Eastman who is considered as the pioneer of photography. He founded the company called Eastman Kodak, which is famous for developing films. The company started manufacturing paper film in 1885. He first created the camera Kodak and then later Brownie. Brownie was a box camera and gained After the advent of feature the film, popularity due to its of the camera industry once Snapshot. again got a boom and one invention led to another. Leica and Argus Leica and Argus are the two analog cameras developed in 1925 and 1939 respectively. The camera Leica was built using a 35mm cine film. Argus was another camera analog camera that used the 35mm format and was rather inexpensive as compared to Leica and became very popular. Analog CCTV cameras In 1942 a German engineer Walter Bruch developed and installed the very first system of analog CCTV cameras. He is also credited for the invention of color television in the 1960. Photo Pac The first disposable camera was introduced in 1949 by Photo Pac. The camera was only a onetime use camera with a roll of film already Digital Cameras Mavica by Sony Mavica (the magnetic video camera) was launched by Sony in 1981 and was the first game-changer in the digital camera world. The images were recorded on floppy disks and images can be viewed later on any monitor screen. It was not a pure digital camera, but an analog camera. But got its popularity due to its storing capacity of images on a floppy disk. It means that you can now store images for a long-lasting period, and you can save a huge number of pictures on the floppy which are replaced by the new blank disc when they got full. Mavica has the capacity of storing 25 images on a disk. The Fuji DS-1P camera by Fuji Films 1988 was the first true digital camera Nikon D1 was a 2.74-megapixel camera and the first commercial digital SLR camera developed by Nikon and was very much affordable by professionals. Today digital cameras are included in mobile phones with very high resolution and quality. Applications and Usage Since digital image processing has very wide applications and almost all of the technical fields are impacted by DIP, we will just discuss some of the major applications of DIP. Digital Image processing is not just limited to adjusting the spatial resolution of the everyday images captured by the camera. It is not just limited to increasing the brightness of the photo, etc. Rather it is far more than that. Electromagnetic waves can be thought of as streams of particles, where each particle is moving with the speed of light. Each particle contains a bundle of energy. This bundle of energy is called a photon. The electromagnetic spectrum according to the energy of photons is shown below. In this electromagnetic spectrum, we are only able to see the visible spectrum. The visible spectrum mainly includes seven different colors that are commonly term as (VIBGOYR). VIBGOYR stands for violet, indigo, blue, green, orange, yellow, and Red. But that does not nullify the existence of other stuff in the spectrum. Our human eye can only see the visible portion, in which we see all the objects. But a camera can see the other things that the naked eye is unable to see. For example: X-rays, gamma rays, etc. Hence the analysis of all that stuff too is done in digital image processing. This discussion leads to another question which is why do we need to analyze all that other stuff in the EM spectrum too? The answer to this question lies in the fact that other stuff such as X-ray has been widely used in the field of Applications of Digital Image Processing Some of the major fields in which digital image processing is widely used are mentioned below Image sharpening and restoration Medical field Remote sensing Transmission and encoding Machine/Robot vision Color processing Pattern recognition Video processing Microscopic Imaging Others Image sharpening and restoration Image sharpening and restoration refers here to processing images that have been captured from the modern camera to make them a better image or to manipulate those images in a way to achieve the desired result. It refers to doing what Photoshop usually does. This includes zooming, blurring, sharpening, grayscale to color conversion, detecting edges and vice versa, Image retrieval, and Image recognition. The original image The zoomed Blur image Edges Sharp image Medical field The common applications of DIP in the field of medicine is Gamma ray imaging PET (positron emission tomography) scan X Ray Imaging Medical CT UV imaging UV imagingthe area of the earth is In the field of remote sensing, scanned by a satellite or from a very high ground and then it is analyzed to obtain information about it. One particular application of digital image processing in the field of remote sensing is to detect infrastructure damages caused by an earthquake. It takes a longer time to grasp damage, even if serious damage is focused on. Since the area affected by the earthquake is sometimes so wide, it is not possible to examine it with the human eye to estimate damages. Even if it is, then it is a very hectic and time-consuming procedure. So, a solution to this is found in digital image The key steps included in the analysis are: The extraction of edges Analysis and enhancement of various types of edges Transmission and encoding The very first image that was transmitted over the wire was from London to New York via a submarine cable. The picture that was sent is shown below. The picture took three hours to reach from one place to another. Now just imagine, that today we can see live video feed, or live CCTV footage from one continent to another with just a delay of seconds. It means that a lot of work has been done in this field too. This field does not only focus on transmission but also encoding. Many different formats have been developed for high or low bandwidth to encode photos and then stream them over the internet. Machine/Robot vision Apart from the many challenges that a robot face today, one of the biggest challenges still is to increase the vision of the robot. Make the robot able to see things, identify them, identify the hurdles, etc. Much work has been contributed by this field and a complete other field of computer vision has been introduced to work on it. Hurdle detection Hurdle detection is one of the A common task that has been done through image processing, by identifying different types of objects in the image and then calculating the distance between robot and hurdles. Line follower robot Most of the robots today work by following the line and thus are called line follower robots. This helps a robot to move on its path and perform some tasks. This has also been achieved through image processing. Color processing Color processing includes the processing of colored images and different color spaces that are used. For example, RGB color model, YCbCr, and HSV. It also involves studying the transmission, storage, and encoding of these color images. Pattern recognition Pattern recognition involves the study of image processing and various other fields that include machine learning ( a branch of artificial intelligence). In pattern recognition, image processing is used for identifying the objects in images and then machine learning is used to train the system for the change in pattern. Pattern recognition is used in computer-aided diagnosis, recognition of handwriting, recognition of images, etc. Video processing A video is nothing but just the very fast movement of pictures. The quality of the video depends on the number of Concept of Dimensions Consider you have a friend who lives on the moon, and he wants to send you a gift for your birthday present. He asks you about your residence on Earth. The only problem is that the courier service on Moon does not understand the alphabetical address, rather it only understands the numerical coordinates. So how do you send him your position on earth? That’s where comes the concept of dimensions. Dimensions define the minimum number of points required to point a position of any particular object within a space. So let us go back to our example again in which you have to send your position on Earth to your friend on the Moon. You send him three pairs of coordinates. The first one is called longitude, the second one is called latitude, and the third one is called altitude. These three coordinates define your position on the Earth. The first two define your location, and the third one defines your height above sea level. So that means that only three coordinates are required to define your position on earth. That means you live in a world which is 3 dimensional. And thus this not only answers the question about dimension but also answers the reason, that why we live in a 3d world. Since we are studying this concept of digital image Dimensions of image If we live in the 3d world, which means a 3-dimensional world, then what are the dimensions of an image that we capture? An image is 2-dimensional, that’s why we also define an image as a 2-dimensional signal. An image has only height and width. An image does not have depth. Just have a look at this image below. If you look at the above figure, it shows that it has only two axes which are the height and width axis. You cannot perceive depth from this image. That’s why we say that an image is a 2-dimensional signal. But our eye can perceive 3-dimensional objects, but this will be more explained in the next slides of how the camera works, and the image is perceived. How does television work? If we look at the previous image, we see that it is a 2dimensional image. To convert it into three dimensions, we need one other dimension. Let us take time as the third dimension, in that case, we will move this two-dimensional image over the 3rd dimension time. The same concept that happens in television, helps us perceive the depth of different objects on a screen. Does that mean that what comes on the TV or what we see on the television screen is 3d? Well, we can yes. The reason is that, in the case of TV if we are playing a video. Then a video is nothing else but twodimensional pictures that move over time dimension. As two-dimensional objects are moving over the third dimension which is the time, we can say it is 3 dimensional. Image Formation on Camera How human eye works? Before we discuss, the image formation on analog and digital cameras, we have to first discuss the image formation on the human eye. Because the basic principle that is followed by the cameras has been taken from the way, the human eye works. When light falls upon a particular object, it is reflected after striking through the object. The rays of light when passed through the lens of the eye, form a particular angle, and the image is formed on the retina which is the back side of the wall. The image that is formed is inverted. This image is then interpreted by the brain and that makes us able to understand things. Due to angle formation, we can perceive the height and depth of the object we are As you can see in the above figure, when sunlight falls on the object (in this case the object is a face), it is reflected and different rays from different angles when they are passed through the lens and an inverted image of the object has been formed on the back wall. The last portion of the figure denotes that the object has been interpreted by the brain and re-inverted. Now let’s take our discussion back to the image formation on analog and digital cameras. Image formation on analog cameras In analog cameras, the image formation is due to the chemical reaction that takes place on the strip that is used for image formation. A 35mm strip is used in an analog camera. It is denoted in the figure by a 35mm film cartridge. This strip is coated Light is nothing but just the small particles known as photon particles. So when these photon particles are passed through the camera, it reacts with the silver halide particles on the strip and it results in the silver which is the negative of the image. This is just the basics, although image formation involves many other concepts regarding the passing of light inside, and the concepts of shutter and shutter speed and aperture and its opening but for now we will move on to the next part. Image formation on digital cameras In digital cameras, the image formation is not due to the chemical reaction that takes place, rather it is a bit more complex than this. In the digital camera, a CCD array of sensors is used for the image formation. CCD stands for charge-coupled Image through CCDand array device.formation It is an image sensor, like other sensors, it senses the values and converts them into an electric signal. In the case of CCD, it senses the image and converts it into an electric signal, etc. This CCD is actually in the shape of an array or a rectangular grid. It is like a matrix with each cell in the Like analog cameras, in the case of digital too, when light falls on the object, the light reflects after striking the object and is allowed to enter inside the camera. Each sensor of the CCD array itself is an analog sensor. When photons of light strike the chip, it is held as a small electrical charge in each photo sensor. The response of each sensor is directly equal to the amount of light or (photon) energy strike on the surface of the sensor. Since we have already defined an image as a two-dimensional signal and due to the two-dimensional formation of the CCD array, a complete image can be achieved from this CCD array. It has a limited number of sensors, and it means a limited detail can be captured by it. Also, each sensor can have only one value against each photon particle that strikes on it. So the number of photons striking is counted and stored. To measure accurately these, external CMOS sensors are also Introduction to pixel The value of each sensor of the CCD array refers to the value of the individual pixel. The number of sensors = number of pixels. It also means that each sensor could have only one and only one value. Storing image The charges stored by the CCD array are converted to voltage one pixel at a time. With the help of additional circuits, this voltage is converted into digital information and then it is stored. Each company that manufactures digital camera, make its own CCD sensors. That includes Sony, Mitsubishi, Nikon, Samsung, Toshiba, Fuji-Film, Canon, etc. Apart from the other factors, the quality of the image captured also depends on the type and quality of the CCD array that has been used.