02 Chap2 Digital Image Fundamentals.pdf

Chap 2 Digital Image Fundamentals 1 2.1 Elements of Visual Perception Although the digital image processing built on a foundation of mathematical and probabilistic formulations, human intuition and analysis generally play a central role in the choice of...

Chap 2 Digital Image Fundamentals 1 2.1 Elements of Visual Perception Although the digital image processing built on a foundation of mathematical and probabilistic formulations, human intuition and analysis generally play a central role in the choice of one technique versus another, and this choice often is made based on subjective, visual judgements. Developing a basic understanding of human visual perception as a first step is appropriate. The interest is in the mechanics and parameters related to how images are formed and perceived by humans. Physical limitations of human vision are also important. 2 1  2.1 Elements of Visual Perception (角膜) A simplified diagram of a cross section of (水晶體) the human eye. The eye is nearly a sphere, with an average diameter of approximately 20 mm. (視網膜) (中央凹) (鞏膜) (脈絡膜) 3 2.1 Elements of Visual Perception Three membranes enclose the eye: cornea and sclera outer over; choroid ; retina There are two classes of receptors in retina: cones and rods The cones are located primarily in the central portion of the retina, called the fovea, and are highly sensitive to color. Human can resolve fine details with these cones largely. The rods are distributed over the retinal surface. Rods serve to give a general, overall picture of the field of view. 4 2  2.1 Elements of Visual Perception (Ref: http://users.rcn.com/jkimball.ma.ultranet/BiologyPages/V/Vision.html) 5 2.1 Elements of Visual Perception 6 3 2.1.3 Brightness Adaptation and Discrimination Two phenomena demonstrate that human perceived brightness is not a simple function of intensity. The first one is based on the fact that the visual system tends to undershoot or overshoot around the boundary of regions of Actual intensity different intensities. The figure shows the Mach band effect, in which the perceived intensity is not Perceived intensity simple function of actual intensity. 8 2.1.3 Brightness Adaptation and Discrimination The second phenomenon, called simultaneous contrast ( 聯合對比度)  A region’s perceived brightness does not depend simply on its intensity.  Ex., in this figure, all the inner squares have the same intensity, but they appear progressively darker as the background becomes lighter. 9 4  Other examples of human perception phenomena are optical illusions: the eye fills in non-existing information or wrongly perceives geometrical properties of objects. 10 Illusion (Ref: http://www.ritsumei.ac.jp/~akitaoka/index-e.html) 11 5  Illusion (Refs: 1. T.N. Cornsweet, Visual Perception, Harcourt Brace Jovanovich, 1970 2. D.H. Hubel, Eye, Brain, and Vision, Scitific American, 1988 ) 12  Illusion The Starry Night by van Gogh (References: Self-Animating Images: Illusory Motion Using Repeated Asymmetric Patterns, SIGGRAPH 2008) 13 6 14 15 7 16 2.3 Image Sensing and Acquisition A sensor array In digital cameras 17 8  2.3 Image Sensing and Acquisition Images are denoted by two-dimensional functions of the form 𝑓(𝑥, 𝑦). The value of 𝑓 at spatial coordinates (𝑥, 𝑦) is a positive scalar quantity whose physical meaning is determined by the source of the image. A simple image formation model is given by 𝒇 𝒙, 𝒚 = 𝒊 𝒙, 𝒚 𝒓 𝒙, 𝒚 + 𝒏 𝒙, 𝒚  𝑓 𝑥, 𝑦 : Image function  𝑖 𝑥, 𝑦 : Illumination function  𝑟(𝑥, 𝑦): Reflectance function (0.0 < 𝑟 𝑥, 𝑦 < 1.0)  𝑛(𝑥, 𝑦): Random noise function 𝐿𝑚𝑖𝑛 ≤gray level 𝑓(𝑥, 𝑦) ≤ 𝐿𝑚𝑎𝑥 0 ≤gray level 𝑓 𝑥, 𝑦 ≤ 𝐿 − 1, gray level normalization 𝑖 𝑥, 𝑦 is determined by illumination source and 𝑟(𝑥, 𝑦) is determined by the characteristics of the imaged objects. 18 2.4 Image Sampling and Quantization To acquire digital images from the continuous sensed data 𝑓(𝑥, 𝑦):  Digitization in coordinate values: Sampling  Digitization in amplitude values: Quantization The resulting image has 𝑀 rows and 𝑁 columns as  f (0,0) f (0,1) f (0, N  1)   f (1,0) f (1,1) f (1, N  1)   f ( x, y )         f ( M  1,0) f ( M  1,1) f ( M  1, N  1) 19 9 20  Left: continuous image projected onto a sensor array. Right: result of image sampling and quantization. 21 10 2.4.2 Representing Digital Images Three basic ways to represent 𝑓(𝑥, 𝑦) A 2.5D plot of the function. Look like a topography (useful for some specific algorithms). An image appear on a monitor (for visual display). Print the numerical values in an array (for implementation). 22  2.4 Image Sampling and Quantization The digitization process requires to determine the 𝑀, 𝑁, and 𝐿.  𝑀 and 𝑁 : image size  𝐿: gray-level resolution (radiometric resolution) 𝐿 = 2𝑘 , 𝐿 = gray-level Dynamic range: the range of values spanned by the gray scale, 𝐿𝑚𝑖𝑛 , 𝐿𝑚𝑎𝑥.  High dynamic range = high contrast image The number of bits required to store the image 𝑏 (bit)= 𝑀 × 𝑁 × 𝑘 or b (bit)= 𝑁 2 × 𝑘 23 11  2.4 Image Sampling and Quantization 𝑁=256, 𝑘=8: 65536 bytes 𝑁=2048, 𝑘=8: 12 M bytes 𝑁=8192, 𝑘=8: 192 M bytes 24 Image representation Binary image: 𝒇 𝒙, 𝒚 = 𝒂 𝐨𝐫 𝒃  Only two values for a pixel: represented by 1 bit (0 or 1)  Size of a 256×256 image file: 65,536 bits = 8,192 bytes 25 12 Image representation Gray-level image: 𝟎 ≤ 𝒇(𝒙, 𝒚) ≤ 𝟐𝟓𝟓  A pixel value is represented by 1 byte (28); 0~255: from black to white (256 gray levels)  Size of a 256×256 image file: 65,536 bytes 26 Image representation Color models of color images  RGB model: Red, Green, and Blue primaries  Number of representable colors: (28)3 = 224 = 16,777,216 (true color)  E.g., (0, 255, 0): green, (255, 0, 0): red, (0, 120, 0): light green, (100, 100, 0): yellow 27 13 2.4.3 Spatial and Intensity Resolution Spatial resolution is the smallest discernible (可識別的) detail in an image. Quantitatively, spatial resolution can be stated as dots (pixels) per unit distance. Spatial resolution is highly related to image size, but they have different meaning. 28 The image sizes are the same, but spatial resolutions are different. 29 14  Gray-level resolution: similarly refers to the smallest discernible change in intensity level. 256 128 levels levels 64 32 levels levels 30  Gray-level resolution: similarly refers to the smallest discernible change in intensity level. 16 levels 8 levels 4 levels 2 levels 31 15 Empirical Study of Resolutions Goal: How 𝒌 and 𝑵 affect the image quality 2𝑘 -level digital image of size 𝑁 × 𝑁 Details (frequency) 32  Empirical Study of Resolutions Iso-preference Curves Curves tends to shift right and upward. It simply means larger values for 𝑁 and 𝑘 implies better picture quality. Curve tends to become more vertical as the detail in the image increases. This result suggests that for images with a large amount of detail only a few intensity levels may be needed, and vice versa. 33 16 2.4.5 Zooming and Shrinking Digital Images Idea: adjust the grid size over the original image 34  2.4.5 Zooming and Shrinking Digital Images Zooming: Create several new pixel locations. Assign a gray-level to each of those new locations  Nearest neighbor interpolation Pixel replication: a chessboard effect  Bilinear interpolation: using four nearest neighbors  Higher-order non-linear interpolation: using more neighbors for interpolation 35 17 Zooming Example Nearest neighbor interpolation Bilinear interpolation 36  Shrinking Digital Images An image is too big to fit on the screen. How to reduce it? How to generate a half-sized version? Shrinking:  Direct shrinking (remove some rows and columns) causes aliasing  Blur the image before shrinking it, which can reduce aliasing 37 18 Image Shrinking -- Naïve method 1/8 1/4 Throw away every some rows and columns to create a half-sized image 38 Image Shrinking -- The common method G 1/8 G 1/4 Solution: subsampling with Gaussian pre-filtering Filter the image using Gaussian filter, then subsample Gaussian 1/2 39 19 Hierarchical Structure 40 Newest Method – Image Retargeting 41 20 2.5 Basic Relations between Pixels Neighbors of a pixel (𝑥, 𝑦)  Horizontal neighbors (𝒙 + 𝟏, 𝑦), 𝒙 − 𝟏, 𝑦  Vertical neighbors (𝑥, 𝒚 + 𝟏), (𝑥, 𝒚 − 𝟏)  Four diagonal neighbors: 𝑁𝐷 (𝑝) (𝑥 + 1, 𝑦 + 1), (𝑥 + 1, 𝑦 − 1), (𝑥 − 1, 𝑦 + 1), (𝑥 − 1, 𝑦 − 1)  4-neighbors of p: 𝑁4 (𝑝) (including horizontal and vertical neighbors).  8-neighbors of p: 𝑁8 (𝑝). 𝑁8 𝑝 = 𝑁4 (𝑝) ∪ 𝑁𝐷 (𝑝) Adjacency, Connectivity, Regions, Boundary 42 2.5 Basic Relations between Pixels Adjacency p p 4-adjacency: 𝑁4 (𝑝) 8-adjacency: 𝑁4 (𝑝) ∪ 𝑁𝐷 (𝑝) 43 21 Connectivity Path:  𝑥0 , 𝑦0 , 𝑥1 , 𝑦1 , ⋯ , 𝑥𝑛 , 𝑦𝑛 where (𝑥𝑖 , 𝑦𝑖 ) and (𝑥𝑖+1 , 𝑦𝑖+1 ) are adjacent.  Closed path: (𝑥𝑛 , 𝑦𝑛 ) = 𝑥0 , 𝑦0 Connectivity:  Two pixels are said connected if they have the same value and there is a path between them.  If a S is a set of pixels, For any pixel p in S, the set of pixels that are connected to it is called a connected component of S. If S has only one connected component, S is called a connected set. 44 Regions R is a region if R is a connected set. The pixel in the boundary (contour) has at least one 4- adjacent neighbor whose value is 0. 45 22  2 Distance measures 2 1 2  Euclidean distance 2 1 0 1 2  City-block distance or 𝐷4 distance. 𝐷4 𝑝, 𝑞 = 𝑥 − 𝑠 + 𝑦 − 𝑡 2 1 2  𝑫𝟖 distance or chessboard distance. 2 𝐷4 𝐷8 𝑝, 𝑞 = 𝑚𝑎𝑥 𝑥 − 𝑠 , 𝑦 − 𝑡 2 2 2 2 2 2 1 1 1 2 2 1 0 1 2 2 1 1 1 2 2 2 2 2 2 𝐷8 46 23