Lecture 1 - Introduction.pdf

Basics of Computer Vision Instructor: Karam Almaghout 28.08.2024  Contact Information Instructor: Karam Almaghout Telegram: @karamalmaghout Email: [email protected] Teaching Assistants Karim ElDakroury...

Basics of Computer Vision Instructor: Karam Almaghout 28.08.2024  Contact Information Instructor: Karam Almaghout Telegram: @karamalmaghout Email: [email protected] Teaching Assistants Karim ElDakroury Kirill Yakovlev Course materials will be sent to you through moodle Introduction to Computer Vision Fall 2024 2  Grading Criteria Group Project (3 students per group) 20% Homework 20% Mid-term Exam + Lab Exam 20% + 10% Final Exam + Lab Exam 20% + 10% A. 90-100 B. 75-89 C. 60-74 D. 0-59 (Fail) Attendance is mandatory in the course Less than 75% attendance in the labs → 0 in the project Introduction to Computer Vision Fall 2024 3  Goals of this Course This course is designed for undergraduate students to provide comprehensive introduction and practical knowledge of computer vision. Student will learn to implement different computer vision methods in Python programming environment. The end of the day they will be able to apply computer vision technique to solve real-world problems. Introduction to Computer Vision Fall 2024 4  Prerequisites Courses Prerequisites—these are essential! Data structures Linear algebra Vector calculus A good working knowledge of Python programming Course does not assume prior imaging experience Image processing, graphics, etc. Introduction to Computer Vision Fall 2024 5  Learning Outcome After this course, you will be able to: Understand how machine can visualize the objects and understand images/videos Significant exposure to real-world implementations To develop research interest in the theory and application of computer vision Introduction to Computer Vision Fall 2024 6  Materials Textbook Dawson-Howe, A Practical Introduction to Computer Vision with OpenCV, Wiley, 2014. Reference Materials Szeliski, R. Computer Vision: Algorithms and Applications, Springer, 2010. Trucco, E. and Verri, A. Introductory Techniques for 3-D Computer Vision, Prentice-Hall, 1998. Vernon, D. Machine Vision: Automated Visual Inspection and Robot Vision, Prentice-Hall, 1991 Lecture slides and shared materials through moodle Introduction to Computer Vision Fall 2024 7  Course Plan Week 1: Introduction to Computer Vision, Image Acquisition, Basic Image Processing Week 2: Image Filtering, Noise, Convolutions, Kernels, Smoothing and Blurring Week 3: Thresholding, Histograms, Image Gradients, Edge Detection and Morphological operations Week 4: Object Detection, Template Matching, Image Pyramids, HOG with SVM and Contours (Homework Out) Week 5: Image Features: Interest Points, Descriptors and Matching (Homework Submission) Week 6: Bag of Visual Words, Image Classification (Group Project Out) Week 7: Midterm Exam Week 8: Deep Learning for Object Detection and Localization (YOLO) (Group Project Proposal Submission) Week 9: Generative Adversarial Networks (GAN) for Computer vision Week 10: Face Detection, Viola Jones, Haar-like Features, Integral Image, Adaboost, Classifier Cascade Week 11: Hough Transform Week 12: Semantic Segmentation Week 13: Video Processing and Tracking (Group Project Report Submission) Week 14: Project Demo Presentations Week 15:Final Exam Introduction to Computer Vision Fall 2024 8  Ready to go …..☺ Introduction to Computer Vision Fall 2024 9  Contents Introduction to computer vision Computer vision in action The human vision system Image sensing and acquisition Sampling and quantization Fundamentals of color images Converting images from one color space to another space Summary Introduction to Computer Vision Fall 2024 10  Source of the material This lecture is based on the following resources Lecture slides of Prof. Muhammad Fahim. Based on A Practical Introduction to Computer Vision with OpenCV by Kenneth Dawson-Howe, lecture slides of David Vernon and his book. Computer vision: Foundation and applications by Ranjhay Krishna. Found material over the internet to aligned the subject according to the need of the students. Introduction to Computer Vision Fall 2024 11  Introduction Computer vision is inspired by the capabilities of the human vision system and initially addressed in the 1960s and 1970s Definition of Computer Vision Computer vision is the automatic analysis of images or videos by computers in order to gain understanding of the world Introduction to Computer Vision Fall 2024 12  Introduction – Every image tells a story Goal of computer vision: perceive the “story” behind the picture Introduction to Computer Vision Fall 2024 13  Introduction Computer Vision is about understanding images. These images can be Black and white, greyscale, colour or multi-spectral Snapshots or video sequences Taken with a static or moving camera Taken of a stationary or dynamic scene Taken with a calibrated or un-calibrated camera What computer vision aims to do is to extract some useful information from these images for… Inspection purposes Analysis purposes Control purposes Introduction to Computer Vision Fall 2024 14  Why study computer vision? Billions of images/videos captured per day Huge number of useful applications Introduction to Computer Vision Fall 2024 15  Computer Vision in Action!! https://www.youtube.com/watch?v=NrmMk1Myrxc Introduction to Computer Vision Fall 2024 16  Computer Vision in Action!! https://www.youtube.com/watch?v=R_tzMk34cX8 Introduction to Computer Vision Fall 2024 17  Computer Vision in Action!! Digit recognition, AT&T labs (1990’s) http://yann.lecun.com/exdb/lenet/ Introduction to Computer Vision Fall 2024 18  Computer Vision in Action!! Google Lens Search what you see https://lens.google.com/ Introduction to Computer Vision Fall 2024 19  Computer Vision in Action!! Special Effects: Motion Capture Introduction to Computer Vision Fall 2024 20  Computer Vision in Action!! It is a robotic spacecraft mission of Chinese Lunar Exploration Program Change 5: It launched on 23 November 2020 The first panorama from the far side of the Moon Introduction to Computer Vision Fall 2024 21  Computer Vision in Action!! NASA’s Mars Curiosity Rover Amazon Prime Air Introduction to Computer Vision Fall 2024 22  Computer Vision in Action!! Fingerprint scanners on many new Login without a password smartphones and other devices Introduction to Computer Vision Fall 2024 23  Computer Vision in Action!! Iris Recognition Access control, e.g., border [John Daugman: How Iris Recognition Works, 2004] Genetically identical eyes also have different iris. Introduction to Computer Vision Fall 2024 24  Computer Vision in Action!! Berkeley Deep Drive (Driverless cars) https://bdd-data.berkeley.edu/wad-2018.html https://www.youtube.com/watch?v=fKXztwtXaGo Introduction to Computer Vision Fall 2024 25  Computer Vision in Action!! Image Super Resolution Source: [Goodfellow, 2016] https://arxiv.org/abs/1701.00160 Introduction to Computer Vision Fall 2024 26  Computer Vision in Action!! Image Inpainting Ugur Demir and Gozde Unal. Patch-Based Image Inpainting with Generative Adversarial Networks. arXiv preprint, 2018. URL http://arxiv.org/abs/1803.07422. Introduction to Computer Vision Fall 2024 27  Computer Vision in Action!! Create Art Ahmed Elgammal, Bingchen Liu, Mohamed Elhoseiny, and Marian Mazzone. CAN: Creative Adversarial Networks, Generating ”Art” by Learning About Styles and Deviating from Style Norms. arXiv preprint, (Iccc):1–22, 2017. doi: 10.1089/cyber.2017.29084.csi. URL http://arxiv.org/abs/1706.07068. Introduction to Computer Vision Fall 2024 28  Computer Vision in Action!! Sports Check the video: https://www.smt.com/ Introduction to Computer Vision Fall 2024 29  Computer Vision in Action!! Medical Imaging 3D imaging (MRI) Skin cancer classification with deep learning (Published: 25 January 2017) https://cs.stanford.edu/people/esteva/nature/ Introduction to Computer Vision Fall 2024 30  Computer Vision – A difficult Problem Motion Viewpoint variation Intra-class variation Illumination Occlusion Background clutter Introduction to Computer Vision Fall 2024 31  The Human Vision System If we could duplicate the human visual system, then the problem of developing a computer vision system would be solved. So why can’t we? The human visual system is incredibly complex, and our understanding of it is still limited for several reasons like the system's complexity, non-linearity, adaptability, and integration with higher cognitive functions. Introduction to Computer Vision Fall 2024 32  Structure of Human Eye Eye is nearly a sphere, with an average diameter of approximately 20 mm. Three membranes enclose the eye: o Cornea (Tough and Transparent tissue) and sclera outer cover o Choroid (i.e., Membrane contains a network of blood vessels and major source of nutrition) o Retina the light-sensitive layer at the back of the eye. Introduction to Computer Vision Fall 2024 33  Structure of Human Eye When the eye is properly focused, light from an object outside the eye is imaged on the retina Two cell classes (photoreceptors) convert light into electrical signals, which are then processed by the brain to form visual images Cones 6-7 million located primarily near the center of the retina (the fovea) highly sensitive to color can resolve fine details because each is attached to a single nerve ending Cone vision is called photopic or bright-light vision Rods 75-150 million distributed over the retinal surface multiple rods connected to a single nerve ending give a general overall picture of the field of illumination not color sensitive but are sensitive to low levels of illumination Rod vision is called scotopic or dim-light vision Around the region of the emergence of the optic nerve, there is no receptors and results in the so-called blind spot Introduction to Computer Vision Fall 2024 34  Structure of Human Eye Distribution of rods and cones in the retina Introduction to Computer Vision Fall 2024 35  Blind-Spot Experiment Draw an image similar to that below on a piece of paper (the dot and cross are about 15 cm apart) Close your right eye and focus on the cross with your left eye Hold the image about 50 cm away from your face and move it slowly towards you The dot should disappear! Introduction to Computer Vision Fall 2024 36  Image Formation in the Eye Suppose a person is looking at a palm tree 15 m high at a distance of 100m. 15/100 = h/17 or approximately 2.55 mm Introduction to Computer Vision Fall 2024 37  Image Formation Introduction to Computer Vision Fall 2024 38  Optical Illusions Introduction to Computer Vision Fall 2024 39  Image Sensing and Acquisition Images are generated by the combination of an “illumination” source and the reflection or absorption of energy from that source by the elements of the “scene” being imaged. An example of the digital image acquisition process Digital images represent the reflectance function of a scene, but they do so in a sampled and quantized form Introduction to Computer Vision Fall 2024 40  What is an Image? A grid (matrix) of intensity values Introduction to Computer Vision Fall 2024 41  Processing the Images Starting pixel f(x,y) pixel Introduction to Computer Vision Fall 2024 42  Processing the Images 1 1 1 1 1 1 1 0 1 0 1 1 1 0 0 0 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 Image f(x, y) In Computer (Digitized Image) Introduction to Computer Vision Fall 2024 43  Digitizing an Image – Sampling Four different samplings of the same image; top left 256x192, top right 128x96, bottom left 64x48 and bottom right 32x24 Introduction to Computer Vision Fall 2024 44  Digitizing an Image – Quantization Each quantized integer value is known as a pixel It is the smallest discrete accessible sub-section of a digital image. Four different quantization's of the same grey-scale image; top left 8 bits, top right 6 bits, bottom left 4 bits and bottom right 2 bits Introduction to Computer Vision Fall 2024 45  Sampling Rate – Issue If the sample rate is not sufficient, the sampled image will be aliased This is also known as “moire pattern” Introduction to Computer Vision Fall 2024 46  Sampling Rate – Solution Nyquist Shannon’s Sampling Theorem The minimum sampling frequency required to reconstruct a signal from its instantaneous samples must be at least twice the highest frequency Introduction to Computer Vision Fall 2024 47  Image Representation Pixel content depends on the image type Gray level pictorial digital images: intensity Color pictorial digital images: color (modeled as triplets, e.g., RGB) Range images: depth information Medical images: radiation absorbance level Thermal images: heat Introduction to Computer Vision Fall 2024 48  Check Point!! Black and White image contains only two levels 1 0 Gray image represent by black and white shades or combination of levels How many colors in gray scale images? How many bits are required for gray scale image? 8-bit gray image means total 28 levels from black to white. Where 0 = black and 255 is White Introduction to Computer Vision Fall 2024 49  Color Images – Motivation Color is a powerful descriptor that often simplifies object identification and extraction from a scene. Human can distinguish thousands of color shades and intensities, compared to about only two dozen shades of gray. Introduction to Computer Vision Fall 2024 50  Color Fundamentals In 1666 Sir Isaac Newton discovered that when a beam of sunlight passes through a glass prism, the emerging beam is split into a spectrum of colors Introduction to Computer Vision Fall 2024 51  Electromagnetic Spectrum Visible light wavelength: from around 400 to 700 nm Image Source: https://www.researchgate.net/figure/The-complete-electromagnetic-spectrum-with-the- spectral-subdivisions-of-the-visible_fig2_320616988 Introduction to Computer Vision Fall 2024 52  Color Fundamentals The color that human perceive in an object = the light reflected from the object Illumination source Scene Reflection Eye Introduction to Computer Vision Fall 2024 53  Primary and Secondary Colors Primary colors: In 1931, CIE(International Commission on Illumination) defines specific wavelength values to the primary colors Blue = 435.8 nm 16.8 million colors can be reproduced by mixing Green = 546.1 nm an appropriate set of three primary colors Red = 700 nm Secondary colors: G+B=Cyan R+G=Yellow R+B=Magenta *CIE = Commission Internationale de l’Eclairage Introduction to Computer Vision Fall 2024 54  Primary and Secondary Colors Additive Primary Colors: RGB use in the case of light sources such as color monitors RGB add together to get White Subtractive Primary Colors: CMY use in the case of pigments in printing devices White subtracted by CMY to get Black Introduction to Computer Vision Fall 2024 55  Colour Spaces There are many colour representations RGB Red, Green, Blue -- monitor, video camera CMY Cyan, Magenta, Yellow -- model for color printing YUV Luminance (Y), Blue minus Luminance (U), Red minus Luminance (V) YCrCb Scaled version of YUV CIE XYZ Standard reference colour space based on the response of human eye CIE L*u*V* Perceptually uniform colour space CIE L*a*b* Device independent colour space (all colours perceived by humans) HSV Hue, Saturation, Value HLS Hue, Luminance, Saturation HSI Hue, Saturation, Intensity. -- close to human interpret color Introduction to Computer Vision Fall 2024 56  Full-Color Image and Various Color Space Components Introduction to Computer Vision Fall 2024 57  RGB In the RGB model each color appears in its primary spectral components of red, green and blue Purpose of color models: To facilitate the specification of colors in some standard Introduction to Computer Vision Fall 2024 58  RGB Introduction to Computer Vision Fall 2024 59  RGB Introduction to Computer Vision Fall 2024 60  The HSI Color Model RGB is useful for hardware implementations and is luckily related to the way in which the human visual system works However, RGB is not a particularly intuitive way in which to describe colors Rather when people describe colors they tend to use hue, saturation and brightness RGB is great for color generation, but HSI is great for color description Introduction to Computer Vision Fall 2024 61  The HSI Color Model The HSI model uses three measures to describe colours: Hue: Dominant color Saturation: Gives a measure of how much a pure color is diluted with white light Intensity: Brightness Introduction to Computer Vision Fall 2024 62  Converting From RGB To HSI Given a color as R, G, and B its H, S, and I values are calculated as follows: Introduction to Computer Vision Fall 2024 63  Converting From HSI To RGB Given a color as H, S, and I it’s R, G, and B values are calculated as follows: RG sector (0