Image Processing and Computer Vision

• Introduction to OpenCV, prepared by Mark O. Montances.

Understanding Image Processing vs Computer Vision

• Image Processing is a subset of signal processing which is focused specifically on images
• Image processing typically involves pixel-level manipulations to enhance, transform, or analyze visual data.
• Computer Vision is a broader field that builds on image processing but goes beyond it.
• Computer vision aims to replicate human visual understanding and decision-making.
• The purpose of Image Processing is to improve image quality or extract basic information for further processing.
• Image Processing does not usually involve understanding the content of the image.
• Computer Vision extracts semantic, meaningful information to perform tasks like detection, recognition, and prediction.
• Image Processing examples: Noise removal, color corrections, basic segmentation.
• Computer vision examples: Object detection(YOLO), pose estimation(MediaPipe), Optical Character Recognition (OCR), scene understanding.
• Image Processing is often a necessary step in Computer Vision pipelines
• Computer Vision integrates techniques from image processing but extends them with machine learning and deep learning for higher-level tasks.

Why OpenCV?

• OpenCV is popular with an open-source, large community, and is cross-platform
• OpenCV Supports real-time applications with C++ optimization
• OpenCV has an extensive library for image processing, video handling, and ML integration
• OpenCV is useful for real-time applications
• OpenCV is useful when there is a need for performance and advanced computer vision tasks.

Basic Image handling in OpenCV

• To load, show, and close an image, use this code: import cv2; img = cv2.imread('image.jpg'); cv2.imshow('Display Window', img); cv2.waitKey(0); cv2.destroyAllWindows()

Cv2.imshow() vs plt.imshow()

• OpenCV stores image channels in BGR format by default (Blue-Green-Red)
• Most other libraries like matplotlib and images standards use RGB (Red-Green-Blue)
• To load and image as BGR and display it as RGB, use this code: img = cv2.imread('image.jpg'); plt.imshow(img); plt.show()

Simple Image Manipulations

• To convert BGR to RGB use this code: img_rgb = cv2.cvtColor(img, cv2.COLOR_BGR2RGB); cv2.imshow(‘RGB Image', img_rgb); cv2.waitKey(0); cv2.destroyAllWindows()
• To convert BGR to Grayscale use this code: gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY); cv2.imshow('Grayscale Image', gray); cv2.waitKey(0); cv2.destroyAllWindows()
• To threshold from (Grayscale to Binary) use this code: gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY); thresh = cv2.threshold(gray, 127, 255, cv2.THRESH_BINARY); cv2.imshow('Thresholded Image', thresh); cv2.waitKey(0); cv2.destroyAllWindows()

Syntax Comparison - scikit-image vs OpenCV

• Load Image: scikit-image Syntax: io.imread('image.jpg'), OpenCV Syntax: cv2.imread('image.jpg')
• Save Image: scikit-image Syntax: io.imsave('output.jpg', image), OpenCV Syntax: cv2.imwrite('output.jpg', img).
• Convert to Gray: scikit-image Syntax: color.rgb2gray(image) ,OpenCV Syntax: cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
• Resize Image: scikit-image Syntax: transform.resize(image, (width, height)), OpenCV Syntax: cv2.resize(img, (width, height)
• Rotate Image: scikit-image Syntax: transform.rotate(image, angle), OpenCV Syntax: cv2.warpAffine(img, rotation_matrix, dimensions)
• Edge Detection: scikit-image Syntax: filters.sobel(image), OpenCV Syntax: cv2.Canny(img, threshold1, threshold2)
• Gaussian Blur: scikit-image Syntax: filters.gaussian (image, sigma=1), OpenCV Syntax: cv2.GaussianBlur(img, (5, 5), sigmaX=1)
• Threshold: scikit-image Syntax: filters.threshold_otsu(gray_image), OpenCV Syntax: cv2.threshold (gray, 127, 255, cv2.THRESH_BINARY)
• Histogram: scikit-image Syntax: exposure.histogram(image), OpenCV Syntax: cv2.calcHist([img], [0], None, [256], [0, 256])
• Draw Shape: scikit-image Syntax: draw.circle(image, center, radius), OpenCV Syntax: cv2.circle(img, center, radius, color, thickness)
• Flip Image: scikit-image Syntax: np.flipud(image) or np.fliplr(image), OpenCV Syntax: cv2.flip(img, flipCode=0 or 1)
• Convert Color Space: scikit-image Syntax:color.rgb2hsv(image) ,OpenCV Syntax: cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
• Contour Detection: scikit-image Syntax: measure.find_contours(image, level), OpenCV Syntax: cv2.findContours (thresh, mode, method)

Basic Drawing Functions in OpenCV

• Drawing shapes like rectangles, circles, and lines is essential for annotating images.
• Drawing shapes like rectangles, circles, and lines is essential for drawing bounding boxes for object detection results.
• Drawing shapes like rectangles, circles, and lines is essential for visualizing points, landmarks, or regions of interest.

Drawing a Rectangle (Bounding Box)

• To draw a rectangle (bounding box) use the following code: start_point = (50, 100); end_point = (1000, 200); color = (0, 255, 0); thickness = 2; cv2.rectangle(image, start_point, end_point, color, thickness); cv2.imshow("Rectangle Example", image); cv2.waitKey(0); cv2.destroyAllWindows()
• The start point is the Top-left corner
• The end point is the Bottom-right corner
• color = (0, 255, 0) this is Green in BGR
• thickness = 2 is the Line thickness, use -1 for filled rectangle

Drawing a Circle

• To draw a circle use: center = (900, 500); radius = 150; color = (255, 0, 0); thickness = 2; cv2.circle(image, center, radius, color, thickness)
• center = (900, 500) is the Center of the circle
• color = (255, 0, 0) is Blue in BGR
• thickness = 2 is the Line thickness

Drawing a Line

• To draw a line use: start_point = (200, 400); end_point = (1000, 100); color = (0, 0, 255); thickness = 3; cv2.line(image, start_point, end_point, color, thickness)
• color = (0, 0, 255) is Red in BGR

Adding a Text

• To add text use: text = "Hello World"; position = (160, 140); font = cv2.FONT_HERSHEY_SIMPLEX; font_scale = 5; color = (255, 255, 255); thickness = 4; cv2.putText(image, text, position, font, font_scale, color, thickness)
• position = (160, 140) is the Bottom-left corner of the text
• font = cv2.FONT_HERSHEY_SIMPLEX is the Font type
• font_scale = 5 is the Font size
• color = (255, 255, 255) is White in BGR
• thickness = 4 is the Thickness of text

Videos and Live Video Feed Processing

• Videos are formed by combining multiple images to create motion.
• FPS(Frames per Second): Number of images shown per a second.
• Access webcam with this code: cap = cv2.VideoCapture(0)
• Capture live video with this code: while True: ret, frame = cap.read(); cv2.imshow('Webcam Feed', frame); if cv2.waitKey(1) & 0xFF == ord('q'): break; cap.release(); cv2.destroyAllWindows()
• The ASCII of 'q' is 113 or b01110001
• Capture grayscale video: while True: ret, frame = cap.read(); if ret: gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY); cv2.imshow('Grayscale Feed', gray); if cv2.waitKey(1) & 0xFF == ord('q'): break; else: break; cap.release(); cv2.destroyAllWindows().

Typical Computer Vision Tasks

• Object Detection: Identifies and localizes objects in an image or video.
• Object Detection Use cases: Autonomous vehicles, surveillance, retail inventory management.
• Object Detection Algorithms/Libraries: YOLO, SSD, Faster R-CNN, Detectron2.
• Object Tracking: Tracks the motion of objects across frames in a video.
• Object Tracking Use cases: Sports analytics, surveillance, traffic monitoring.
• Object Tracking Algorithms/Libraries: DeepSORT, ByteTrack, OpenCV Tracking API.
• Facial Recognition: Identifies or verifies a person based on facial features.
• Facial Recognition Use cases: Authentication, attendance systems, social media tagging.
• Facial Recognition Algorithms/Libraries: Dlib, FaceNet, OpenCV, DeepFace.
• Pose Estimation: Determines body or object poses by detecting keypoints (e.g., joints).
• Pose Estimation Use cases: Fitness tracking, gesture recognition, sports analytics.
• Pose Estimation Algorithms/Libraries: MediaPipe, OpenPose, AlphaPose.
• Optical Character Recognition (OCR): Extracts and converts text from images to machine-readable formats
• Optical Character Recognition (OCR) Use cases: Document digitization, license plate recognition, text-based analytics.
• Optical Character Recognition (OCR) Algorithms/Libraries: EasyOCR, Tesseract, PaddleOCR, Google Vision API.
• Image Classification: Categorizes an image into predefined classes.
• Image Classification Use cases: Product identification, wildlife monitoring, spam detection.
• Image Classification Algorithms/Libraries: ResNet, EfficientNet, MobileNet, VGG.
• Image Segmentation: Divides an image into regions or objects by assigning each pixel to a class.
• Image Segmentation Use cases: Medical imaging, scene understanding, autonomous navigation.
• Image Segmentation Algorithms/Libraries: Mask R-CNN, U-Net, DeepLab.
• Anomaly Detection: Identifies unusual patterns or outliers in visual data.
• Anomaly Detection Use cases: Industrial defect detection, security surveillance, fraud detection.
• Anomaly Detection Algorithms/Libraries: Autoencoders, Isolation Forest, PyTorch-based models.
• Video Analysis: Processes and interprets videos for insights like action recognition or event detection.
• Video Analysis Use cases: Behavior analysis, sports video analytics, event monitoring.
• Video Analysis Algorithms/Libraries: OpenCV, PyTorchVideo, MMAction2.

Explore image processing basics like enhancing images and extracting key features. Understand how computer vision uses these techniques to enable machines to 'see' and interpret images. Learn the differences between image processing and computer vision.