Photometric Image Formation

• Photometric Image Formation refers to how images are created based on light, color, and intensity values
• Discrete color or intensity values, geometry & projection
• Camera optics & sensor properties, lighting & surface properties must be considered for image formation.
• In Photometric image formation, light is emitted from a source, strikes a surface, interacts with the surface, and then reflected light enters a camera aperture

How are images formed?

• Light is emitted from a light source
• Light hits a surface
• Light interacts with the surface
• Reflected light enters a camera aperture
• Sensor of camera interprets light

Simplified Photometric image formation

• Simplified model emits light from one or more light sources
• Light is then reflected from an object’s surface
• A portion of this light is directed towards the camera
• This simplified model ignores multiple reflections, which often occur in real-world scenes.

Light Emission

• Point light sources radiate (emit) light uniformly in all directions.
• Properties of light include color spectrum (wavelength distribution) and intensity (Watts / Area * Solid Angle)
• The visible light spectrum is the section of the electromagnetic radiation spectrum that is visible to the human eye, ranging from 435-380 to 740-625 nanometers.
• A solid angle is like a cone
• “Area” light sources, like fluorescent lights, are a little different.

Light Interaction with a Surface

• Some light is absorbed by the surface color, while some is reflected.
• The orientation of a surface is defined by its "normal vector" which sticks straight up out of the surface.
• The bidirectional reflectance distribution function (BRDF) expresses the amount, direction, and color spectrum of reflected light.
• BRDF depends on the amount, direction, and color spectrum of incoming light
• Simplified BRDF is modeled with "Lambertian", "flat" or "matte" component where light is radiated equally in all directions.
• Also models, "Specular", "shiny", or "highlight" component where radiated light is reflected across the normal from the incoming light
• The diminution of returned light depends on the cosine of the angle between the incident light direction and the surface normal
• Mirror (specular) reflection has the incident light ray direction reflected onto the specular direction around the surface normal

Light Entering a Camera

• Red triangle (behind camera) and blue triangle (in front of camera) are similar
• The relationship between focal length (f), image distance (u) and object distance (X), is shown by: f/u = Z/X, where Z is the scene depth
• With any three terms, the fourth can be determined
• For a given focal length, “Lens Equation” is represented by 1/z₀ + 1/zᵢ = 1/f
• From the lens equation, Zi = 1 / (1/f - 1/Zo)
• When projections of objects aren't focused on the image plane, a "blur circle" or "circle of confusion" results
• The size of the blur circle depends on the distance to the object and the size of the aperture
• The allowable size of the blur circle (e.g., a pixel) determines the allowable range of depths in the scene ("depth of field")
• Images are formed through Film, Digital cameras, and the Eye

Sensor Arrays

• Images from Sensors are created through Continuos image projected onto a sensor array; Result of image sampling and quantization when generating a digital image
• The basic process of generating a digital image includes taking a scan line from the continuous image, illustrating sampling and quantization
• Sampling is taking sample digital representations and quantizing by assigning values.

Images as Functions

• An image can be thought of as a function, f, from R² to R
• f(x, y) gives the intensity at position (x, y).
• With images, expect the image only to be defined over a rectangle, with a finite range: f : [a,b]x[c,d] → [0,1]
• A color image is just three functions pasted together, expressed as a vector-valued function: f(x,y) = r(x,y) g(x,y) b(x,y)

Digital Images

• (x, y) are the spatial coordinates in an image
• The values a and b define the range of x (horizontal) coordinates.
• The values c and d define the range of y (vertical) coordinates.
• A finite rectangle [a, b] x [c, d] represents where the image exists.
• The function f(x, y) is undefined outside this rectangular region because the image doesn't exist beyond its borders.
• A digital image with width = 1920 pixels and height = 1080 pixels is defined as: x ranging from a = 0 to b = 1919, y ranging from c = 0 to d = 1079
• Each pixel at (x, y) has an intensity value (for grayscale) or RGB values (for color images).
• Digital (discrete) images are used to Sample the 2D space on a regular grid, and quantize each sample (round to the nearest integer)
• If our samples are Δ apart, then f[i,j] = Quantize{f(iΔ, jΔ)}
• The values can be represented as a matrix of integer values.
• Bayer color filter arrays capture the full colors in an image

Visible light

• EM energy from the sun, in specific wavelengths, is what is identified as colors.

Color Sensing

• RGB values are estimated at 'G' cells from neighboring values in the Bayer grid

Image representation

• Images are represented as a matrix for NxM RGB images.
• im(1,1,1) = top-left pixel value in R-channel
• im(y, x, b) = y pixels down, x pixels to the right in the bth channel
• im(N, M, 3) = bottom-right pixel in B-channel
• imread(filename) returns a uint8 image (values 0 to 255) that is convertible to a double format (values 0 to 1) with im2double

Color space

• Colors are represented in multiple combinations of Red, Green and Blue.

RGB Colour Space

• RGB color space is easy for devices, but not perceptual
• RGB does not easily identify properties such as, where do the grays live or where is hue and saturation.

CMYK Color Space

• CMYK works like paint and includes white is background color (all colors)
• Opposes the RGB color space

HSV Color Space

• A perceptual dimensions of the color space includes Hue ( "kind" of color), Saturation, and Value (total amount of light)
• Use rgb2hsv() and hsv2rgb() in Matlab, in Python w/skimage

Linear Systems

• Linear system is defined as a unit that converts an input function f(x) into an output (or response) function g(x) where x is an independent variable like time, in the case of images it's spatial position
• Assume for simplicity that x is a continuous variable, results derived should be equally applicable to discrete variables.

Lenses

• A lens focuses parallel rays onto a single focal point.
• Focal point is at a distance f beyond the plane of the lens, where f is a function of the shape and index of refraction of the lens
• Aperture of diameter D restricts the range of rays, which may be on either side of the lens
• Lenses are typically spherical (easier to produce).
• Real cameras use many lenses together to correct for aberrations
• Thin lens equation defines the relationship between object and images as 1/d₀ + 1/dᵢ = 1/f
• Any object point satisfying this equation is in focus
• The thin lens assumption assumes the lens has no thickness, which isn't true.

Open CV Library

• Read an image from file using cv::imread
• Display an image in an OpenCV window using cv::imshow
• Write an image to a file using cv::imwrite
• Basic drawing functions include drawing lines, rectangles, and circles, and adding text to said objects
• Images are read as Numpy arrays and split into channels
• dogface = input_image[60:250, 70:350]
• plt.imshow(dogface)
• flipped_code_0=cv2.flip(input_image,0) # vertical flip
• plt.imshow(flipped_code_0)
• flipped_code_1=cv2.flip(input_image,1) # horizontal flip
• plt.imshow(flipped_code_1)
• transposed=cv2.transpose(input_image)
• plt.imshow(transposed)

Image transformation

• Image filtering involves changing the range of an image: expressed as g(x) = T(f(x))
• Image warping involves changing the domain of an image, expressed as g(x) = f(T(x))

Parametric (global) Warping

• Examples of parametric warps: translation, rotation, aspect, affine, perspective, cylindrical
• Transformation T is a coordinate-changing machine: p' = T(p)
• If T is global then it's the same for any point p and it can be described by just a few numbers (parameters)
• A linear T can be represented as a matrix: p' = Mp

Scaling

• Scaling a coordinate means multiplying each of its components by a scalar
• Uniform scaling means this scalar is the same for all components
• Non-uniform scaling uses different scalars per component
• In matrix form X'=aX and Y=bY and is expressed as a scaling matrix

2D Rotation

• The 2d rotation can be captured in matrix form where the non linear functions of is: x' = x cos(θ) - y sin(θ) + y cos(θ)

Homogenous coordinates

• In homogeneous coordinates, it's possible to add an extra dimension to 2D points.
• Point (x, y) in 2D becomes (x, y, 1) in homogeneous coordinates
• Now, translation can be written as a 3x3 matrix multiplication:

x = x+ t and y = y+ t

• a general 3x3 affine transformation matrix looks like this and its kept compatible with homogeneous coordinate systems