Photogrammetry Basics - Geomatics for Urban and Regional Analysis PDF

Geomatics for Urban and Regional Analysis (2024-2025) G. Bitelli Geomatics for Urban and Regional Analysis Basics of Photogrammetry From the basic principles to the new approaches...

Geomatics for Urban and Regional Analysis (2024-2025) G. Bitelli Geomatics for Urban and Regional Analysis Basics of Photogrammetry From the basic principles to the new approaches 2D/3D surveying techniques defined by object complexity and size Photogrammetry By Photogrammetry it is possible to perform 3D surveys from 2D photographic Airborne vs Terrestrial Photogrammetry images, providing vector data, 3D models or point clouds of objects 3D scanning is a range-based method and active technique, photogrammetry is a airborne or aero-photogrammetry, if the acquisition is carried image-based method, passive technique out from above. In this case, the photo camera is on board of This technique has been totally revolutionized in recent years, although its mathematical foundations are always the same… aircrafts/helicopters/drones and the object is the ground as it appears from that higher position. All the present maps are derived from the application of this kind of survey. terrestrial (close-range) photogrammetry, if images refer to objects located on the Earth’s surface, or next to it. It is carried out using photo cameras positioned at ground level e.g. architectural surveys, landslides monitoring, industrial applications, cultural heritage, forensics, … Geomatics for Urban and Regional Analysis (2024-2025) 1 G. Bitelli  Geomatics for Urban and Regional Analysis (2024-2025) G. Bitelli Basics of Photogrammetry How the Photogrammetry works Photogrammetry is a technique that allows the measurement of The four pillars have If an object is not flat the same height, an object without touching it. Measurement can be performed in you can not make although their images three dimensions exploiting photograms acquired by analogical clearly are not of the measurements from same length because and digital cameras. a single photo of perspective effects Originally born for architectural survey, it became the primary method for mapping through airplane flights; it is the first The 3D coordinates of points on an object can be determined by remote sensing technology based on the acquisition of objects’ measurements made in two or more photographic images taken geometric properties from photographic images. from different points of view. An appropriate processing of such images determines the artificial stereoscopic vision. This technique has been totally revolutionized in recent years, although its mathematical foundations are always the same The identification of common points on different images of the same object defines a line, or ray, of sight geometrically linking the 3D surveys can be today performed by image point to the corresponding object-point. Photogrammetry for different applications at very different scales, providing cloudpoints, The intersection in space of two or more of these rays determines 3D vector models of objects, orthophotos, etc. the 3D position of the point (according to the forward intersection principle). Basic concepts: why two (or more) images are needed to reconstruct the 3D How it works ground coordinates of a point The 3D coordinates of points on an object are determined by measurements made in two or more photographic images taken from different points of view The identification of common points on different images of the same object defines a line, or ray, of sight geometrically linking the image point to the corresponding object-point. The intersection in space of two or more of these rays The points A1, A2, A3 generate The point A is univoquely determines the 3D position of the point (→ forward the same A’ image point on defined from the two A’ intersection principle). the picture as the point A and A’’ image points A single image does not contain sufficient information to define the position and size of a three-dimensional object, which instead can be obtained from two photographs (a stereo-couple) that show the same object observed from two different points O1 and O2 (in general it is a single camera which photographs the object in different positions and times, first from O1 and then from O2). Basic concepts: the reconstruction of the 3D model for mapping acquired pictures O1-O2 → B = base at the acquisition time (during the flight) reduced 3D model O1-O1’ → b = reduced base (in lab) A little bit of history During the reconstruction phase (→ restitution or plotting), by approaching real world the acquisition centers O 1 A story that goes back a long way in time. and O2 along their joining line, a 3D model of the But what were the origins? ground (for the overlapping area between 2 photos) is obtained at a reduced scale equal to the ratio b/B. This model will be used to produce the map. map Geomatics for Urban and Regional Analysis (2024-2025) 2 G. Bitelli  Geomatics for Urban and Regional Analysis (2024-2025) G. Bitelli The term Photogrammetry was introduced by Jordan in 1876 The photos are placed in the The beginning same relative orientation of Although the studies on a projective transformation from a perspective image to the acquisition moment an orthogonal one certainly preceded the invention of photography, the idea of using this as a means of measuring the territory was born almost simultaneously with the emergence of the photographic procedure. Already in 1851, the captain of the French genius Aimé Laussedat began his studies to replace hand-drawn A 3D model of the overlapping area is visually perspectives with photographic ones. explored by the operator, who In 1858 he created a first photogrammetric machine, with can move on it a mark along which he carried out topographic surveys; the process was the objects to be mapped called iconography by Laussedat. At the same time, Prof. Ignazio Porro in Italy carried out similar research, but using a more rigorous scientific procedure, which he defined as spherical photography. His experiments, starting from 1855, gave rise to photogrammetric studies in Italy. Schema of projection analogue plotting (early 1900s) The interest aroused in military environment by the Laussedat experiments quickly spread beyond the French borders. Photography Small-scale military maps were considered vital. from above The construction of the first phototheodolites, that is, of cameras joined to a goniometer, was a progress. An example was the one produced for Albrecht 1861 – The US Army Baloon Corp is founded, Meydenbauer, the father of architectural balloons are used during the American Civil War photogrammetry. From the historical collection at DICAM Dept. in Bologna The phototheodolite generally also allowed Glass sheet 40x40 cm2 inclined pictures and its birth coincided with the requests photographs on glass plate for greater precision aimed at 1858 - Gasper Felix photogrammetry, both in the Tournachon «Nadar»: case of cartographic first aerial photo of applications and in that of Paris architecture. Kites and kite trains 1903 – Bavarian Pigeon Corp A picture every 30’’ 1906 – Terremoto di San Francisco (fot. obliqua) Geomatics for Urban and Regional Analysis (2024-2025) 3 G. Bitelli  Geomatics for Urban and Regional Analysis (2024-2025) G. Bitelli The advent of the airplane First development of aerial photography for military purposes during the First World War 1903, Carolina del Nord: the «Flyer» by the Wright brothers photographed during the first extended flight of a motorized and controllable aircraft On April 15, 1909 Wilbur Wright took off for the first time in Italy with a biplane with a 22 HP four-cylinder engine: the photographic documentation of this historic event was taken with the aircraft still on the ground and during the flight. flying over the plane with a balloon. The first Italian aircraft was built by Aristide Faccioli, from Bologna, in 1908. Airborne Photogrammetry A stereo-couple Bronte, ripresa realizzata con macchina Santoni Mod. I, il 24.10.1932, quota 3500 metri, negativi su lastre di vetro formato 13 x 18 cm. 1929 can be considered the beginning year of the productive exploitation of aerial photogrammetry. The achievements of Eng. Santoni (operating in Florence at Officine Galileo since = 3D model 1940) allowed the IGM to launch a large program of aerial photogrammetric survey at the 1:25,000 scale. In 1948, both the new film camera model IV, which introduces the new 23 x 23 cm2 format, and the new Stereosimplex stereoplotter will be presented with great success. photo Classical airborne photogrammetry number Nominal focal lenght S/N camera data strip repère Classical analogic aerial picture (23 cm x 23 cm) Geomatics for Urban and Regional Analysis (2024-2025) 4 G. Bitelli  Geomatics for Urban and Regional Analysis (2024-2025) G. Bitelli From the Analogue historical Analogic stereoplotter: the relative position of the two plates collection at Photogrammetry DICAM Dept. reproduces the situation at the moment of the photo acquisition in Bologna The situation at Right image the photo Left image acquisition stage is reproduced by a complex Stereocartografo Santoni 3 (1934) mechanical-optical system Examples of stereoplotters from the DICAM collection of ancient surveying binocular eyepiece (left eye sees the left instruments… image, right eye sees the right image) Stereosimplex IIc (1948) Analogue Analytical Digital Analogue vs Analytical vs Digital Photogrammetry Photogrammetry analogue photogrammetry: the image is generated and supplied to the process on a photographic support (film, slides or printings), orientation and plotting process performed by a very precise mechanical-optical system analytical photogrammetry: image on a photographic support, orientation and plotting process performed by a computer system digital photogrammetry: the image is supplied and processed in digital format, many parts of the process are Three generations performed automatically. Digital images can be directly acquired by sensors or obtained from digitization of analogue images (slides or printed copies). Analytical Photogrammetry: the plotting process is supported by a computer Analytical stereoplotting Kern DSR14 PG2 The operator puts the collimation mark on a point into the 3D model, and the program solves in real- time the collinearity equations storing the X,Y,Z coordinates of the point in the computer. The operator can see his work in graphical form on Galileo Siscam Digicart40 the monitor. Leica SD3000 Geomatics for Urban and Regional Analysis (2024-2025) 5 G. Bitelli  Geomatics for Urban and Regional Analysis (2024-2025) G. Bitelli Digital Photogrammetric Workstations A cheap simple system for stereoscopic vision: 3D anaglyph = digital imagery (acquired directly or obtained scanning a film) + automatic procedures Red/cyan glasses Anaglyph 3D is the name given to the stereoscopic 3D effect achieved by means of encoding each eye's image using filters of different (usually chromatically opposite) colors, typically red and cyan. Recent digital photogrammetric approaches, derived from Computer Vision Products from aerial photogrammetry Structure from Motion (SfM) technique: cloud of points produced by automatic photogrammetric processes. From the point clouds, meshes (surfaces) can also be realized. Note that in the classical photogrammetry the points are instead individually collimated and identified by the operator! Products from aerial photogrammetry Products from close-range photogrammetry Mile High Stadium, Denver 5/1998 8/1999 11/2000 Variazione dell’area e dello spessore del ghiacciaio Triglav nel corso di 25 anni QuickBird Pan 4 agosto 2003 Geomatics for Urban and Regional Analysis (2024-2025) 6 G. Bitelli  Geomatics for Urban and Regional Analysis (2024-2025) G. Bitelli Industrial Products from close-range photogrammetry applications Forensic applications Altri crash scene esempi… near Salt Lake photogrammetric evidence markers City, UT Civil Engineering Vector (drawing) products topographic maps: 2D or 2.5D maps, if the third dimension is treated by contour lines and spot heights, and buildings can be eventually represented by simple boxes). Rarely true 3D. thematic maps: they give information about specific territorial themes like road conditions, hydrography, land cover, slope and exposition. height profiles: they represent the way the terrain height varies compression test for a pillar along a line (profile) defined through the area of interest; 3D models of objects: the geometric primitives are represented by their vertices coordinates, measured in a 3D mode; object volumes are inferable in this representation. Cloud of points and meshes representing the surface (most recent techniques). Raster (image) products Orthophoto or orthophoto-plane: A photo-map (where the objects are represented in their correct plane position) obtained by an analytical transformation of the acquisition perspective into an orthogonal projection. It requires one (or more) oriented picture/s and the knowledge of the object surface Basic geometrical aspects Orthophoto-mosaic: Mosaic of contiguous orthophotos Rectified image or photoplane: Central projection of the original image with the same geometric characteristics of an orthophoto. This process can be adopted only in case of acquisition of plane objects, also by slant (oblique) imagery Photo-mosaic: Mosaic of rectified images Geomatics for Urban and Regional Analysis (2024-2025) 7 G. Bitelli  Geomatics for Urban and Regional Analysis (2024-2025) G. Bitelli A photograph is realizing a central projection Straight lines remain straight Lines meet that are parallel in reality Image courtesy: Schindler Image space The central perspective geometry linking the object and the image space Object space Image courtesy: Förstner Note: positive representation Basic geometric aspects A common A map is a picture is a orthogonal projection central projection Image point (x,y) Reference plane Reference plane (map) Ground coordinates to be calculated (X,Y,Z) l (23 cm) c (focal lenght of the camera)) H (flight H0 (absolute height) height relative to the terrain) B pL p=ricoprimento area of the model L c l   s photoscale H L f The base of Photogrammetry: the collinearity equations Classic orientation processes and related parameters Interior orientation Aligned on the same line are: From image coordinates to object space: (the parameters are the same for all the images if a single camera is used) Projection center r x  r12 y  r13c c focal lenght Image point X  X 0  (Z  Z0 ) 11 Ground/Object point r31 x  r32 y  r33c Parameters of the lens distortion Coordinates of the repères (defining the 2D system useful to measure photocoordinates r21 x  r22 y  r23c on the image) Y  Y0  (Z  Z0 ) r31 x  r32 y  r33c They are provided in the calibration certificate (produced by laboratory operations on the camera) From object space to image coordinates: Exterior orientation (each image has different data): r ( X  X 0 )  r21(Y  Y0 )  r31(Z  Z0 ) x  c 11 r13( X  X 0 )  r23(Y  Y0 )  r33(Z  Z0 ) X0, Y0, Z0 coordinates of the projection center angles of the rotation matrix R (rij) r ( X  X 0 )  r22 (Y  Y0 )  r32 (Z  Z0 ) y  c 12 r13( X  X 0 )  r23(Y  Y0 )  r33(Z  Z0 ) They were normally derived in indirect mode by the knowledge of the X, Y, Z coordinates of some Ground Control Points (measured by GPS, or by surveying instruments, or eventually extracted by a good larger scale existing numerical map). X, Y, Z 3D ground reference system (e.g. cartographic system for airborne photogrammetry) Today they can be acquired in direct mode using on-board devices (kinematic GPS for X0, x, y coordinates of the image point (measured on the 2D image reference system) Y0, Z0 coordinates, Inertial Mapping Unit (IMU) for the angles) Geomatics for Urban and Regional Analysis (2024-2025) 8 G. Bitelli  Geomatics for Urban and Regional Analysis (2024-2025) G. Bitelli After the internal and external orientation it is possible to obtain the The process of cartographic three coordinates X, Y, Z, of each point of the object that is in common between at least two images. It is a forward intersection production operation → drawing by collimating the single points Mapping is today realized almost always by aerial photogrammetry A well established process Different skills required Complex task, to be tested in its various phases Expensive, the production of a map requires a lot of time Approximate cost for map production by aerial photogrammetry 1:10000 3-5€ / ha 1:5000 6-10€ / ha 1:2000 25-45€ / ha (not up-to-date, to be verified) 1:1000 200-300€ / ha Image courtesy: Schindler 1:500 500-1250€ / ha A strip of aerial images model B base B pL model p=longitudinal coverage between two images (es. 60%) In terrestrial (close- range) photogrammetry Automation of procedures in the acquisition schema can be completely Digital Photogrammetry different in respect to aerial case (e.g. convergent images) Geomatics for Urban and Regional Analysis (2024-2025) 9 G. Bitelli  Geomatics for Urban and Regional Analysis (2024-2025) G. Bitelli The process in a Structure from Motion – Multi View software The “Matching” procedure in digital photogrammetry The basic problem of Matching: given two digital images, automatically find the homologous (conjugates) points 3D survey by UAV Main phases: 3D survey of a statue Selecting a single entity Search of the homologous entity on the other image(s) on the basis of a criterion of similarity (correlation) on the pixels Calculation of the 3D position of the entity in the final reference system Evaluation of the quality of the matching Notes: not all the points may have a homologous (eg. occlusions,...) stereoscopy is not required, there is no a priori constraints on the geometry and nature of the images (terrestrial or aerial photographs, satellite, etc.). the technique can also be adopted for more than two images SfM-MVS workflow 1. Feature Point Detection - Characteristic points (descriptors) extracted by specific algorithms on each image - Quality: linked to the resolution of the image (original or resampled at lower resolution) - Number of points: e.g. 40000 Interior and Exterior - Saved for each image in a database Orientation elements determined and 2. Point/Descriptor Matching Typical workflow in generation of the - Point comparison for each image pair in the dataset a Structure from sparse cloud - Maximum number of tie points for each pair - Location information from EXIF data can be Motion software considered, if available (Metashape by Agisoft) - note: if the image number doubles, the time for the alignment does not double but is much greater - At the end, a tie points database is available, that will be used in the projective bundle adjustment 3. Bundle Adjustment - Collinearity equations: from the image space to the object: external and internal orientation parameters are derived for each image - The tie points constitute the sparse cloud When a point is computed automatically (automatic tie point), or marked by the user (manual tie point or ground control point, GCP) on at least two images, the Extraction of features from the images 3D coordinates of this point are computed using the camera's internal and external parameters as well as the position of the point in the images. The objective of this step is to extract distinctive groups of pixels that are, to some extent, invariant to changing camera viewpoints during image acquisition. Hence, a feature in the scene should have similar feature descriptions in all images. The most well-know feature detection method is the SIFT (Scale- invariant feature transform) algorithm. The initial goal of SIFT is to extract discriminative patches in a first image that can be compared to discriminative patches of a second image irrespective of rotation, translation, and scale. From the representation of one image at different scales, which is technically done by computing a pyramid of downscaled images, SIFT computes for each keypoint a description of the associated patch. The description is typically stored in 128 bits. Geomatics for Urban and Regional Analysis (2024-2025) 10 G. Bitelli  Geomatics for Urban and Regional Analysis (2024-2025) G. Bitelli Dense point cloud Sparse point cloud generated from keypoints detected in images With the now known orientation of the images, it becomes possible to create a dense 3D model and compute orthophotographs. The essential step in this process is the computation of this denser 3D model. A Multi-View Stereo (MVS) algorithm is used to compute a dense estimate of the surface geometry of the observed scene. Because these solutions operate on pixel values instead of on feature points, this additional step enables the generation of detailed 3D point clouds (which can be meshed) from the initially calculated sparse point clouds, hence reproducing fine details present in the scene. Using a triangulation approach,from the calculated positions and In a sense, it's sort of like laser scanning, but applied as a processing orientation ofthe photos,a firstsparse pointcloud is determ ined by an operation to the images instead of directly in the field. iterative process using the tie points previously individuated Image Capturing - Recommendations SfM-MVS process: time required and kind Your scene/object should be well lit. of Processing Unit for the different phases Avoid shadows, reflections, and transparent objects. Take the photos in diffuse or indirect lighting, such as on an overcast day (outdoor) or using multiple light sources (indoor). Don´t use the flash setting on the camera. Do not change the focal length/zoom while shooting. Use a fixed focal length lens if possible. Try to take pictures from all angles. Avoid moving objects in the scene or background. If taking pictures using a rotating rig, make sure to use a plain color background with no distiguishable features. The object of interest should always fill most of the image. Take images with a side overlap of 60% minimum For each shot, move to a new position (or rotate the object). Do not take multiple images from the same spot. For better coverage, you can photograph an area multiple times in different acquisition patterns. Avoid shaky, blurry, or warped images. The more images you have, the better. You can always filter out repetitive or poor quality images to reduce processing time. Example of reconstruction of the 3D model of an object with approach Structure from Motion: an ancient element of a hydraulic system in Peru, Nasca valley 1) Automatic determination of the For all possible pairs of interior orientation of images, the software the cameras and their recognized the potential position and homologous points (link orientation (exterior points): in the figure, the blue orientation), thanks to lines indicate the pairings the extraction of with a high degree of approximately 400,000 probability between two homologous points generic images, the red lines recognized through those with a low level of matching between the probability available images. 2) Starting from this data, a 3D point model of the object is 110 images analyzed, corresponding to as many blue reconstructed rectangular frames in the 3D representation Geomatics for Urban and Regional Analysis (2024-2025) 11 G. Bitelli  Geomatics for Urban and Regional Analysis (2024-2025) G. Bitelli Sparse point cloud, made up The various of 400 thousand tie points products of the Structure from Motion process (detail of the Dense point cloud, made of structure) 31.5 millions of 3D points Simplified mesh, made up of approximately 6 million triangles Mesh textured with the RGB color of the images The final, explorable and measurable 3D model. The model is georeferenced by knowledge of the GNSS coordinates of some Ground Control Points. Real case examples Real case examples Ground survey (Bridge) Drone 3D mapping (Bentley Systems office) Real case examples Photogrammetry vs LiDAR in vegetated areas City mapping (Marseille) Geomatics for Urban and Regional Analysis (2024-2025) 12 G. Bitelli