Untitled

Introduction to 3D Photogrammetry

• Photogrammetry and Remote sensing combine to measure the 3D world using sensors.
• Key tasks include 3D data acquisition, object detection, geometric modeling, and semantic/topological recovery.
• Point clouds are sets of discrete 3D points that measure surfaces, with 3D coordinates and attributes, potentially containing noise which is managed using signal processing, pattern recognition, and machine learning to create digital twins.

Challenges and Key Techniques

• Challenges presented are Noise, outliers, complex backgrounds, incomplete objects, uneven densities, and a lack of topology.
• Key processing techniques include registration (aligning multiple datasets), segmentation (separating data into segments), and classification (assigning labels).

Acquisition of Point Clouds

• Point clouds consist of geometric (position) and radiometric (attributes) data, acquired using sensors.
• Data is recorded as a list of points [x, y, z, attributes] including rgb, intensity, number of returns, and incidence angle.
• Active methods (laser scanning) directly emit signals (LiDAR), measuring distances/angles to calculate 3D positions.
• Passive methods use image pairs or single images; stereo vision uses image pairs for depth triangulation, while monocular vision uses single images with prior knowledge/deep learning.
• SfM/MVS refers to Structure from Motion (image pairs) and Multi-view stereo (multiple images).

Representation

• Point clouds represent 3D values like elevation, color, and intensity.
• Interpretation requires understanding geometry, topology, and semantics.
• Segmentation divides point clouds into labeled segments (objects).

Point cloud Definitions

• Segmentation partitions data into segments with shared characteristics, assigning each segment a unique label.
• Semantic Segmentation partitions data into segments with the same characteristics and assigns predefined labels, ensuring labels are consistent.
• Classification assigns a single label to an entire point cloud.
• Object detection detects and locates objects of interest.

Semantic Understanding With Al

• AI mimics human intelligence.
• ML(Machine Learning) uses data algorithms to make predictions and is a subset of AI.
• DL(Deep Learning) is a subset of ML that uses multiple layers for data representation.
• Supervised learning uses labeled data.
• Unsupervised learning does not use labeled data.

Clustering – DBSCAN-unsupervised learning

• DBSCAN is a density-based clustering method, that uses parameters of minimum points and distance (epsilon).
• It identifies core and non-core points iteratively, clustering segments if core points are connected.
• DBSCAN handles any shape, needs no cluster number, good for spatial data
• Parameter selection is not trivial, computationally heavy, memory intensive.

Machine Learning Approaches

• Hand-crafted features are designed by experts for specific tasks, but may not generalize well.
• Random Forest (RF) builds decision trees using different subset data and features
• Predictions on unseen data involve a majority vote (classification) or average (regression) via trees.
• Geometric features are attributes from the 3D coordinates of points within a point cloud, higher-level understanding local surface characteristics leads to segmentation classification
• The decision trees in RF use geometric features as input data for predictions.
• Eigenvalue-based methods can determine Linearity, Planarity, and Scattering.
• Determining Gom 3D properties includes computing the radius of k-nearest neighbors and local point density
• Hand-crafted features can beat DL in user tasks, and data requirements are available.

Deep Learning Approaches

• Deep learning learns features via multiple layers of encoding features automatically.
• 3D deep learning methods-semantic segmentation models:
  • View-based methods use 2D snapshots and CNNs (convolutional neural networks), which pose scalability issues.
  • Voxel-based methods enforce structure on voxelized point clouds but are computationally heavy.
  • Graph-based methods computationally heavy operate on point cloud topologies.
  • Point-based methods operate on unstructured point sets directly.
• PointNet is a point-based method using T-Nets, MLPs, and max pooling, it inputs n points and outputs classification scores or semantic segmentation.

Data Science in Earth Observation

• Core Theme: Al's Increasing Role in Earth Observation (EO).
• Data Science in Earth Observation involves explorative signal processing, data fusion, information mining, ML, DL, big data analytics, and HPC.

Key Remote Sensing Technologies

• Satellite Remote sensing (RS) enables large-scale, contact-free Earth information gathering.
• Electromagnetic Radiation (EMR) carries the information that can be recorded:
  • Passive sensors record reflected solar EMR, such as optical and thermal data.
  • Active sensors emit and record reflected EMR, examples are LIDAR and SAR.
• Optical remote sensing deals with reflected sunlight, is affected by the atmosphere.
• Spectral signatures help identify materials like vegetation, soil, and water.
• Radar (SAR) uses complex-valued measurements.
• Atmospheric remote sensing studies ozone and other phenomena.
• EnMAP (hyperspectral), WorldView-2 and Sentinel-2 (multispectral), Sentinel-1 (SAR), and GOME-2/Sentinel series (atmospheric) are examples of key sensors.

Artificial Neural Networks (ANNs)

• History: Early models were limited (e.g., Perceptron).
• Minsky and Papert demonstrated the limitation of Perceptrons for non-linearly separable data.
• Backpropagation enabled multi-layer networks and the rise of Deep Learning (DL).
• Single-Layer Networks: Can only solve linearly separable problems, and cannot solve non-linear problems like XOR.
• Multi-Layer Networks:
  • Can approximate any continuous function with enough hidden layers via the Universal approximation theorem.
  • A one layer is equivalent to 1 line on the plot
  • Theoretical capabilities of NN using Boolean functions represented by a network with 2 hidden layers, and continuous functions approximated by a network with n hidden layers
• Deep Neural Networks (DNNs):
  • DL involves NN architecture, a loss function, training data, a training algorithm, initialization.
  • Convolutional Neural Networks (CNNs) are used for image analysis, including classification, object detection, and segmentation along with automatic feature extraction, and natural language processing -Recurrent Neural Networks (RNNs) manage sequential data processing, image captioning, and speech recognition. -Generative Adversarial Networks (GANs) handle image generation tasks like super-resolution, image translation, and text-to-image generation.

Al for Earth Observation (AI4EO) – Key Points

• AI4EO is not just about classification also about retrieving physical and biochemical variables;
• AI4EO demands high accuracy, traceability, and reproducibility and uses expert domain knowledge and well-controlled data.
• Data Characteristics multi-dimensional (x-y-z-t-x), complex-valued, and multi-modal (SAR, LIDAR, hyperspectral), its training data often is limited.
• Applications of AI4EO:
  • Object detection, segmentation and classification (buildings, ships, etc.)
  • Land use/land cover classification, change detection
  • Fusion of multi-modal data (SAR/Optical, 2D/3D)
  • Image synthesis (e.g. SAR to optical or vice versa)
  • Cloud removal, atmospheric sensing, climate studies, and social media data fusion.
• Open Methodological Challenges:
  • Integrating physics, Bayes and expert knowledge,transferability and uncertainty.
  • Ensuring reasoning, transferability, and uncertainty.
  • Addressing explainability, ethics and quantum machine learning, Focusing on Green Al.

EORS- Remote Sensing & Earth Observation

• Remote Sensing & Earth Observation (EO) Fundamentals Remote Sensing acquires info about Earth without contact:
• Sensing and recording energy, processing, analysis, and application is technology for obtaining info about Earth and environment:
• Its objectives are to understand Earth using remote data and insight for action, and intervention.
• Sensors: passive measures reflected/emitted natural energy, (e.g., sunlight), multispectral (few bands and hyperspectral (lot of bands)
• Active emits own energy and measures reflection (e.g., radar-long waves (micro) LiDAR-short VNIR); perspectives: In-situ, hand-held, air-borne, space-borne.

EO Data

• EO Data: #bigdata, few labels, multi-modal, global, contactless/non-destructive.
• Electromagnetic Radiation is Crucial for understanding remote sensing.
• EO Technologies & Missions:
  • Optical Sensors measure reflected sunlight, various bands-passive NASA: Landsat
  • ESA: Copernicus Sentinel-2 Sentinel-5(p). DigitalGlobe: GeoEye
  • WorldView Planet: RapidEye/SkySat
  • Dove DLR: DESIS- earth sensing imaging spectrometer, EnMAP-enviro mapping and analysis program.
• Laser & LiDAR: Active, measures distance, elevation; ICES at -2; ESA: Active
• RaDAR: Active, radio waves, all-weather; ESA: Copernicus Sentinel-1; And DLR TerraSAR-X and TanDEM-X.
• Combined Missions are multiple sensors and Mega-Contstellations swarms of CubeSats and increased temporal resolution, high spatial resolution
• Resolution Trade-Off's challenges are in achieving different types of resolution.
• Spatial (Smallest discernible feature (Sentinel-2: 10 mapx, Maxar 3 mapx.).Temporal (time it visits a site)
• The key points are Environmental Monitoring: Climate change, Agriculture: Vegetation, anomalies, crop yield, plant health
• Climate Modeling involves Biogeochemical cycles, Memory, water balance, Water Resources can track reservoirs and predict groundwater
• Other variables are temperature, sea level, biomass, magnetic field, trace gases.
• What can and can't be seen.

Summary

• Remote sensing involves multi data using ML.
• The observer sees reflection and emissions in electromagnetic radiation
• Limitations cloud cover, spatial resolution constraints or subsurface imaging with high resolution
• Interactions are observed with electromagnetic radiation, challenges and data analysis

Industrial Metrology

• Engineering Geodesy: Key ConceptsReality Capture: Geometric and semantic data acquisition/modeling of objects/areas is setting out through the transfer of geometric plan from model to construction site measuring object's geometric state over time.

Measuring & Applications

• Focus: Quality assessment, sensor systems, reference frames,Applications include facility and construction management, aerospace, Industrial Metrology measures shape & dimensions, ensure design compliance, and the range varies small to medium, Accuracy: High (µm to mm).
• The process includes data acquisition, pre-processing, processing target values.
• Sensor Systems for Industrial Metrology: Laser Tracker detects reflector-target beam, Measures angle & distance.
• Interferometry uses 2 frequencies, measures the difference, Most accurate length measurement.
• Measuring volume varies and carbon fiber is used

Other Key Sensors

• LIDAR sensor, carbon fiber arm
• Refraction index affected by temperature, humidity

Resolution

• Table showing various resolutions of different sensors

Mobile Mapping

• Static or Mobile. Platforms used: Cars, UAVs and handheld
• Methods overlap with aerospace sensor systems
• Challenges: GNSS limitations in cities caused limitations with vision

Global Navigation Satellite Systems

Definition:

• Satellite based system for navigation and positioning Goal:
• User obtain at any time, anywhere, static / moving his position Key features:
• Applications:
• Surveying
• Autonomous Driving
• Monitoring
• Scientific
• Atmosphere: Weather
• Space: Timing data
• The system is dependent on how it coordinates with time
• The reference frame is the benchmark

Key Concepts

• PSeudoranges with modilated frequencies give robust signaling
• Key factors are location satellites and interference
• The system requires integration with existing systems

Geodesy

• Techniques
• GOal, significance
• space techniques:

System

• Geometry of Earth’s gravity measurements are required
• Models and monitoring gravity with respect to mass transport from space
• Essential and varied climates and signals
• Future missions are constantly required