Lecture 1_Introduction to Remote Sensing.pdf

Orientation LSGI536 Remote Sensing Image Processing Dr. Zhiwei Li Research Assistant Professor Department of Land Surveying and Geo-Informatics The Hong Kong Polytechnic University Email: [email protected] LSGI536 Remote Sensing Image Processing Instructor: Dr. LI Zhiwei Email: zhiwei.li@polyu....

Orientation LSGI536 Remote Sensing Image Processing Dr. Zhiwei Li Research Assistant Professor Department of Land Surveying and Geo-Informatics The Hong Kong Polytechnic University Email: [email protected] LSGI536 Remote Sensing Image Processing Instructor: Dr. LI Zhiwei Email: [email protected] Office: ZN613, Block Z, PolyU Website: https://zhiweili.net/ Teaching Assistants: YE Longjie, [email protected], Z418 XU Shaofen, [email protected], ZS1015 (For Lab Exercise only) Lecture: 18:30 to 21:20 PM, Wednesday, Room Z406 Lab Exercise: 18:30 to 21:20 PM, Wednesday, Room ZN604 2 Objectives principles and technology for remote sensing image acquisition, characteristics of remote sensing image data; methodology for geometric and radiometric processing of remote sensing images for quality improvement; techniques for interpreting information from remote sensing image data; principles of machine learning; the applications of machine learning for remote sensing image processing and analysis 3 Intended Learning Outcomes be familiar with the basic physical principles of remote sensing imaging; be familiar with common remote sensing platforms, sensors, and images; master practical skills in remote sensing image processing and analysis, including geometric and radiometric pre-processing, image enhancement, and image interpretation; discuss the various factors that influence the accuracy of information and features extracted from remote sensing images; acquire knowledge of machine learning algorithms; master how to apply machine learning for remote sensing image processing and analysis; design and implement research projects in the fields of remote sensing and machine learning; 4 Tentative Teaching Schedule Week 1 Date (Wednesday) Jan 17 2 Jan 24 3 Jan 31 4 Feb 7 Lecture Topic Lab Exercise Lecture 1. Introduction to Remote Sensing Lecture 2. Remote Sensing Fundamentals: Platforms, Sensors, and Image Characteristics Lecture 3. Remote Sensing Image Preprocessing Exercise #1. Remote Sensing Image Pre-processing Lunar New Year Break Lecture 4. Remote Sensing Image Enhancement 5 Feb 21 6 Feb 28 7 Mar 6 Lecture 5. Remote Sensing Image Interpretation 8 Mar 13 Lecture 6. Machine Learning for Remote Sensing Image Processing: Part I 9 Mar 20 10 Mar 27 11 Apr 3 12 Apr 10 13 Apr 17 Apr 25 - May 11 Exercise #2. Remote Sensing Image Enhancement Exercise #3. Remote Sensing Image Interpretation Lecture 7. Machine Learning for Remote Sensing Image Processing: Part II Exercise #4. Deep Learning for Remote Sensing Image Processing Lecture 8. Selected Topics in Advanced Remote Sensing Applications Project presentation Final examination Assessment Methods Tasks Weighting Level Lab exercises 30% Easy Requirements Submit lab report for each lab exercise ▪ Group project 40% (15% for presentation, 25% for report) ▪ ▪ ▪ Final examination 30% Total 100% Easy Each group includes 4-5 students (male & female). Project covers the topic, introduction, data, method, results, discussion, and conclusions. Final PPT presentation (April 17, 15 min for presentation and Q&A). Final project report (April 30, similarity < 15% and AI rate < 15% in Turnitin). 2 hours, close-book exam 6 Assessment Policy To pass this subject, students must attain a minimum grade of "D-" in final examination. There will be 3-5 random checks of class attendance, and their outcomes will be considered in the final grade calculation. Absence from in-class tests must be accompanied by a valid certificate. Mark ≥ 93 ≥ 88 and < 93 ≥ 83 and < 88 ≥ 78 and < 83 ≥ 73 and < 78 ≥ 70 and < 73 ≥ 68 and < 70 ≥ 63 and < 68 ≥ 60 and < 63 ≥ 58 and < 60 ≥ 53 and < 58 ≥ 50 and < 53 < 50 Grade A+ A AB+ B BC+ C CD+ D DF 7 Reading List and References Journals and Magazines Remote Sensing of Environment ISPRS Journal of Photogrammetry and Remote Sensing IEEE Transactions on Geoscience and Remote Sensing IEEE Geoscience and Remote Sensing Magazine International Journal of Applied Earth Observation and Geoinformation GIScience & Remote Sensing Books Campbell, J. B., & Wynne, R. H., Thomas V. A. (2022). Introduction to Remote Sensing (6th Edition). Guilford Press. Gonzalez, R. C. (2017). Digital Image Processing (4th Edition). Pearson. Lillesand, T., Kiefer, R., Chipman, J. (2015), Remote Sensing and Image Interpretation (7th Edition). Wiley. Morain, S. (2019), Manual of Remote Sensing (4th Edition), American Society for Photogrammetry and Remote Sensing. 8 LSGI536 Remote Sensing Image Processing Lecture 1 Introduction to Remote Sensing Dr. Zhiwei Li Research Assistant Professor Department of Land Surveying and Geo-Informatics The Hong Kong Polytechnic University Email: [email protected] Acknowledgment Instructors at LSGI who have taught this subject previously, including Dr. ZHU Rui, provided most of the slides for this lecture. 10 Outlines 1. Introduction to Remote Sensing 2. Physical Principles of Imaging 3. Digital Image Processing and Analysis 4. Remote Sensing Applications 11 Section 1 Introduction to Remote Sensing 12 What is “Remote Sensing”? “Remote sensing is the art and science of obtaining information about an object, area, or phenomenon through the analysis of data acquired by a device that is not in contact with the object, area, or phenomenon under investigation” — Lillesand and Keifer, 1987 “Remote sensing is the process of collecting, storing and extracting environmental information from images of the ground acquired by devices not in physical contact with the features being studied” — Robinson et al. 13 What is “Remote Sensing”? “Remote sensing is the art and science of obtaining information about an object, area, or phenomenon through the analysis of data acquired by a device that is not in contact with the object, area, or phenomenon under investigation” — Lillesand and Keifer, 1987 “Remote sensing is the process of collecting, storing and extracting environmental information from images of the ground acquired by devices not in physical contact with the features being studied” — Robinson et al. 14 What is “Remote Sensing”? Video Link 15 Scope of Remote Sensing Platform carries instruments to measure Electromagnetic Radiation reflected/emitted from the Earth Atmospheric system. The sensor may be at different locations with respect to the object under consideration. Yamazaki, F. (2001). Applications of remote sensing and GIS for damage assessment. 16 System Types Passive or Active Passive systems record reflected, naturally occurring radiation (e.g., natural light, infrared radiation) Active systems record the reflection of emitted radiation (e.g.， radar, sonar) 17 Remote Sensing Process 1. Energy source or illumination 2. Radiation and the atmosphere 3. Interaction with the target 4. Recording of energy by the sensor 5. Transmission, reception, and processing 6. Interpretation and analysis 7. Application Source: Dr. Tanushree Kain, 2023 18 Why “Remote Sensing”? Because it can Observe and collect data across a large region, in which some areas are extremely difficult or even dangerous to access, e.g. arctic, ocean; Provide global coverage; Allow repetitive measurements over years as historical data and provide long-term change; Cost effective, once data is collected, it can be shared with many people; and Instruments/sensors can detect light that beyond human vision. 19 Brief History ▪ 1910-20 World War I: the beginning of photo interpretation. ▪ 1962 The term "Remote Sensing" first appeared. ▪ 1972 The launch of Landsat-1, originally ERTS-1, remote sensing has been extensively investigated and applied. Today images are recorded in different ways - close to the ground, aircraft, satellites, and ocean vessels. Each of these has many different forms but each records information about the object of interest without contact through the recording of radiation of some form of energy. 20 Aerial image of Paris in 1858 21 First aerial photograph Gaspard-Félix Tournachon (1820 – 1910), known by the pseudonym Nadar, was a French photographer... In 1858, he took the first ever aerial photograph of Paris from a balloon. Experiments were also conducted with pigeon cameras. Afterwards, cameras were mounted in aero planes taking photos for large ground areas. https://en.wikipedia.org/wiki/Nadar 22 Famous photograph: San Francisco in Ruins San Francisco lies in ruins on May 28, 1906, about six weeks after the 1906 San Francisco earthquake and fire. It was taken from a camera suspended on a kite, perhaps 1,000 feet above the city. It is one of the most well-known photographs of George R. Lawrence. https://en.wikipedia.org/wiki/File:San_Francisco_in_ruin 23 Aerial photograph in World War I Some English observers started using cameras to record enemy positions and found aerial photography easier and more accurate than sketching and observing. The aerial observer became the aerial photographer. [Link1, Link2] 24 Aerial photograph in World War II Photo taken June 23, 1943 of the V-2 test lunch site at Peenemunde after bombing raid [link] 25 Spy Planes U-2 Spy Plane 1954-1960 Flew at 21,300 meters over USSR 26 Spy Planes U-2 aerial photograph of an airfield in the Soviet Union [link] 27 Satellites 705 km 28 Section 2 Physical Principles of Imaging Electro-magnetic radiation Energy interactions in the atmosphere 29 Units of measurement Common units for wavelength: Meters (m) Centimeters (cm) Micrometers (μm) Nanometers (nm) For example: Visible light is in the range from about 0.4 μm to approx. 0.7 μm or about 400 nm to approx. 700 nm. Light Year astronomical distances 30 Electro-magnetic radiation https://youtu.be/lwfJPc-rSXw 5 mins > What is crest and what is trough? > What is the wavelength? > What is the frequency of the wave? What is the unit? > What are the shortest and highest energy waves? Video Link 31 Electro-magnetic radiation What is crest and what is trough? What is the wavelength? What is the frequency of the wave? What is the unit? What are the shortest and highest energy waves? 32 Electro-magnetic radiation What is crest and what is trough? The highest and the lowest points of the wave. What is the wavelength? The distance between two consecutive crests or troughs. What is the frequency of the wave? What is the unit? The number of crests/troughs that pass a given point within one second is described as the frequency of the wave. Hertz or Hz What are the shortest and highest energy waves? Gamma rays. 33 Electro-magnetic spectrum 34 Electro-magnetic spectrum 35 Electro-magnetic spectrum Gamma Rays, Approximate locations of the 16 channels we use to peer through the atmosphere with the GOES-R series satellites. > The places where energy passes through are called “atmospheric windows”. > The wavelength ranges in which the atmosphere is particularly transmissive of energy are referred as Atmospheric Windows. > We use these "windows" in remote sensing to peer into the atmosphere from which we can obtain much information concerning the weather. 51 Atmospheric effects in visible (due to scattering) Radiance indicates how much of the power emitted, reflected, transmitted or received by a surface, which will be received by an optical system looking at that surface from a specified angle of view. The atmosphere can scatter light, adding to the radiation that is detected by the sensor. This is known as path radiance. ρ=Reflectance of object E=irradiance, incoming energy T=transmission of atmosphere https://en.wikipedia.org/wiki/Radiance 52 Energy interactions in the atmosphere All radiation detected by remote sensors passes through some distance, or path length, of Earth’s atmosphere. Particles and gases in the atmosphere can affect the incoming light and radiation. The net effect of the atmosphere varies with: Difference in path length Magnitude of the energy signal Atmospheric condition Wavelengths involved Two types of interactions are normally observed: Scattering Absorption 53 Energy interactions in the atmosphere 54 Atmospheric scattering Scattering occurs when particles or large gas molecules present in the atmosphere interact with and cause electromagnetic radiation to be redirected from its original path. The intensity of scattering depends on: the wavelength of the radiation the abundance of particles or gases, and the distance the radiation travels through the atmosphere Three types of scattering Rayleigh or molecular scattering Mie or non-molecular scattering Non-selective scattering 55 Why is the sky blue? https://youtu.be/ehUIlhKhzDA 3 mins 56 Why is the sky blue? The sunlight is scattered in all directions by the gases in the air when it reaches the Earth’s atmosphere. The types of gases mostly scatter the shorter and choppier waves of blue lights. 57 Atmospheric absorption Absorption is the other main type of interaction that electromagnetic radiation interacts with the atmosphere. Ozone: absorb the harmful ultraviolet radiation from the sun. Without Ozone layer our skin would get burnt when exposed to sunlight. Carbon dioxide: absorbs strongly in the far infrared region – the area associated with thermal heating – thus heat is trapped inside the atmosphere. Water vapor: absorbs much of the infrared and microwave radiation (between 22μm and 1m). 62 Section 3 Digital Image Processing and Analysis 63 Atmospheric correlation A MODIS image over South Africa Color processed to Atmospheric Optical Thickness (AOT) 64 Feature enhancement https://apps.sentinel-hub.com/sentinel-playground/ Enhance land features with different image band combinations 65 Contrast enhancement Example of Image Enhancement on IKONOS image 66 Image enhancement False color Infra-red SPOT image over Yuen Long 67 Image enhancement 68 Missing data reconstruction 69 Image fusion A small region of the (a) panchromatic image (0.6 m), (b) multispectral image (2.4 m), and (c) fused image (0.6 m). [link] 70 Image super-resolution An example of the 275 m Multi-angle Imaging Spectro Radiometer (MISR) red band image (left) super-resolved to 68.75 m super-resolution restoration (SRR) image (right). [link] 71 Object segmentation A framework for city-scale rooftop solar PV potential estimation was developed. Labor cost of DL was significantly reduced with proposed spatial optimization sampling strategy. Rooftop extraction model was proved to be robust in different districts. 311,853 GWh rooftop solar PV potential was estimated for Nanjing in 2019. 330.36 km2 rooftop area and 66 GW installed capacity were estimated for Nanjing. 72 Remote sensing image analysis tasks Sensor Sun Atmosphere Radiation Reflection Earth surface Imaging process of optical satellite imaging system for Earth Observation (Image Source: Prof. Shen H.F. et al.) 73 Remote sensing image analysis tasks 1. Image pre-processing Fundamental image processing procedures (e.g. image quality/resolution improvement) 2. Image processing and interpretation 3. Geophysical parameters retrieval Identify objects in images from a qualitative perspective Measuring the physical properties of objects from a quantitative perspective (e.g. object detection, image classification) (e.g. estimation of air pollutants, soil moisture) Example tasks of remote sensing image processing of water Image pre-processing (enhance water features, radiometric accuracy, spatial resolution, etc.) Water extent mapping Water quality monitoring (extent and its changes) (color, turbidity, etc.) 74 Remote sensing image analysis tasks 1. Image pre-processing (Restoration) Impulse noise reduction Stripe noise removal Deblurring Image Source: Prof. Shen H.F. et al. 75 Remote sensing image analysis tasks 1. Image pre-processing (Correction) Thin cloud removal Building shadow removal Radiometric normalization Image Source: Prof. Shen H.F. et al. 76 Remote sensing image analysis tasks 1. Image pre-processing (Reconstruction) Gap filling Thick cloud removal Land surface temperature reconstruction Image Source: Prof. Shen H.F. et al. 77 Remote sensing image analysis tasks 2. Image interpretation (Scene classification) General pipeline of three types of scene classification methods AID: A Benchmark Data Set for Performance Evaluation of Aerial Scene Classification 78 Remote sensing image analysis tasks 2. Image interpretation (Object detection) DOTA: A Large-scale Dataset for Object Detection in Aerial Images 79 Remote sensing image analysis tasks 2. Image interpretation (Image classification) Land use and land cover (LULC) classification 80 Remote sensing image analysis tasks 2. Image interpretation (Land use/cover change detection) Landsat data used for illustrating the Continuous Change Detection and Classification (CCDC) results 81 Remote sensing image analysis tasks 3. Geophysical parameters retrieval (Earth surface monitoring) PM2.5 mapping Urban heat island monitoring Monitoring of historical glacier recession Image Source: Prof. Shen H.F. et al. 82 Remote sensing image analysis tasks 3. Geophysical parameters retrieval (Earth surface monitoring) Vegetation index Soil moisture O3 mapping Image Source: Prof. Shen H.F. et al. 83 Section 4 Remote Sensing Applications 84 Land reclamation in Hong Kong Corona (1969) Landsat-8 (2016) http://www.geocarto.com.hk/edu/PJ-HKRECLAM/main_RECL_bootstrap.html 85 Meteorological satellite - FengYun-2 China's first geostationary meteorological satellites were named FengYun-2, or FY-2 satellites. https://youtu.be/Q-KXI9Ha3PU?si=-9TwOGXeI6nrAdfE 86 Nighttime lights of Earth Lights of Human Activity Shine in NASA's Image of Earth at Night https://www.youtube.com/watch?v=8dc58ZrOuck&ab_ channel=NASAGoddard What are the phenomena revealed from Earth lights? 87 Nighttime lights of Earth Night Lights Change in the Middle East (2012 & 2016) https://earthobservatory.nasa.gov/images/90100/night-lights-change-in-the-middle-east 88 Lake Chad 1963 to 2013 60,000 sq.km in 1963 40x size of HK The 6th largest lake in the world 89 Farmers around Lake Chad 90 SPOT image of fire Overview 2.5m resolution SPOT 5 images on 8th August 2005 showing smoke from forest fires in Sumatra 93 IKONOS image of fire 1m resolution 94 Terra ASTER image over Chengdu region before and after earthquake February 19, 2003 May 23, 2008 95 SPOT image over New Orleans October 30, 2001 and August 30, 2005 (24hrs after passage of Katrina hurricane) 96 Natural hazards in the World https://earthobservatory.nasa.gov/topic/natural-event 97 Landsat 7 ETM+ Collection 2 level-2 products Left: Landsat 7 level-2 surface reflectance image. Right: Landsat 7 level-2 surface temperature image. The data was acquired on August 19, 2020 (path 179 row 28). 98 Landsat 8 Ground-level PM2.5 Concentration https://doi.org/10.1117/12.2068886 99 Nuclear reactor in North Korea July 2012 US Military satellite images 101 Questions What are the remote sensing applications introduced? Can you introduce some different applications? 102 Summary Introduction to Remote Sensing Physical Principles of Imaging Digital Image Processing and Analysis Remote Sensing Applications 103 Homework - Satellite sensor specifications ❑ Landsat-8/9 ❑ Sentinel-1/2 ❑ MODIS ❑ SPOT ❑ ASTER ❑ PlanetScope ❑ WorldView 104 End of Lesson 1 105