Lecture 5 Remote Sensing Image Interpretation PDF

LSGI536 Remote Sensing Image Processing Lecture 5 Remote Sensing Image Interpretation Dr. Zhiwei Li Research Assistant Professor Department of Land Surveying and Geo-Informatics The Hong Kong Polytechnic University Email: [email protected] Outlines 1. Introduction to Image Classification & Mach...

LSGI536 Remote Sensing Image Processing Lecture 5 Remote Sensing Image Interpretation Dr. Zhiwei Li Research Assistant Professor Department of Land Surveying and Geo-Informatics The Hong Kong Polytechnic University Email: [email protected] Outlines 1. Introduction to Image Classification & Machine Learning 2. Supervised Classification k-Nearest Neighbour Classifier (k-NN) Minimum Distance to Means Classifier (MDM) Maximum Likelihood Classifier (MLC) 3. Unsupervised Classification K-means ISODATA 4. Accuracy Assessment Confusion matrix Accuracy metrics 5. Change Detection Visual Inspection Post-classification comparison Temporal image differencing 2 Section 1 Image Classification & Machine Learning 3 Image classification Image to Information Earth Observation Land-Use/Land-Cover Classification Objective: automatically categorize all pixels using numerical, Spectral Pattern for each pixel. Uses more than one band: otherwise = Density Slice (just rely on brightness) This is a learning problem, i.e. learn from data, then predict what we want to know. 4 Image classification Land Use refers to what people do on the land surface (e.g., agriculture, commerce, settlement) Land Cover refers to the type of material present on the landscape (e.g., water, sand, crops, forest, wetland, human-made materials such as asphalt) 5 Machine learning Think about tremendous size of remote sensing images available from the Internet (e.g. 50 years of Landsat images), this deluge of data calls for automated methods of data analysis, which is what Machine Learning provides. Machine Learning is a set of methods that can automatically detect patterns in data (given image), and then use the uncovered patterns to predict future data (new image), or to perform other kinds of decision making under uncertainty (such as planning how to collect more data) 8 Machine learning Supervised learning Unsupervised learning 9 Remote Sensing Image Classification Methods Parameter Training? Output Classes Target Type Parametric Supervised Hard Per-pixel Non-Parametric Unsupervised Soft (fuzzy) Object-oriented Non-Metric Hybrid 13 Section 2 Supervised Classification 14 Supervised classification There are four distinct stages: Date preparation Training – Require splitting sample data into training sample and test sample. Classification – Select and apply model and algorithm Output Production of thematic maps, tables or statistics The classification output becomes a GIS input Needs knowledge of area and/or reference data 15 Supervised classification 17 Training stage in supervised classification Training stage – analyst identifies representative training areas – develops numerical description of spectral attributes of each cover type in scene. Analyst must develop training statistics for all spectral classes constituting each information class to be discriminated by the classifier. Examples 1) 2) ‘water’ may be represented by 4 or 5 spectral classes ‘agriculture’ may contain several crop types and each crop type may contain several spectral classes due to different planting dates, moisture conditions, etc. 18 Training area creation Main objective – to assemble a set of descriptive statistics Training areas must be representative and complete (rooftop segmentation example) ‘The point that must be emphasized is that all spectral classes constituting each information class must be adequately represented in the training set statistics used to classify an image’ (Lillesand and Keifer) 19 Training area statistics for one class, 6 bands 20 Training area Sample histograms for pixels in training areas for a cover type ‘hay’ 21 Training area Coincident spectral plots for training areas in 5 bands for 6 cover types including Water (W), Forest (F), Urban (U), Corn (C), Hay (H) and Sand (S) Very useful for examining the data before classifying Thermal band often very useful if optical bands have overlap 22 Classification stage Each pixel in the dataset is categorised into the land cover class it most closely resembles. If not similar to any, labelled ‘unknown’ Category label set to ‘output image’ Every pixel is classified – a pixel-by-pixel operation (unlike image enhancement and image filtering) 23 Supervised classification methods Several different Supervised Classification approaches: k-Nearest Neighbour Classifier (k-NN) Minimum Distance to Means Classifier (MDM) Maximum Likelihood Classifier (MLC) 24 Scattergram of training areas Pixel in 2-Dimensional space. Training area pixels in 2-D space. Higher dimension is possible if more bands are considered. Scattergram illustrates the pixel observations of six cover types. 25 Scattergram of real image Spectral ranges of the training areas on the scattergram (right) and the image classes to which they belong (left). 26 k-Nearest Neighbour classifiers (k-NN) Euclidean distance: No model to be fit. Given a query point x, find k training points closet to x and classify using majority vote among the k neighbours. If k=1, training error rate is 0, but test error may be quite high. Distance metric 28 Minimum Distance to Means Classifier (MDM) Classified by minimum distance between unknown pixel and each of the category means. May define a distance, if unknown pixel is farther than that, classified as unknown. Insensitive to different degrees of variance, i.e. pixel 2 is closed to mean of “S” but it is within variability of “U”. 29 Scatterplot training area pixels for SPOT image of Hong Kong, with cluster means 30 Maximum Likelihood Classifier (MLC) Equiprobability Contours defined by Maximum Likelihood Classifier Assumption of normality As we know distribution of the features, each can be described in terms of a Gaussian density, which has a mean vector and a covariance matrix. Given the parameters, it can be used to compute the probability of a given pixel being a member of a particular land cover class. Since a pixel can be a member of different classes at different degree, meaning clusters may overlap, so called soft cluster. 31 Probability density function for Maximum Likelihood Classifier 32 Equiprobability contours plotted against scatterplot of Hong Kong training area data 33 Comparison of SPOT image classified by Maximum Likelihood, and Minimum Distance to means classifiers 34 Comparison of SPOT image classified by Maximum Likelihood, and Minimum Distance to means classifiers 35 Section 3 Unsupervised Classification 36 Unsupervised classification Examine all bands for each pixel in an image - the process is iterative. Aggregate them into a number of classes based on natural groupings or clusters present in the image values. Result in spectral classes (clusters) and initially, their identities are unknown. Each class is then identified from knowledge or reference data. Operator intervention only after data has been placed into classes. 37 K-means It is a method for finding clusters and cluster centers in a set of unlabeled data. One chooses the desired number of cluster centers, say R, and the K-means procedure iteratively moves the centers to minimize the total variance within cluster. It is an iterative procedure. K-means works best for spherical cluster with same variance but elongated / oval cluster (larger variability). Use elbow method to determine k value. 38 K-means 39 K-means 40 ISODATA Iterative Self-Organizing Data Analysis Technique (ISODATA) is an unsupervised classification calculates class means evenly distributed in the data space then iteratively clusters the remaining pixels using the minimum distance techniques. No need to know the number of clusters prior Similar to k-means but it adds more operations to refine the classification result, e.g., split and merge. 41 ISODATA Cluster centers are randomly placed and pixels are assigned based on the shortest distance to center The standard deviation within each cluster, and the distance between cluster centers is calculated. Clusters are split if one or more standard deviation is greater than the user-defined threshold. Clusters are merged if the distance between them is less than the user-defined threshold. Adjust cluster centers based on pixel assignment Further iterations until Average inter-center distance falls below user-defined threshold Average change in the inter-center distance between iterations is less then a threshold, or The maximum number of iterations is reached 42 ISODATA classification of part of Guangdong 44 ISODATA Disadvantage: Time consuming if data is very unstructured. Advantages: Treat as a black box Little user effort Effective at identifying spectral clusters in data. 45 Classes resulting from Unsupervised Classification 46 Classification and post processing Classified image before (left) and after (right) postclassification process (majority filter) 47 Section 4 Accuracy Assessment 51 Accuracy assessment Error matrix (or confusion table, confusion matrix) Comparison of the relationship between known reference data and the corresponding results of an automated classification It is common to average the correct classifications and regard this as the overall classification accuracy 52 Accuracy assessment Overall Accuracy (OA): (TP+TN)/total = (100+50)/165 = 0.91 Producer’s Accuracy (PA) / Recall: TP/actual yes = 100/105 = 0.95 User’s Accuracy (UA) / Precision: TP/predicted yes = 100/110 = 0.91 Misclassification Rate: (FP+FN)/total = (10+5)/165 = 0.09 False Positive Rate: FP/actual no = 10/60 = 0.17 True Negative Rate: TN/actual no = 50/60 = 0.83 53 Example of confusion matrix Omission Error (col), Wrongly classified as non-water features, (FN) Training data (known cover types) Classification Water Concrete High Bare Grass Forest Row total results buildings soils slopes Water 93 0 2 1 0 0 96 Concrete 0 65 4 6 0 0 75 High buildings 2 3 124 5 9 12 155 Bare soils 2 3 21 165 24 12 227 Grass slopes 0 0 6 16 201 45 268 Forest 0 0 8 9 76 512 605 Column total 97 71 165 202 310 581 1426 Recall Commission Error (row), Wrongly classified as Water (FP) Precision W = 93/97 = 96% B = 165/202 = 82% W = 93/96 = 97% B = 165/227 = 73% C = 65/71 = 92% G = 201/310 = 65% C = 65/75 = 87% G = 201/268 = 75% H = 124/165 = 75% F = 512/581 = 88% H = 124/155 = 80% F = 512/605 = 85% Overall accuracy = (93 + 65 + 124 + 165 + 201 + 512)/ 1426 = 81% 124 sample points have been correctly classified as high buildings. For high buildings (H), recall is 124/165, and the precision is 124/155. But 2 genuine high building samples have been classified as water (1strow, 3rdcol), and 2 water samples have been classified as high buildings (3rd & 4th rows in 1stcol). 54 Interpretation of confusion matrix Recall (True Positive Rate) Correct classified pixels / column total (total pixels of pixels allocated to the class), e.g. 124/165=75% Meaning: There are 165 pixels which are actually High Buildings and only 124 of those are correctly assigned to High Buildings. Precision (Positive Predictive Value) Correct classified pixels / row total (total number of pixels classified as a class), e.g. 124/155=80% True Positive / Total Predicted Count Meaning: A total 155 pixels have been classified as High Buildings but only 124 pixels are actually High Buildings. 55 Accuracy assessment: kappa coefficient Training data (known cover types) Classification Water Concrete High Bare Grass Forest Row total results buildings soils slopes Water 93 0 2 1 0 0 96 Concrete 0 65 4 6 0 0 75 High buildings 2 3 124 5 9 12 155 Bare soils 2 3 21 165 24 12 227 Grass slopes 0 0 6 16 201 45 268 Forest 0 0 8 9 76 512 605 Column total 97 71 165 202 310 581 1426 Agreement between classification and true value 𝑝0 = 1 means perfect classification Agreement between classification and true value by chance xi+ total of observations in row i x+i total of observations in column i Where x+ixi+/N2 is the probability when raters A and B classify item i 0 < 𝜅̂< 1 If 𝑘෠ approaching 1 mean better classification result not by chance If 𝑘෠ = 0, it means the classification result just like random assignment 56 Accuracy assessment: kappa coefficient Kappa is always less than or equal to 1. A value of 1 implies perfect agreement, and values less than 1 imply less than perfect agreement. In rare situations, Kappa can be negative. This is a sign that the two observers agreed would be less than expected just by chance. Here is one possible interpretation of Kappa. Poor agreement = Less than 0.20 Fair agreement = 0.20 to 0.40 Moderate agreement = 0.40 to 0.60 Good agreement = 0.60 to 0.80 Very good agreement = 0.80 to 1.00 57 Section 5 Change Detection 58 Change detection Involves use of multi-temporal images to discriminate land cover change between different dates. Can be short term change e.g. flooding or vegetation ripening, or long term e.g. urban growth or desertification or sea level change. Choosing comparable images, e.g. same sensor, bands, spatial resolution, time of day. Anniversary dates (same day of the month, same day of the year, same time of day, same season) minimize seasonal and sun angle differences. Accurate spatial registration of images important e.g. 1/4 to 1/2 pixel Cloud free Radiometrically and atmospherically corrected. 59 Change detection Information that can be derived: Where and when does the change take place? How much and what kind of change has occurred? What are the cycles and trends in the changes? Examples: Changes in tree cover due to wild fire or land clearing. Urbanization Land degradation due to over grazing Categories of change: Change in shape, size, slow changes, quick changes 60 Change detection Three Decades of Earth Seen From Space https://youtu.be/2n8XX5NwYyI ~ 3 mins - Learn from it! You may also create your short video and add your group names in the video for the last few seconds. Landsat in Action - Land Cover and Land Cover Change https://youtu.be/5i_ZSvCruRk ~ 4 mins 61 Change detection Changes on the landscape can be detected as changes in the spectral value of pixels. Examples: Before burn, vegetation has high G and NIR reflectance but low in SWIR. After burn, burned areas have low G and NIR but high SWIR. The goal of change detection is to identify the geographical location and types of changes, quantification of changes and assessment of the accuracy of the change detection results. However, identifying the location and quantifying change is easy, identifying the cause of change is not. 62 Change detection methods 1) Visual Inspection 2) Post-classification comparison 3) Temporal image differencing Lillesand and Keifer Plate 9. LANDSAT MSS, Las Vegas 63 Visual inspection Visually describe the changes on a computer using GIS. Good for large changes such as size of large patches. Not good for subtle change. Usually based on specific band combination forming false colour image for easy visualization. 64 Band combinations for Landsat 8 Here’s a rundown of some common band combinations applied to Landsat 8, displayed as a red, green, blue (RGB): https://www.esri.com/arcgis-blog/products/product/imagery/band-combinations-for-landsat-8/ 65 Visual inspection Deforested areas are highlighted in magenta /purple and the areas that have increased vegetation are highlighted in green. r=SWIR, g=NIR, b=R Landsat 5 - 5 4 3 (1993 Tanzania) Landsat 8 - 6 5 4 (2016 Tanzania) 66 Visual inspection using spectral response Simple visualization of change using NIR band Red band = NIR band of 1st image, e.g. 1993 Green band = NIR band of 2nd image, e.g. 2016 Blue band = NIR band of 1st image, e.g. 1993 As a result, the image will show green colour indicates increasing vegetation, purple (Red + Blue) indicates decreasing vegetation and grey indicates no change. 67 Visual inspection using spectral response Green indicates increasing vegetation Purple (Red + Blue) indicates decreasing vegetation Grey indicates no change 68 Post-classification comparison Co-register images of different dates Classify land cover of two images separately Compare two land cover maps to identify change This identifies change after classification Shortcomings of this method Errors from each classified map will be multiple in the change map. Tends to ignore subtle changes within a class as it tries to classify land cover for two dates independently but not the changes, and some detailed changes could have been generalized at the first pace. 4 A 5 A 69 Temporal image differencing (single band) Image preprocessing Co-register images of different dates Convert DN to TOA reflectance Apply atmospheric correction Subtract image pixel values to generate change image No change can be indicated by change image values near zero Areas of change give larger negative or positive values Possible values -255 to +255 for 8-bit image: rescale by dividing the values by 2 and add 127 Determine threshold for change (remove marginal cases and increase the certainty) As a result, change can be identified within tails of histogram distribution 70 Temporal image differencing (single band) Can be used to detect subtle changes Easy to compute Can be difficult to interpret 71 Vegetation change detection based on NIR radiance For example: T1 = 20, T2 = 90, Difference: T1 - T2 = -70 (-70/2) + 127 (Scaling) = 92 Change Histogram No difference = 127 Thresholds for change = 150 72 Vegetation change detection based on NIR radiance Graph of lookup table for mapping change values (0100, and 150-255) to white Area without changes Areas of less or no change (101-149) set to black 73 Temporal image differencing (using NDVI) NDVI for 1993 Tanzania NDVI for 2016 Tanzania Red area indicates negative NDVI Yellow area indicates zero NDVI Blue area indicates positive NDVI 74 Temporal image differencing (using NDVI) Red colour indicates area with decreasing healthy vegetation (may due to urbanization, deforesting or fire), while blue colour indicates area with increasing healthy vegetation over the years. (NDVI in 2016) – (NDVI in 1993) 75 Summary 1. Introduction to Image Classification & Machine Learning 2. Supervised Classification k-Nearest Neighbour Classifier (k-NN) Minimum Distance to Means Classifier (MDM) Maximum Likelihood Classifier (MLC) 3. Unsupervised Classification K-means ISODATA 4. Accuracy Assessment Confusion matrix Accuracy metrics 5. Change Detection Visual Inspection Post-classification comparison Temporal image differencing 76 Summary of the eleven Land Cover and Land Use products Note: AI = Artificial Intelligence method, RF = Random Forest, DT = Decision Tree, CCDC = Continuous Change Detection and Classification, Various = MLC, DT, RF and SVM, MLC = Maximum Likelihood Classifier, SVM = Support Vector Machine. Source: Wang, Z., & Mountrakis, G. (2023). Accuracy Assessment of Eleven Medium Resolution Global and Regional Land Cover Land Use Products: A Case Study over the Conterminous United States. Remote Sensing, 15(12), 3186. 77 Near real-time mapping – Dynamic World Google Dynamic World https://dynamicworld.app/ Brown, C. F., Brumby, S. P., Guzder-Williams, B., Birch, T., Hyde, S. B., Mazzariello, J.,... & Tait, A. M. (2022). Dynamic World, Near real- 78 time global 10 m land use land cover mapping. Scientific Data, 9(1), 251. Near real-time mapping – Dynamic World Global distribution of annotated Sentinel-2 image tiles used for model training and periodic testing (neither including 409 validation tiles). Confusion matrix of Dynamic World to Three Expert Strict, i.e. valid where all three experts labeled and all agreed (n =27,841,623). 79 Near real-time mapping – Dynamic World Example of Dynamic World mode composite (February - September 2021), time series of class probabilities for single pixel (location indicated by circled white point), and select Dynamic World predictions with corresponding single-date Sentinel-2 images for temperate deciduous forest in Massachusetts, USA (centered on latitude: 42.491°, longitude: −72.275°). 80 Near real-time mapping – Dynamic World Mode composite of all Dynamic World NRT products from 2021-04-01 to 2021-05-01. Areas of black correspond to no data over land (due to cloud cover) with white corresponding to no data over water. 81 Near real-time mapping – Dynamic World Composited Dynamic World LULC map over Hong Kong during Jan 2024 https://dynamicworld.app/explore 82 Homework NASA ARSET: Change Detection for Land Cover Mapping Video tutorial: https://www.youtube.com/watch?v=-IOiwRJPtlY&ab_channel=NASAVideo Slides: https://appliedsciences.nasa.gov/sites/default/files/ChangeDetection_S2.pdf 83 End of Lecture 5 84