OBIA Knowledge Representation (PDF)

Object-based Image Analysis S LANG / D TIEDE Tutor: M DANESHFAR 4 – Knowledge representation [summary] What we will learn in this chapter … To work with intelligent image analysis, knowledge representation is one of the prerequisites. There are different ways to reach a higher intelligent level...

Object-based Image Analysis S LANG / D TIEDE Tutor: M DANESHFAR 4 – Knowledge representation [summary] What we will learn in this chapter … To work with intelligent image analysis, knowledge representation is one of the prerequisites. There are different ways to reach a higher intelligent level, e.g. Rule-based Systems, which foster…  Transparency  Transferability Content: 1. Why knowledge representation? 2. Experience and learning 3. Rule-based production systems 4. Fuzzy rule sets 5. Image understanding and OBIA 6. Object categories ---------------------------------------------------------------------- Human being has a concept of how our geographical environment is organised, which is…  Intuitive  Experience-based  culture-driven  Instant To achieve such level in image analysis we need to make implicit knowledge explicit! Campbell 2001 Knowledge plays a key role in the interpretation-oriented parts of the remote sensing processing chain. Factors below evoke certain parts of our experience and knowledge: 1   Colour  Contrast  Context-related information (form and arrangement) Note:  Object hypotheses are constantly tested and verified against what we see  Visual impression and knowledge and experience work closely together  Implicit knowledge, by training (formalized learning)  explicit knowledge In Artificial Intelligence (AI), we deal with:  Procedural knowledge o Specific computational functions o Can be represented by set of rules  Structural knowledge o Domain-related o Implies how concepts of a domain are interrelated o Characterized by high semantic contents, organized in Knowledge Organizing systems (KOS)  KOS:  Realised by graphic notations such as semantic networks + mathematical theories like formal concept analysis (FCA)  Semantic net: o needs to be created manually o allow for controlling each and every existing connection once being established o By increasing complexity, the transparency and operability will reach a limit o e.g. Bayesian networks are manually built, but the weighting of connections can be trained, for every connection In OBIA, we need both procedural and structural knowledge…  Structural knowledge for establishing links between image objects and real world and geographical features o In image analysis, semantic nets and frames offer a formal framework for semantic knowledge representation using an inheritance concept (is part of, is instance of…)  Procedural knowledge for preparing the rule-bases Artificial Neural Network (ANN):  Designed as neuron-like machines  Optimize themselves through adaptive vector coding  In a perfect ANN, the system of weights and synapses is tuned to correctly classify images 2   Even optimized neural net classifiers remain black box system  cannot be restructured any more  information have the disadvantage of low transparency (hidden layers)  This is the problem of ANN Case-based reasoning (CBR):  Reuses learnt cases of previously experienced concrete problem situations (exemplars) stored in a case- base  An ongoing learning process, the system permanently o reuses stored knowledge in an instrumental mode o improves its knowledge by retaining and learning Natural Computing (Binning at al. 2002)  To stress the parallel nature of natural thinking and computing  To oppose it against the incoherency of AI and natural thinking Binning et al. believe…  Expert systems do not sufficiently focus on the net-like character of our thinking  Knowledge is usually represented in a too static manner They propose… Self-organizing, Semantic, and Self-similar network (Triple S)  Represents scene-related procedural knowledge in a hierarchical form  The classification process itself is controlled by excitements and thresholds  The state of activation depends on the activation state of former connections and so on  The entire construct makes up a ‘fractal machine’, where sub-networks are self-similar to super-networks  To connect properties and procedures, an inheritance concept is used  Activation depends on neighbourhood like in a cellular automata machine  Classifications and segmentation in an alternating manner, in a cyclic process (by an optimally hierarchically structured network) Binnig et al. 2002 The creation of objects and their relations on and across different hierarchical levels is equivalent to transform information into knowledge. Rule-based production system  Logical interface (if…then)  Knowledge needs to be encoded in rules  a great advantage in the communication with experienced manual interpretations and users of the information product  Transparency of intelligent system  The transfer of experienced knowledge into a consistent set of rules is neither a trivial nor an unambiguous task, challenges… o How to encode implicit knowledge into codes o We need control mechanisms to control the integrity of the ruleset… As Winston, 1984 mentioned: a set of conventions about how to describe a class of things 3   There are two dimensions o Syntactical=procedural: provides a set of symbols that can be used or combined o Semantic: specifies what meaning those symbols and combinations have  Within such production systems, rules are incorporated in a knowledge engineer to classify image objects through logical inference Different approaches in terms of level of controlling: Level of Control  Rule-based production system  Case-based reasoning (CBR)  Artificial Neural Network (ANN) Expert knowledge is intuitive and not directly transferable into crisp decision – so we should use… Fuzzification:  Allows statements about real-world phenomena with some degree of uncertainty  Converts a binary membership into a graduated degree of certainty  Every object obtains a degree of membership rather than classifying in a strict and binary manner Whenever our conceptual understanding of a real world implies vagueness, we use fuzzy terms  Concepts like ‘heat‘ (e.g. for tea), ‘Height‘ (e.g. for a person) and ‘greenness‘ (e.g. for meadow) may slightly differ from person to person due to an induvial sense of them  We simplify a gradual phenomenon and categorize it  The boundaries between these categories are not crisp, and there is no specific threshold  Objects can be assigned to multiple classes  With increasing distance from these boundaries, the assignment of a certain category is indisputable (the probability of a class assignment is higher) In image analysis there are also vagueness in class assignment, e.g. in vegetation (NDVI) Considering NDVI:  Positive NDVI: vegetated area  Negative NDVI: non-vegetated area It is more appropriate, when we take into account a certain gradual limit between these two binary classes! Jensen, 2005: Fuzzy set theory is better suited for dealing with real‐world problems than traditional logic because most human reasoning is imprecise. 4 Within image analysis we are facing the following challenges that may cause vagueness in the interpretation and the assignment of classes (Tsatsoulis, 1993): Uncertainty in sensor measurements Parameter variations due to limited sensor calibration Class mixtures due to resolution Vague (linguistic) class descriptions Within OBIA, we work with objects instead of single pixels. Therefore we also work on aggregated values (e.g. the mean spectral value within an object)  for fuzzy sets that means we have usually a more stable assignment Fuzzy rule sets Transition from a crisp to a fuzzy system (To decide if a feature value belongs to a fuzzy set) Define membership function μ(x) Assigning to every object feature value x a membership value μ If μ > 0, then x belongs to the fuzzy set A Relation between object feature and classification Choice and parameterisation of the membership function influence the quality of the classification Introducing expert knowledge Certain features can have multiple class assignments Fuzzy rule-base Fuzzy rule “if – then” for assigning an object to a class o If feature value x (of the object) is member of the fuzzy set (e.g. associated with the class forest), the image object is a member of the land-cover forest Combination of fuzzy sets to create advanced fuzzy rules o Operator “AND” – Minimum operation 5  o Operator “OR” – Maximum operation o Operator “NOT” – inversion of a fuzzy value: returns 1 – fuzzy value Fuzzy rule-base (combination of the fuzzy rules of all classes) delivers a fuzzy classification o Every object has a tuple of return values assigned to it with the degrees of membership to each class/degrees of class assignment o Since these values are possibilities to belong to a class, they don’t have to add up to 1 (unlike probabilities) μforest = μx AND μy = Min(μx, μy) = Min(0.7, 0.9) = 0.7 μforest = 0.7 μpasture = 0.4 μwater = 0.05 … Comparison of membership degrees  Reliability of class assignment o The higher the degree of the most possible class, the more reliable is the assignment  Stability of classification o Stable classification for differences between highest membership value and other values  Equal membership degrees o high values – reliability for both classes: classes cannot be distinguished with the provided classification o Low values – unreliable classification (use threshold of a least required membership degree to ensure quality of classification) 6 Defuzzification  Ultimately, the fuzzy results have to be translated back into crisp values  Maximum membership degree of fuzzy classification used as crisp classification  We can control how this process works (e.g. we could decide that no class assignment should be done, if membership values are too small) Image Understanding (IU) Ibrahim, 2000 Major advances have been made in AI including feature detection algorithms. Yet image understanding is more than just feature extraction on a specified scene. Axel Pinz, 1994 Image understanding (IU), is commonly regarded as a process, by which we arrive at a complete description of the image content, i.e. the reconstruction of an imaged scene.  Extent of IU: o Reaching from signals (image data) to a symbolic representation of the scene content  Conditions for IU: Outcome depends on the domain of interest of the interpreter, defined by: o Underlying research questions o Specified field of application o Pre-existing knowledge and experience of the interpreter  Output description: Description of real-world objects and their relationships in the specific scene o Resulting in thoroughly described features (not mere listing and labelling of features) 7  o Within image interpretation of EO data, target class is less defined and rather ambiguous (especially for natural features) o Knowledge input: process is driven by  utilisation of procedural knowledge  transformation of structural knowledge o Involved disciplines:  Image processing  Pattern recognition  Artificial intelligence  Gaining insight into the content of a scene requires familiarity with:  Potential content  Personal acquaintance with the imaged area  General experience The field of IU is interlinked with:  image processing  provides the sources in a pre-processed way  pattern recognition  incorporate methods for knowledge representation and expert systems  Artificial intelligence (AI)  covers major field of computer-based image understanding Ibrahim, 2000 AI covers a major field of computer-based image understanding. Yet, a certain portion is left uncovered which is related to unsolved challenges of knowledge transfer to an automated system. Image understanding (IU) and OBIA Gorte, 1998 By forming the conceptual link to human perception image segmentation is considered an essential prerequisite for image understanding. Burnett and Blaschke, 2003 OBIA offers valuable methodological assets in breaking down scene complexity into meaningful image primitives. By providing “candidate discretizations of space”. 8 Note:  A profound prerequisite of image object modelling is the provision of a clear underlying concept regarding the domain of interest (domain of interest of a skilled interpreter may differ from that of a simple user: the former look specifically for certain feature, while the latter may be more interested in general information or impression), which means understanding…  Target scale  Target object set  Target class scheme Utilization and transformation of knowledge: 1. Knowing what we are looking for (target objects, scale and classes, dependant on the domain of interest – multi-scale representation ) 2. Class modelling (categorizing image objects, characteristics and relationships) 3. Scene descriptions as conceptual reality 1. Knowing what we are looking for: FROM real-world scene TO an image of a high complex content  Provision of scaled representations by aggregation information and reducing complexity  Multi-scale segmentation with its sub-level and super-level aggregates  Critical choice of an appropriate segmentation level to make up the 1 st match of the scene view with conceptual reality  Defining domain of interest with target scale, objects and classes 2. Class modelling:  encoding expert knowledge into a rule system  Categorizing the image objects by their spectral and spatial properties and their mutual relationships (semantic features)  This is the 2nd match and the shift to an object-centred view is accomplished 3. Scene description as conceptual reality:  In the final stage of IU, a full set of categorized target objects is achieved which should meet the conceptual reality of the interpreter or user  The entire process is characterized by the utilization and transformation of knowledge  A body plan is established for the relevant features through class descriptions, modelling, rules and labelling  The procedure makes the expert knowledge explicit  Knowledge is progressive and step-wise adopted  Experience grows, as knowledge will be enriched by analyzing unknown scenes  The transfer of knowledge may incorporate or simulate new rules 9   This provides the 3rd match: semantic system  conceptual reality Object categories  Bona fide objects o Natural boundaries o Correspond to local physical discontinuities o Perceived by different users more or less the same o E.g. single patches of forests, settlements, grassland  Composite objects o Real-world modelled objects o Correspond to functional homogeneity o Perceived by experts in similar way (convention) o Expert-based delineation o e.g. crop/grassland mixed arable land  (Conceptual) Fiat objects o Used more in the domain of conditioned, policy-oriented information with specific user group or application o With unnatural boundaries o Correspond to no genuine heterogeneity o Objects not obvious in the landscape Bona-fide objects Modelled composite objects Concept-related fiat objects Example(s) Land cover classes, e.g. Biotope complexes, e.g. mixed Units with uniform behaviour in meadow, field, forest arable land and grassland area vulnerability Main Scale-specific delineation, Meet requirements in terms of Derive nominal unit labels from methodological boundary generalisation, appropriateness and matching regionalised groups challenge key Real world Existing and recordable in field Perceived by expert vision, Reference to be constructed, reference agreement with mapping key units not nominal, but of like and existing planning units behaviour (here: cadastral data) Label Binary or fuzzified In stages, appropriateness as a A priori nomenclature not verification term to be operationalized existing 10 Boundary Scale-depending, low degree of Expert-based, high degree of Concept-related, boundaries validation freedom freedom hardly found in reality Policy-driven On an elementary level: yes Yes, high expectations in EO- Yes, strongly, but in early stage based techniques of conceptualisation Delineation Automated Automated, partly manual Fully expert-based, based on regrouping Delphi exercise, but automated 11