Lecture 5: Learning for Robot Navigation PDF

Learning for Robot Navigation An Overview Dr. Alex Mitrevski Master of Autonomous Systems Winter semester 2023/24 Structure ▶ Why learning for robot navigation ▶ Navigation learning methods ▶ The case of autonomous driving...

Learning for Robot Navigation An Overview Dr. Alex Mitrevski Master of Autonomous Systems Winter semester 2023/24 Structure ▶ Why learning for robot navigation ▶ Navigation learning methods ▶ The case of autonomous driving Learning for Robot Navigation: An Overview 2 / 23 Why Learning for Robot Navigation Learning for Robot Navigation: An Overview 3 / 23 Typical Robot Navigation Workflow ▶ A typical robot navigation scenario starts by creating a map of the environment in which the robot needs to navigate Environment mapping ▶ In that map, the robot then has to: ▶ localise ▶ find path plans that bring it from its current location to a goal location Localisation ▶ find and apply low-level motion commands that will bring the robot to the goal Path Motion ▶ Under this framework, a robot can only navigate in a planning planning known environment with a (more or less) static environment (so that the map does not change) Learning for Robot Navigation: An Overview 4 / 23 Continuous Mapping Using SLAM ▶ A more generic workflow is one in which simultaneous Environment localisation and mapping (SLAM) is performed, which mapping allows the map to be updated (in principle) ▶ SLAM also makes it possible to navigate in unknown environment, as a robot can create a map on the fly Localisation while using it for navigation ▶ This eliminates the limitation of only being able to Path Motion navigate in a known environment. Problem solved? planning planning Learning for Robot Navigation: An Overview 5 / 23 Global and Local Planning ▶ As mentioned on the previous slides, the actual navigation act involves path and motion planning Learning for Robot Navigation: An Overview 6 / 23 Global and Local Planning ▶ As mentioned on the previous slides, the actual navigation act involves path and motion planning ▶ Path (aka global) planning is the problem of finding a viable (typically collision-free) path from the robot’s current location to its destination ▶ A path plan is created within a known map ▶ A path is usually decomposed into a sequence of waypoints through which the robot should pass Learning for Robot Navigation: An Overview 6 / 23 Global and Local Planning ▶ As mentioned on the previous slides, the actual navigation act involves path and motion planning ▶ Path (aka global) planning is the problem of finding a viable (typically collision-free) path from the robot’s current location to its destination ▶ A path plan is created within a known map ▶ A path is usually decomposed into a sequence of waypoints through which the robot should pass ▶ Motion (aka local) planning is concerned with finding appropriate robot motion commands ▶ Unlike global planners, local planners consider current sensor measurements ▶ Local planners typically ensure that the robot passes through the waypoints, potentially based on certain motion constraints ▶ Local planning requires a motion model of the robot Learning for Robot Navigation: An Overview 6 / 23 Challenges with the Traditional Navigation Approach (Too) Geometric navigation Typical navigation algorithms use only the geometry of the environment, but important semantic aspects and cues are ignored Learning for Robot Navigation: An Overview 7 / 23 Challenges with the Traditional Navigation Approach (Too) Geometric navigation Planar navigation Typical navigation algorithms use only the Many motion planners assume that a robot geometry of the environment, but important navigates on a plane — this does not always semantic aspects and cues are ignored hold, particularly in outdoor environments Learning for Robot Navigation: An Overview 7 / 23 Challenges with the Traditional Navigation Approach (Too) Geometric navigation Planar navigation Typical navigation algorithms use only the Many motion planners assume that a robot geometry of the environment, but important navigates on a plane — this does not always semantic aspects and cues are ignored hold, particularly in outdoor environments Hand-designed motion planning Robot motion planning algorithms are usually hand-tuned, and reusability between robot platforms is usually not guaranteed Learning for Robot Navigation: An Overview 7 / 23 Challenges with the Traditional Navigation Approach (Too) Geometric navigation Planar navigation Typical navigation algorithms use only the Many motion planners assume that a robot geometry of the environment, but important navigates on a plane — this does not always semantic aspects and cues are ignored hold, particularly in outdoor environments Hand-designed motion planning Inability to use prior experiences Robot motion planning algorithms are usually Every navigation trial is treated hand-tuned, and reusability between robot independently of prior trials, which makes it platforms is usually not guaranteed impossible to improve based on experience Learning for Robot Navigation: An Overview 7 / 23 Challenges with the Traditional Navigation Approach (Too) Geometric navigation Planar navigation Typical navigation algorithms use only the Many motion planners assume that a robot geometry of the environment, but important navigates on a plane — this does not always semantic aspects and cues are ignored hold, particularly in outdoor environments Hand-designed motion planning Inability to use prior experiences Robot motion planning algorithms are usually Every navigation trial is treated hand-tuned, and reusability between robot independently of prior trials, which makes it platforms is usually not guaranteed impossible to improve based on experience Challenges with new environments Most standard navigation frameworks are designed for known environments; adapting them for new environments can be challenging Learning for Robot Navigation: An Overview 7 / 23 Indoor vs. Outdoor Navigation Indoor navigation ▶ Structured environments overall, but with a large diversity in how this structure manifests itself ▶ Environments typically change dynamically ▶ Although that depends on the application domain (e.g. a package delivery center may be rather static) ▶ Often navigation among people necessary Learning for Robot Navigation: An Overview 8 / 23 Indoor vs. Outdoor Navigation Indoor navigation Outdoor navigation ▶ Structured environments overall, but with a ▶ A large variety of environments and terrains large diversity in how this structure (including extraterrestrial environments) manifests itself ▶ Environments usually more static ▶ Environments typically change dynamically ▶ But again, that depends on the application ▶ Although that depends on the application domain (e.g. roads are not very static) domain (e.g. a package delivery center may be rather static) ▶ In some domains, navigation needs to follow well-defined rules (e.g. in autonomous ▶ Often navigation among people necessary driving) Learning for Robot Navigation: An Overview 8 / 23 Navigation Learning Strategies Learning for Robot Navigation: An Overview 9 / 23 Formal Framework ▶ From a formal point of view, the overall problem of applying learning for navigation does not differ much from that of using learning for manipulation ▶ Thus, we can also observe navigation learning in terms of a task family of t tasks, which can be represented as a collection of Markov Decision Processes (MDPs): P (M) = {(Si , A, Ti , Ri , Ci , γ) | 1 ≤ i ≤ t} ▶ The overall state representation Si also the same: Si = Sr ∪ Se ▶ But navigation is not concerned with handling objects, so a factored object-centric representation would typically not be useful for Se Learning for Robot Navigation: An Overview 10 / 23 State Representations ▶ The state for learning-based navigation can be represented in a variety of ways ▶ Some of these are illustrated in the figure below GPS data IMU mesurements RGB cameras Raw sensor data Lidar points State representations Distances to obstacles Extracted features Detected objects / people Direction Learning for Robot Navigation: An Overview 11 / 23 What to Learn for Navigation? End-to-end navigation learning Learning for navigation Learning dedicated Parameter navigation components learning Learning for Robot Navigation: An Overview 12 / 23 Parameter Learning ▶ Learning can be used within the established navigation framework, where parameters that are used by components can be learned (e.g. parameters of local planners) ▶ Learning can be useful for a large variety of parameters, e.g. weights of cost functions used during planning, motion model parameters, parameters of predictive models, etc. ▶ This makes it possible to both preserve the properties of existing algorithms and improve those based on the experiences of a robot Learning for Robot Navigation: An Overview 13 / 23 Learning Dedicated Navigation Components ▶ Another alternative to incorporate learning into a robot navigation framework is to replace complete components by a learning-based version ▶ Different components can be replaced in this manner, for instance: ▶ a trained policy can be used instead of a local planner ▶ predictions produced by a learned model can be used for motion planning, e.g. as in BADRG (a diagram of the component learned there is shown below) G. Kahn, P. Abbeel and S. Levine, “BADGR: An Autonomous Self-Supervised Learning-Based Navigation System,” IEEE Robotics and Automation Letters, vol. 6, no. 2, pp. 1312–1319, 2021. Learning for Robot Navigation: An Overview 14 / 23 End-To-End Navigation Learning ▶ A more radical approach is to replace the complete navigation pipeline with learning-based components that are trained in an end-to-end fashion (i.e. where the learned model outputs motion commands directly based on sensory input) ▶ The attractive prospect of this is that the engineering effort required by the traditional navigation framework is, in principle, reduced ▶ Although whether that holds depends on how learning is exactly performed ▶ End-to-end learning leads to black-box models whose decision-making process can be difficult to analyse ▶ This is in contrast to components used in traditional navigation, which are usually based on mathematical models that are easier to understand Learning for Robot Navigation: An Overview 15 / 23 Action Spaces for Navigation Policies ▶ Depending on what is learned, navigation policies can have a variety of action spaces: Cartesian Cartesian velocity force Action space types Joint Controller Joint torques parameters velocities Learning for Robot Navigation: An Overview 16 / 23 Action Spaces for Navigation Policies ▶ Depending on what is learned, navigation policies can have a variety of action spaces: Cartesian Cartesian velocity force Action space types Joint Controller Joint torques parameters velocities ▶ The figure looks familiar? It is exactly the same as the one we had in the case of robot manipulation (last lecture)! Learning for Robot Navigation: An Overview 16 / 23 The Case of Autonomous Driving Learning for Robot Navigation: An Overview 17 / 23 Learning in Autonomous Driving ▶ Autonomous driving is a very challenging problem due to the fact that roads are very dynamic and involve many other traffic participants ▶ Pure model-based methods can quickly reach their limit here because there are simply too many relevant aspects to consider ▶ Autonomous driving is an important area where the benefits of learning have been directly visible ▶ Particularly advances in computer vision have made it possible to process complex input and, as a result, produce more complex driving behaviours Learning for Robot Navigation: An Overview 18 / 23 Driving Tasks Where Learning Has Been Applied Lane keeping Lane changing Common Highway merging Overtaking driving tasks Intersection handling Path planning Learning for Robot Navigation: An Overview 19 / 23 Autonomous Driving Framework ▶ From a general point of view, the navigation framework for an autonomous vehicle is the same as that for any other mobile robot (see the figure below) ▶ But the necessity to follow traffic rules is a major difference with other navigation domains ▶ An autonomous vehicle navigates in an outdoor environment, so it can benefit from additional signals (e.g. GPS) that cannot be used in indoor environments Learning for Robot Navigation: An Overview 20 / 23 Autonomous Driving Based on Deep Learning ▶ Due to its ability to process high-dimensional data, deep learning has been applied to a variety of autonomous driving problems ▶ The most common approach is to use deep learning for specific aspects of the driving pipeline, for instance: ▶ detecting people and other vehicles (scene understanding) ▶ controlling particular driving behaviours, such as lane keeping, using a learned policy (motion planning) S. Grigorescu et al., “A survey of deep learning techniques for autonomous driving,” Journal of Field Robotics, vol. 37, no. 3, pp. 362–386, Apr. 2020. ▶ Attempts to use end-to-end deep learning also exist ▶ Deep neural networks need dedicated hardware (e.g. graphic cards) to run efficiently ▶ Autonomous driving requires real-time control, so applying deep learning on autonomous vehicles requires implementations that support efficient inference Learning for Robot Navigation: An Overview 21 / 23 Explainability ▶ When using (deep) learning in robot navigation (but particularly in autonomous driving), it is not always easy to understand why a decision has been made ▶ The property of being able to understand an autonomous decision-making process is referred to as explainability ▶ Particularly relevant in case of failures that lead to accidents ▶ Certifying autonomous driving behaviours is generally difficult if explainability cannot be guaranteed ▶ Various ideas for how explainability should be D. Omeiza et al., “Explanations in Autonomous Driving: A Survey,” achieved exist in the literature (one example is shown IEEE Trans. Intelligent Transportation Systems, vol. 23, no. 8, pp. 10142–10162, Aug. 2022. on the right) ▶ We will discuss techniques for explainability later in the course Learning for Robot Navigation: An Overview 22 / 23 Next Lecture: Learning from Demonstration Learning for Robot Navigation: An Overview 23 / 23