Autonomous Driving and Reinforcement Learning

A software entity that learns and makes decisions within a specific environment.

The surrounding world where the agent operates and interacts.

Specific choices or actions the agent can take within the environment.

The current circumstances or state of the environment that the agent perceives.

A numerical value given by the environment for each action taken by the agent.

A strategy that maps situations to actions, guiding the agent's behavior.

A machine learning technique where an agent learns by interacting with an environment and receiving rewards for its actions.

A learning process where an agent aims to maximize cumulative rewards over time by interacting with its environment.

The value function represents the expected sum of future rewards an agent can achieve by starting in a particular state and following a given policy.

A model in Reinforcement Learning represents the environment's dynamics, including how actions affect state transitions and reward distributions.

The workflow in reinforcement learning involves defining the environment, rewards, creating the agent, training and validating the agent, and finally deploying the learned policy.

Supervised learning involves training a model on labeled data, where the desired output is provided for each input. Reinforcement learning, on the other hand, learns by interacting with the environment and receiving rewards or penalties.

Reinforcement learning relies on feedback from the environment in the form of rewards or penalties.

The agent's decisions in Reinforcement Learning are made in a sequential manner, where the current action influences future states and rewards.

Time is crucial in reinforcement learning, as the agent needs to learn how its actions affect future states and the delay between an action and its reward.

The agent's exploration and exploitation trade-off involves balancing the need to try new actions to learn better strategies (exploration) with the need to exploit what it already knows to maximize immediate rewards.

The real-world complexity makes it hard to create an accurate model for an agent to learn from, leading to potential errors in decision-making.

Learning directly from interacting with the environment, getting feedback in the form of rewards or punishments.

The agent may require lots of experience to learn a good policy, as it learns solely from interactions.

The agent directly learns the best action to take in each situation or the value of each state/action.

This approach is more resilient to the complexities of real-world environments because it doesn't rely on a potentially inaccurate model.

A framework for modeling sequential decision-making where actions impact both immediate rewards and future states.

Expressing your problem as an MDP is the first step toward solving it using dynamic programming or other reinforcement learning techniques.

A mathematical property describing a system where the future state depends only on the present state, not the past history.

Model-Based Approaches can be more efficient for learning and planning, but they are susceptible to model bias and complexity. Model-Free Approaches are simpler and potentially more robust but require more experience.

A matrix representing the probabilities of transitioning between different states in a Markov process.

A model of a system that evolves over time based on probabilities and the Markov Property.

A Markov Process where each state transition has a reward associated with it.

The process of calculating the expected cumulative reward of a strategy over an infinite horizon.

A technique to emphasize immediate rewards over future rewards.

A learning algorithm that aims to find the best policy for an agent by updating the values of actions in different states.

In this reinforcement learning method, the goal is to maximize a value function representing the long-term return of states. This involves learning the optimal policy to achieve this maximizing value.

In this reinforcement learning method, the main goal is to come up with a strategy that helps the agent gain maximum rewards in the future through the actions taken in each state. The agent learns to make optimal decisions to achieve the goal.

This type of policy-based reinforcement learning is used when the actions selected by the agent are always the same for the same state. In other words, once the agent chooses an action, it will always take the same action in that state. This is a deterministic approach based on known state values.

A model-based approach to reinforcement learning where an agent learns by creating a virtual model of its environment. This model helps the agent predict the consequences of its actions and make more informed decisions. It uses the model to simulate various scenarios, like the movement of other objects or the effects of taking different actions.

This type of policy-based reinforcement learning involves actions that are not always the same for the same state. There is some randomness in the responses of the agent to the same state. The agent's actions can be influenced by chance, leading to variability in its behavior.

This reinforcement learning algorithm learns the optimal policy independently of the policy used to gather data for learning. It evaluates the value of the optimal policy regardless of the actions taken during the training process

This reinforcement learning algorithm learns the value of the policy that is being executed by the agent. It learns the value of the policy based on the actions taken by the agent itself.

This reinforcement learning technique involves estimating the transition and reward functions by taking actions in the environment. The agent learns about the environment by interacting with it and then uses this information to predict the outcomes of future actions.

A state in an MDP represents the current circumstance or situation of the environment. It's what the agent observes and uses to decide what action to take.

In an MDP, actions are the choices or options available to the agent. They are the actions the agent can take within a given state.

The reward in an MDP is a numerical value given by the environment for each action taken by the agent. It reflects the goodness or badness of the action.

A policy in an MDP is a strategy that maps states to actions, guiding the agent's behavior. It tells the agent what action to take in each state.

The goal of solving an MDP is to find an optimal policy, which is the best strategy for the agent to follow in order to maximize the total rewards over time.

In the salmon fishing example, the states represent the different levels of salmon population in the area: empty, low, medium, and high. The agent needs to decide what action to take based on the current state.

In the salmon fishing example, the actions are either 'fish' or 'not_to_fish'. These actions have different consequences based on the current state and impact the future state of the salmon population.

Rewards in the salmon fishing example are based on the action taken and the resulting state. Fishing in certain states generates rewards, while taking actions that lead to an empty state result in low rewards due to the cost of rebuilding the population.