Reinforcement Learning
Watch agents learn by trial and error. Balance exploration and exploitation, compute values with dynamic programming, and train a Q-learning agent in a grid world.
Key concepts
Agent and Environment
The agent acts, the environment responds with a new state and a reward
Rewards and Returns
The goal is to maximize total discounted reward, not just the next one
Value Functions
How good it is to be in a state, or to take an action in a state
Exploration vs Exploitation
Try new actions to learn more, or use what already works
Visualizations
Multi-Armed Bandit
BeginnerCompare ε-greedy, UCB and Thompson sampling on slot machines with hidden payouts
Explorationε-GreedyUCBRegret
Open Value Iteration
IntermediateWatch state values and the optimal policy emerge in a grid world with the Bellman equation
Bellman EquationMDPDiscount FactorPolicy
Open Q-Learning
IntermediateTrain an agent to reach a goal by trial and error and watch its Q-table fill in
Q-TableTemporal DifferenceLearning RateEpisodes
Open