Machine Learning Reinforcement Learning 3

Question 1

What is the key innovation of Deep Q-Networks (DQN) over classic Q-learning?

Accepted Answer

Using a deep neural network to approximate Q-values for high-dimensional state spaces

Answer

DQN uses a deep neural network to approximate Q-values, enabling RL to scale to high-dimensional state spaces such as raw pixel inputs from Atari games.

Question 2

What do policy gradient methods optimize?

Accepted Answer

Policy parameters by computing gradients of the expected cumulative reward

Answer

Policy gradient methods directly optimize the policy parameters by computing gradients of expected cumulative reward with respect to those parameters.

Question 3

What is the primary purpose of experience replay in DQN?

Accepted Answer

To store past transitions and sample them randomly to break temporal correlations and stabilize training

Answer

Experience replay stores past transitions in a replay buffer and randomly samples mini-batches for training, breaking temporal correlations and improving data efficiency.

Question 4

What does the Actor-Critic method combine in reinforcement learning?

Accepted Answer

A policy network (actor) that selects actions combined with a value network (critic) that evaluates them

Answer

Actor-Critic methods combine an actor (policy network) that selects actions with a critic (value network) that evaluates those actions, reducing variance compared to pure policy gradient methods.

Question 5

What is the epsilon-greedy strategy used for in reinforcement learning?

Accepted Answer

To balance exploration and exploitation by taking random actions with probability epsilon

Answer

The epsilon-greedy strategy selects a random action with probability epsilon (exploration) and the greedy best-known action with probability 1-epsilon (exploitation), balancing discovery and performance.

Machine Learning Practice Test

Machine Learning Practice Test

Machine Learning Reinforcement Learning 3