Machine Learning Reinforcement Learning 3

Question 1

What is the key innovation of Deep Q-Networks (DQN) over classic Q-learning?

Accepted Answer

Using a deep neural network to approximate Q-values for high-dimensional state spaces

Answer

Using recurrent layers to handle partially observable environments

Answer

Combining supervised and reinforcement learning loss objectives

Answer

Using multiple parallel agents to speed up experience collection

Question 2

What do policy gradient methods optimize?

Accepted Answer

Policy parameters by computing gradients of the expected cumulative reward

Answer

The gradient of the action-value Q function directly

Answer

The value function using standard gradient descent

Answer

State transition probabilities to maximize reward density

Question 3

What is the primary purpose of experience replay in DQN?

Accepted Answer

To store past transitions and sample them randomly to break temporal correlations and stabilize training

Answer

To replay successful training episodes for human review and debugging

Answer

To repeat the most rewarding episodes multiple times to reinforce learned behavior

Answer

To share collected experience between multiple independent RL agents

Question 4

What does the Actor-Critic method combine in reinforcement learning?

Accepted Answer

A policy network (actor) that selects actions combined with a value network (critic) that evaluates them

Answer

A two-player adversarial game framework for competitive training

Answer

One network that acts and a separate network that detects prediction errors

Answer

Multiple agents that simultaneously compete and cooperate to learn

Question 5

What is the epsilon-greedy strategy used for in reinforcement learning?

Accepted Answer

To balance exploration and exploitation by taking random actions with probability epsilon

Answer

To normalize reward values between epsilon and 1 for stable training

Answer

To determine an optimal adaptive learning rate for policy updates

Answer

To set the convergence threshold for stopping training

Question 6

How does the Monte Carlo method estimate value functions in reinforcement learning?

Accepted Answer

By averaging actual returns from complete episodes after visiting each state

Answer

By randomly searching the action space for optimal policies

Answer

By simulating physics-based environment dynamics stochastically

Answer

By applying stochastic gradient descent directly to sampled reward signals

Question 7

What is the 'credit assignment problem' in reinforcement learning?

Accepted Answer

The challenge of determining which past actions were responsible for a delayed reward

Answer

Determining how to split the reward budget among multiple agents

Answer

Assigning credit scores to different environment states based on value

Answer

Deciding how to distribute computational resources across training workers