Artificial Intelligence Reinforcement Learning 2

Question 1

What is Q-learning in reinforcement learning?

Accepted Answer

A model-free algorithm that learns action-value (Q) functions to determine the best action in each state

Answer

Q-learning learns Q(s,a) — the expected cumulative discounted reward of taking action a in state s — and derives the optimal greedy policy from these values.

Question 2

What does the discount factor (γ) control in reinforcement learning?

Accepted Answer

The importance of future rewards relative to immediate rewards; values near 1 make the agent far-sighted

Answer

A discount factor γ ∈ [0,1) exponentially reduces the weight of future rewards; γ near 0 makes the agent myopic (focus on immediate reward), while γ near 1 makes it long-sighted.

Question 3

What is the Bellman equation used for in reinforcement learning?

Accepted Answer

Expressing the value of a state as the immediate reward plus the discounted value of the next state, forming the basis for dynamic programming

Answer

The Bellman equation recursively decomposes the value function: V(s) = max_a [R(s,a) + γ Σ P(s'|s,a) V(s')], enabling iterative computation of optimal values.

Question 4

What is Deep Q-Network (DQN) and what was its breakthrough application?

Accepted Answer

A deep RL algorithm that uses a neural network to approximate Q-values, first achieving human-level performance on Atari games

Answer

DQN (DeepMind, 2015) uses a deep CNN to approximate Q(s,a) directly from raw pixel inputs, using experience replay and target networks to stabilize training on Atari 2600 games.

Question 5

What is 'experience replay' in DQN and why is it important?

Accepted Answer

Storing past transitions in a buffer and sampling random mini-batches to break temporal correlations and stabilize training

Answer

Experience replay stores (s, a, r, s') transitions in a replay buffer and samples random mini-batches, decorrelating consecutive training samples and improving data efficiency.

Artificial Intelligence Practice Test

Artificial Intelligence Practice Test

Artificial Intelligence Reinforcement Learning 2