AML Reinforcement Learning & Decision Making 3

Question 1

What is the primary purpose of reward shaping in reinforcement learning?

Accepted Answer

To provide intermediate rewards that guide the agent more efficiently toward goals in sparse reward settings

Answer

Reward shaping adds supplementary intermediate rewards to the sparse environmental signal, providing denser feedback that helps agents learn faster without waiting for rare terminal rewards.

Question 2

What is Inverse Reinforcement Learning (IRL) primarily designed to accomplish?

Accepted Answer

Inferring an unknown reward function from observed expert demonstrations

Answer

IRL infers the underlying reward function from expert behavioral demonstrations, which is useful when the reward function is unknown, hard to define, or too complex to specify manually.

Question 3

What is the credit assignment problem in reinforcement learning?

Accepted Answer

Identifying which past actions were responsible for a reward received much later in a sequence

Answer

The credit assignment problem is the difficulty of determining which earlier actions in a long trajectory caused a delayed reward, requiring techniques like eligibility traces or TD(λ) to propagate credit backward.

Question 4

In multi-agent reinforcement learning (MARL), why does non-stationarity pose a fundamental challenge?

Accepted Answer

Each agent's changing policy makes the environment appear non-stationary from any single agent's perspective

Answer

As each agent continuously updates its policy, the effective environment experienced by all other agents changes, violating the stationarity assumption that standard single-agent RL convergence proofs rely on.

Question 5

What is the key difference between Monte Carlo and Temporal Difference (TD) methods for value estimation?

Accepted Answer

Monte Carlo methods update values only after a complete episode; TD methods update after each step using bootstrapped estimates

Answer

Monte Carlo methods compute actual returns from complete episode trajectories (unbiased but high variance), while TD methods bootstrap from current value estimates at each step (lower variance but biased).

(AML) Advanced Machine Learning Certification Practice Test

(AML) Advanced Machine Learning Certification Practice Test

AML Reinforcement Learning & Decision Making 3