AML Reinforcement Learning & Decision Making 3

Question 1

What is the primary purpose of reward shaping in reinforcement learning?

Accepted Answer

To provide intermediate rewards that guide the agent more efficiently toward goals in sparse reward settings

Answer

To normalize all rewards to a fixed range for numerical stability

Answer

To penalize agents for taking excessive actions per episode

Answer

To apply heavier discounting in long-horizon environments

Question 2

What is Inverse Reinforcement Learning (IRL) primarily designed to accomplish?

Accepted Answer

Inferring an unknown reward function from observed expert demonstrations

Answer

Training agents through episodes played in reverse chronological order

Answer

Training agents to reverse or undo previously taken actions

Answer

Learning policies that minimize rather than maximize cumulative reward

Question 3

What is the credit assignment problem in reinforcement learning?

Accepted Answer

Identifying which past actions were responsible for a reward received much later in a sequence

Answer

Determining how to allocate compute between training and inference

Answer

Assigning numerical credit scores to competing reward function designs

Answer

Distributing shared rewards fairly among agents in cooperative settings

Question 4

In multi-agent reinforcement learning (MARL), why does non-stationarity pose a fundamental challenge?

Accepted Answer

Each agent's changing policy makes the environment appear non-stationary from any single agent's perspective

Answer

The physics of the simulated environment change over time during training

Answer

Reward functions become numerically unstable as more agents are added to the system

Answer

Neural networks cannot converge when multiple agents share the same architecture

Question 5

What is the key difference between Monte Carlo and Temporal Difference (TD) methods for value estimation?

Accepted Answer

Monte Carlo methods update values only after a complete episode; TD methods update after each step using bootstrapped estimates

Answer

Monte Carlo methods use neural networks; TD methods use tabular representations exclusively

Answer

Monte Carlo methods require an explicit environment model; TD methods do not

Answer

Monte Carlo methods apply only to continuous action spaces

Question 6

What is the function of eligibility traces in reinforcement learning?

Accepted Answer

Maintaining a decaying record of recently visited state-action pairs to propagate credit backward over multiple steps

Answer

Tracking which neurons in a neural network are most active during forward passes

Answer

Determining which experiences to prioritize in the replay buffer

Answer

Calculating the optimal epsilon decay schedule for exploration

Question 7

What is the primary advantage of hierarchical reinforcement learning (HRL) for complex tasks?

Accepted Answer

It decomposes complex long-horizon tasks into sub-goals, improving sample efficiency and enabling sub-behavior reuse

Answer

It enables parallel training across multiple processors with no communication overhead

Answer

It eliminates the need for reward engineering by automatically deriving intrinsic reward signals

Answer

It guarantees convergence to the global optimum in non-convex reward landscapes