Machine Learning Reinforcement Learning 1 — Questions and Answers
Question 1: What is the primary goal of a reinforcement learning agent?
- To label training data accurately
- To maximize cumulative reward over time (Correct answer)
- To minimize classification error on a test set
- To cluster similar data points together
Correct answer: To maximize cumulative reward over time
An RL agent learns by interacting with an environment to maximize the total cumulative reward it receives over time.
Question 2: What does the reward signal represent in reinforcement learning?
- The accuracy of predictions made by the model
- The gradient of the loss function
- Immediate scalar feedback indicating how good an action was (Correct answer)
- The probability of selecting a given action
Correct answer: Immediate scalar feedback indicating how good an action was
The reward signal provides immediate scalar feedback to the agent, indicating the desirability of the action taken in the current state.
Question 3: In reinforcement learning, what is an 'episode'?
- A single mini-batch of training data
- A complete sequence from the initial state to a terminal state (Correct answer)
- One gradient descent update step
- A group of similar states in the environment
Correct answer: A complete sequence from the initial state to a terminal state
An episode is a complete sequence of interactions from the starting state to a terminal or goal state in the environment.
Question 4: How does reinforcement learning fundamentally differ from supervised learning?
- RL uses labeled data while supervised learning uses rewards
- RL learns from environment interaction and reward signals, not labeled examples (Correct answer)
- RL cannot be applied to classification tasks
- RL always requires significantly more data than supervised learning
Correct answer: RL learns from environment interaction and reward signals, not labeled examples
Unlike supervised learning which uses labeled input-output pairs, RL agents learn from interactions with an environment using reward signals as feedback.
Question 5: What is a policy in reinforcement learning?
- A mapping from states to actions (or action probabilities) that defines agent behavior (Correct answer)
- The reward function provided by the environment
- The transition probabilities between environment states
- The maximum total reward achievable in an episode
Correct answer: A mapping from states to actions (or action probabilities) that defines agent behavior
A policy defines the agent's behavior by mapping states to actions or probabilities of actions, determining what the agent does in each situation.
Question 6: What does the state-value function V(s) represent in RL?
- The immediate reward received for entering state s
- The number of times state s has been visited during training
- The expected cumulative discounted reward starting from state s under the current policy (Correct answer)
- The probability of transitioning out of state s
Correct answer: The expected cumulative discounted reward starting from state s under the current policy
The value function V(s) estimates the expected total discounted reward an agent can accumulate starting from state s and following its current policy.
Question 7: What is the exploration-exploitation tradeoff in reinforcement learning?
- Balancing training speed against final model accuracy
- Choosing between trying new actions to discover better strategies versus using known good actions (Correct answer)
- Deciding between model-based and model-free RL approaches
- Trading off reward magnitude versus frequency of reward
Correct answer: Choosing between trying new actions to discover better strategies versus using known good actions
The exploration-exploitation tradeoff involves balancing the need to explore new actions to discover better strategies versus exploiting known good actions to maximize reward.
Machine Learning Reinforcement Learning 1
What is the primary goal of a reinforcement learning agent?