CDP (Big Data Hadoop Administrator) 3 — Questions and Answers
Question 1: Which YARN scheduler allows resource pools to be shared among multiple groups while guaranteeing minimum capacity to each queue?
- FIFO Scheduler
- Capacity Scheduler (Correct answer)
- Fair Scheduler
- Priority Scheduler
Correct answer: Capacity Scheduler
The Capacity Scheduler partitions cluster resources into queues with guaranteed minimum capacities that can burst into shared capacity.
Question 2: A DataNode disk fails but the node remains online. Which parameter determines how many volume failures a DataNode tolerates before it shuts itself down?
- dfs.replication.min
- dfs.datanode.failed.volumes.tolerated (Correct answer)
- dfs.datanode.max.locked.memory
- dfs.namenode.heartbeat.recheck-interval
Correct answer: dfs.datanode.failed.volumes.tolerated
dfs.datanode.failed.volumes.tolerated sets how many data volumes can fail before the DataNode stops serving requests.
Question 3: What is the purpose of HDFS Federation?
- Replicating data across multiple data centers
- Allowing multiple NameNodes to manage separate namespace volumes on the same cluster (Correct answer)
- Federating authentication across multiple Kerberos realms
- Splitting a single large file across multiple clusters
Correct answer: Allowing multiple NameNodes to manage separate namespace volumes on the same cluster
HDFS Federation enables horizontal scaling of the namespace by having multiple independent NameNodes, each managing its own namespace volume.
Question 4: Which command would you use to check the health and block distribution of a specific HDFS path?
- hdfs dfs -stat /path
- hdfs fsck /path -files -blocks -locations (Correct answer)
- hdfs dfsadmin -report
- hdfs namenode -checkpointDir /path
Correct answer: hdfs fsck /path -files -blocks -locations
hdfs fsck with -files -blocks -locations shows detailed block placement and health information for the specified path.
Question 5: In MapReduce, what is a 'speculative execution' and why is it used?
- Pre-fetching input splits before tasks start
- Running duplicate copies of slow tasks to mitigate straggler effects (Correct answer)
- Caching intermediate results between map and reduce phases
- Predicting output size to pre-allocate HDFS space
Correct answer: Running duplicate copies of slow tasks to mitigate straggler effects
Speculative execution launches redundant copies of lagging tasks on other nodes so the faster copy's result is used, reducing overall job latency.
Question 6: Which Hadoop configuration file defines environment variables such as Java heap sizes for Hadoop daemons?
- core-site.xml
- hadoop-env.sh (Correct answer)
- yarn-site.xml
- hdfs-site.xml
Correct answer: hadoop-env.sh
hadoop-env.sh is a shell script sourced at daemon startup that sets JVM and environment settings like JAVA_HOME and heap sizes.
Question 7: What is the function of the Hadoop NameNode's edit log?
- Storing the checksum of each HDFS block
- Recording every metadata change to the file system for crash recovery (Correct answer)
- Logging all client read operations
- Tracking DataNode heartbeat history
Correct answer: Recording every metadata change to the file system for crash recovery
The edit log persistently records every file system metadata transaction so the NameNode can replay them to reconstruct state after a restart.
CDP (Big Data Hadoop Administrator) 3
Which YARN scheduler allows resource pools to be shared among multiple groups while guaranteeing minimum capacity to each queue?