CDP (Big Data Hadoop Administrator) 4 — Questions and Answers
Question 1: Which Hadoop HA component acts as a shared storage for edit logs between the Active and Standby NameNodes?
- ZooKeeper
- Journal Nodes (Correct answer)
- Secondary NameNode
- DataNodes
Correct answer: Journal Nodes
In HDFS HA, Journal Nodes form a quorum that both the Active and Standby NameNodes write to and read from for edit log synchronization.
Question 2: A cluster administrator needs to decommission a DataNode gracefully. What is the correct sequence of actions?
- Stop the DataNode process, then remove it from the cluster
- Add the DataNode to dfs.hosts.exclude, refresh the NameNode, wait for decommission to complete (Correct answer)
- Remove the DataNode from dfs.hosts, then restart the NameNode
- Run hdfs dfsadmin -refreshNodes and immediately shut down the node
Correct answer: Add the DataNode to dfs.hosts.exclude, refresh the NameNode, wait for decommission to complete
Graceful decommission requires adding the node to the exclude list, running refreshNodes, and waiting for HDFS to replicate all blocks elsewhere before stopping the node.
Question 3: What does the 'mapreduce.job.reduces' parameter control?
- The number of map tasks per job
- The number of reduce tasks for a MapReduce job (Correct answer)
- The maximum memory per reduce task
- The compression codec for reducer output
Correct answer: The number of reduce tasks for a MapReduce job
mapreduce.job.reduces specifies how many reducer tasks the framework will spawn for a given MapReduce job.
Question 4: Which Hadoop tool is used to balance block distribution across DataNodes?
- hdfs dfsadmin -report
- hdfs balancer (Correct answer)
- yarn resourcemanager -balance
- hdfs fsck -move
Correct answer: hdfs balancer
The hdfs balancer command redistributes HDFS blocks across DataNodes to equalize disk utilization.
Question 5: In YARN's Fair Scheduler, what is 'preemption' and when does it occur?
- Killing tasks when a node runs out of disk space
- Terminating containers from over-served queues to free resources for under-served queues (Correct answer)
- Pausing tasks when the ResourceManager restarts
- Evicting containers when memory exceeds the configured limit
Correct answer: Terminating containers from over-served queues to free resources for under-served queues
Preemption allows the Fair Scheduler to reclaim resources from queues exceeding their fair share and give them to queues that are below their share.
Question 6: Which configuration property sets the default HDFS block size for newly written files?
- dfs.replication
- dfs.blocksize (Correct answer)
- dfs.namenode.block-placement-policy
- dfs.datanode.block.size
Correct answer: dfs.blocksize
dfs.blocksize (or dfs.block.size in older versions) defines the default size of each HDFS block, typically 128 MB or 256 MB.
Question 7: What is the primary purpose of Apache Oozie in a Hadoop ecosystem?
- Monitoring cluster hardware metrics
- Scheduling and coordinating Hadoop workflow jobs (Correct answer)
- Providing SQL-like queries on HDFS data
- Managing HDFS quotas and permissions
Correct answer: Scheduling and coordinating Hadoop workflow jobs
Oozie is a workflow scheduler that orchestrates dependent Hadoop jobs (MapReduce, Hive, Pig, Spark) into directed acyclic graph workflows.
CDP (Big Data Hadoop Administrator) 4
Which Hadoop HA component acts as a shared storage for edit logs between the Active and Standby NameNodes?