Data Processing Big Data Processing 3

Question 1

What is the role of a NameNode in Hadoop HDFS?

Accepted Answer

It manages the file system namespace and tracks where data blocks are stored across DataNodes

Answer

The NameNode is the HDFS master server that maintains the file system namespace, directory tree, and metadata about where each file's blocks are stored on DataNodes.

Question 2

What does the term 'data skew' mean in distributed processing?

Accepted Answer

An imbalanced distribution of data across processing nodes, causing some nodes to work much harder than others

Answer

Data skew occurs when data is unevenly distributed across partitions or nodes, causing certain nodes to process far more data and creating bottlenecks.

Question 3

What is a distributed cache in big data frameworks?

Accepted Answer

A mechanism to make read-only files available locally on all worker nodes during a job

Answer

A distributed cache distributes read-only files (like lookup tables or configuration files) to all worker nodes at the start of a job, reducing repeated network transfers.

Question 4

What is Lambda architecture in big data systems?

Accepted Answer

A design that combines batch processing for accuracy with stream processing for low latency

Answer

Lambda architecture uses a batch layer for comprehensive historical processing and a speed layer for real-time stream processing, with a serving layer merging both views.

Question 5

What does fault tolerance mean in a distributed big data system?

Accepted Answer

The system continues operating correctly even when individual nodes fail

Answer

Fault tolerance means a distributed system can detect and recover from failures of individual components without losing data or stopping overall processing.

Data Processing Practice Test

Data Processing Practice Test

Data Processing Big Data Processing 3