DSE Big Data Technologies 3

Question 1

What is the role of Apache ZooKeeper in a big data cluster?

Accepted Answer

Providing distributed coordination and configuration management services

Answer

ZooKeeper provides distributed coordination services such as leader election, configuration management, and distributed locking for cluster components.

Question 2

In Apache Flink, what distinguishes it from Apache Spark Streaming?

Accepted Answer

Flink is a true streaming engine where batch is a special case, whereas Spark Streaming uses micro-batches

Answer

Flink treats streaming as the first-class primitive and handles batch as a finite stream, while Spark Streaming originally processed streams as a series of small batches.

Question 3

What does 'replication factor' mean in HDFS?

Accepted Answer

The number of copies of each data block stored across DataNodes

Answer

The replication factor (default 3) defines how many copies of each HDFS block are written to different DataNodes for fault tolerance.

Question 4

Which pattern best describes the Lambda Architecture for big data systems?

Accepted Answer

Combining batch and speed layers to serve both historical and real-time queries

Answer

Lambda Architecture uses a batch layer for comprehensive historical processing and a speed layer for real-time updates, with a serving layer merging both views.

Question 5

What is Apache Sqoop primarily used for?

Accepted Answer

Bulk transfer of data between relational databases and Hadoop

Answer

Sqoop is designed to efficiently transfer bulk data between relational databases (via JDBC) and Hadoop storage systems like HDFS or HBase.

DSE - Data Science Practice Test

DSE - Data Science Practice Test

DSE Big Data Technologies 3