DSE Big Data Technologies 5

Question 1

What does 'exactly-once semantics' mean in a distributed streaming system?

Accepted Answer

Each message is processed and its effect is reflected exactly once, even in the presence of failures

Answer

Exactly-once semantics guarantee that despite failures and retries, each message's effect on state or output appears exactly once, requiring idempotency and transactional support.

Question 2

Which of the following is a key advantage of using Apache Iceberg over plain Parquet files on a data lake?

Accepted Answer

Iceberg provides ACID transactions, hidden partitioning, and schema evolution without full rewrites

Answer

Apache Iceberg adds a transactional metadata layer over Parquet (or ORC) files, enabling ACID operations, time travel, and seamless schema/partition evolution.

Question 3

In HDFS, what is the NameNode's primary function?

Accepted Answer

Maintaining the filesystem metadata and directory tree, tracking where blocks are stored

Answer

The NameNode is the master server that manages the HDFS namespace, tracking which DataNodes hold which blocks, but it does not store data itself.

Question 4

What is the purpose of Apache Oozie in the Hadoop ecosystem?

Accepted Answer

Workflow scheduling and coordination of Hadoop jobs

Answer

Apache Oozie is a workflow scheduler that chains and coordinates Hadoop jobs (MapReduce, Hive, Pig, Spark) based on time or data availability triggers.

Question 5

Which metric is most important when evaluating the performance of a big data storage system under write-heavy workloads?

Accepted Answer

Write throughput measured in MB/s or records/second

Answer

Write throughput (records or bytes per second) directly measures how quickly the system can ingest data, which is the primary bottleneck in write-heavy workloads.

DSE - Data Science Practice Test

DSE - Data Science Practice Test

DSE Big Data Technologies 5