Big Data Flashcards
7 cards from real MS-DS Master of Data science practice questions. Tap to flip, then mark Knew It or Still Learning — missed cards come back until you master them.
Read the first 7 Big Data flashcards as text
In the Lambda architecture, which layer handles real-time, low-latency data processing?
Answer: Speed layer
The speed layer processes incoming data in real time to provide low-latency views that complement the batch layer.
Which compression codec is best suited for Hadoop workloads that require splittable compressed files?
Answer: Bzip2
Bzip2 produces splittable compressed files, allowing Hadoop to divide them across multiple mappers.
In Apache Kafka, what guarantees message ordering within a topic?
Answer: Partition assignment
Kafka guarantees ordering only within a single partition, so messages with the same key are routed to the same partition.
Which HBase component stores the write-ahead log (WAL) to ensure durability?
Answer: HLog
The HLog (WAL) records all mutations before they are applied to the MemStore, enabling recovery after a crash.
The CAP theorem states that a distributed system can simultaneously guarantee at most how many of its three properties?
Answer: 2
CAP theorem proves that during a network partition, a system must sacrifice either consistency or availability, so only 2 of 3 properties can be fully guaranteed.
Which Apache Spark RDD operation is a wide transformation that triggers a shuffle across the cluster?
Answer: groupByKey()
groupByKey() requires data with the same key to be co-located, forcing a shuffle across partitions.
In HDFS, what is the default block replication factor?
Answer: 3
HDFS replicates each block 3 times by default to tolerate node failures while maintaining data availability.