Big Data & Cloud Analytics Flashcards

In HDFS, what is the default behavior for ensuring data durability across the cluster?

Answer: Replicating each block across multiple nodes

HDFS replicates each data block (default 3 copies) across DataNodes for fault tolerance.

Which type of database is best suited for storing and querying highly connected data such as social networks?

Answer: Graph database

Graph databases model entities as nodes and relationships as edges, ideal for connected data.

In stream processing, what does 'windowing' primarily allow you to do?

Answer: Group events into finite time-based buckets for aggregation

Windowing groups continuous stream events into bounded intervals so aggregations can be computed.

Which Apache project is a distributed publish-subscribe messaging system commonly used for real-time data pipelines?

Answer: Apache Kafka

Apache Kafka is a distributed publish-subscribe streaming platform for real-time pipelines.

In Azure, which service provides a managed Apache Spark and Delta Lake analytics platform?

Answer: Azure Databricks

Azure Databricks is a managed analytics platform built around Apache Spark and Delta Lake.

What does the CAP theorem state a distributed data store can guarantee at most two of?

Answer: Consistency, Availability, Partition tolerance

The CAP theorem says a distributed system can guarantee at most two of Consistency, Availability, and Partition tolerance.

Which storage format is column-oriented and optimized for analytical queries in big data systems?

Answer: Apache Parquet

Apache Parquet is a columnar storage format that improves compression and analytical query speed.