MS-DS Master of Data science Master of Data science Big Data 1

Question 1

Which file format is optimized for columnar storage and is commonly used in big data ecosystems for efficient query performance?

Accepted Answer

Parquet

Answer

Parquet is a columnar storage format designed for big data processing. It stores data by column rather than row, enabling efficient compression and query performance when only specific columns need to be read.

Question 2

In the Lambda architecture for big data processing, which layer handles real-time data processing with low latency?

Accepted Answer

Speed layer

Answer

The speed layer (also called the streaming layer) in Lambda architecture processes incoming data in real time with low latency to provide up-to-date results, compensating for the high latency of the batch layer.

Question 3

Which algorithm is commonly used for distributed machine learning to split gradient computation across multiple nodes?

Accepted Answer

Distributed Gradient Boosting

Answer

Distributed Gradient Boosting (as implemented in frameworks like XGBoost and LightGBM) splits the gradient computation and histogram building across multiple nodes, enabling scalable ensemble learning on large datasets.

Question 4

What is the primary role of Apache ZooKeeper in a big data cluster?

Accepted Answer

Distributed coordination and configuration management

Answer

Apache ZooKeeper provides distributed coordination services such as configuration management, naming, synchronization, and group membership, allowing distributed applications like Hadoop and Kafka to maintain consistency across nodes.

Question 5

Which concept in big data describes the phenomenon where storing and processing costs decrease as the volume of data increases per unit?

Accepted Answer

Economies of scale

Answer

Economies of scale in big data refers to the cost efficiency gained as data volume grows — distributed systems like Hadoop and cloud storage reduce the per-unit cost of storage and computation at higher volumes.

MS-DS Master of Data science Practice Test

MS-DS Master of Data science Practice Test

MS-DS Master of Data science Master of Data science Big Data 1