Data Processing Big Data Processing 2 — Questions and Answers
Question 1: What is stream processing in big data?
- Continuously processing data as it arrives in real time (Correct answer)
- Importing data in large batches overnight
- Storing data in sequential log files
- Processing only structured data from relational systems
Correct answer: Continuously processing data as it arrives in real time
Stream processing handles data continuously as it is generated, enabling real-time analysis and immediate actions on incoming data.
Question 2: What is Apache Kafka primarily used for?
- Distributed real-time data streaming and messaging (Correct answer)
- Storing structured relational data
- Running machine learning model training
- Generating PDF reports from databases
Correct answer: Distributed real-time data streaming and messaging
Apache Kafka is a distributed event-streaming platform used for high-throughput, fault-tolerant messaging and real-time data pipeline construction.
Question 3: What distinguishes batch processing from stream processing?
- Batch processing handles data in large collected groups at scheduled intervals; stream processing handles data continuously as it arrives (Correct answer)
- Batch processing is faster than stream processing
- Stream processing stores data permanently; batch processing does not
- Batch processing only works with unstructured data
Correct answer: Batch processing handles data in large collected groups at scheduled intervals; stream processing handles data continuously as it arrives
Batch processing accumulates data over a period and processes it all at once on a schedule, while stream processing ingests and analyzes data continuously in near-real-time.
Question 4: What is a data warehouse?
- A centralized repository of integrated, structured data optimized for reporting and analysis (Correct answer)
- A temporary cache for raw incoming data
- A tool for monitoring ETL job performance
- A distributed file system for unstructured data
Correct answer: A centralized repository of integrated, structured data optimized for reporting and analysis
A data warehouse is a subject-oriented, integrated, time-variant, and non-volatile collection of data designed to support business intelligence and decision-making.
Question 5: What is horizontal scaling in the context of big data systems?
- Adding more machines to a cluster to increase processing capacity (Correct answer)
- Upgrading the CPU and RAM of a single server
- Increasing the size of database indexes
- Adding more columns to existing tables
Correct answer: Adding more machines to a cluster to increase processing capacity
Horizontal scaling (scale-out) adds more nodes to a distributed system to increase capacity, as opposed to vertical scaling which upgrades a single machine.
Question 6: What is a partitioning strategy in big data storage?
- Dividing a large dataset into smaller subsets based on a key for parallel processing efficiency (Correct answer)
- Encrypting data segments individually for security
- Creating backup copies of data across regions
- Sorting data alphabetically before archiving
Correct answer: Dividing a large dataset into smaller subsets based on a key for parallel processing efficiency
Partitioning splits large datasets into smaller chunks based on a key (like date or region) so that queries and processing can target only the relevant subset.
Data Processing Big Data Processing 2
What is stream processing in big data?