Apache Spark Spark Streaming

Question 1

What is the fundamental abstraction in Spark Structured Streaming?

Accepted Answer

Unbounded DataFrame/Dataset

Answer

Structured Streaming treats a live data stream as an unbounded table (DataFrame/Dataset) that is continuously appended.

Question 2

What is a DStream in Spark Streaming?

Accepted Answer

A sequence of RDDs representing a continuous stream of data

Answer

A DStream (Discretized Stream) is a continuous sequence of RDDs, each representing data from a time interval.

Question 3

Which output mode in Structured Streaming writes only new rows added to the result table since the last trigger?

Accepted Answer

Append

Answer

Append mode writes only new rows appended to the result table since the last trigger to the sink.

Question 4

What is a watermark in Spark Structured Streaming?

Accepted Answer

A threshold defining the maximum allowed lateness for event-time data

Answer

A watermark specifies how late event-time data can be and still be included in the computation, allowing Spark to drop old state.

Question 5

Which of the following is a valid trigger for Structured Streaming in Spark?

Accepted Answer

ProcessingTime trigger

Answer

ProcessingTime trigger runs micro-batches at a specified interval, e.g., Trigger.ProcessingTime('10 seconds').

Apache Spark Practice Test