ETL Testing ETL Performance and Scalability Testing 3

Question 1

Which monitoring metric would FIRST indicate that an ETL job is approaching memory exhaustion on its processing server?

Accepted Answer

Rising memory utilization percentage and possible swap space usage on the ETL server

Answer

Monitoring memory utilization and swap usage on the ETL server provides early warning of memory pressure that can cause job slowdowns, spills to disk, or out-of-memory failures.

Question 2

In ETL scalability testing, 'horizontal scaling' refers to:

Accepted Answer

Adding more processing nodes or worker instances to distribute the ETL workload

Answer

Horizontal scaling adds more parallel processing nodes to the ETL cluster, distributing the workload across more workers to handle increased data volumes.

Question 3

What ETL performance anti-pattern does 'row-by-row cursor processing' represent in SQL-based transformations?

Accepted Answer

A slow, inefficient approach that processes one record at a time instead of set-based SQL operations

Answer

Row-by-row cursor processing incurs per-row overhead and does not leverage the database engine's set-based optimization, resulting in dramatically slower performance than equivalent set-based SQL.

Question 4

Which ETL performance test validates that the pipeline can recover and complete within the batch window after a transient failure (e.g., a brief network outage)?

Accepted Answer

Resilience and recovery performance test

Answer

Resilience testing validates that after a recoverable failure, the ETL pipeline restarts from a checkpoint and finishes within the remaining batch window without full re-processing.

Question 5

In Apache Spark-based ETL performance testing, what does 'shuffling' refer to and why is it a performance concern?

Accepted Answer

The redistribution of data across Spark partitions during operations like joins and groupBy, which is network-intensive and a major performance bottleneck

Answer

Spark shuffle operations move data across the network between nodes to co-locate records with matching keys, generating heavy network I/O that is one of the most significant performance bottlenecks in Spark ETL.

ETL Testing - extract, transform, load Practice Test

ETL Testing - extract, transform, load Practice Test

ETL Testing ETL Performance and Scalability Testing 3