LFC Photon Engine Optimization 3

Question 1

Which file size scenario best utilizes Photon's performance capabilities?

Accepted Answer

Optimally-sized files of 128 MB to 1 GB

Answer

Photon performs best with optimally-sized files that minimize the small-files problem overhead while providing sufficient parallelism and data for vectorized processing.

Question 2

When using Photon with Delta Lake's OPTIMIZE command, which index structure improves data skipping effectiveness?

Accepted Answer

Z-order clustering on high-cardinality filter columns

Answer

Z-order clustering co-locates related data by multiple columns, making Delta Lake's file-level statistics more selective and enabling Photon to skip more files.

Question 3

Which SQL operation type is NOT natively accelerated by Photon?

Accepted Answer

Arbitrary Python UDFs wrapped in pandas_udf

Answer

Python UDFs, including pandas UDFs, run in the Python interpreter and cannot be executed natively by Photon's C++ engine.

Question 4

In a Photon-enabled cluster, what is the effect of increasing the number of shuffle partitions on a large aggregation query?

Accepted Answer

Too many partitions create small shuffle files that reduce Photon's vectorized batch efficiency

Answer

Excessively many shuffle partitions produce very small data batches per partition, reducing the effectiveness of Photon's vectorized processing which benefits from larger batches.

Question 5

Which runtime statistic in the Spark UI is most useful for identifying whether Photon is providing CPU efficiency gains?

Accepted Answer

CPU time vs. wall-clock time ratio per stage

Answer

A high CPU utilization relative to wall-clock time indicates Photon is efficiently using CPU resources rather than waiting on I/O or coordination overhead.

(LFC) Lakehouse Fundamentals Certification Practice Test

(LFC) Lakehouse Fundamentals Certification Practice Test

LFC Photon Engine Optimization 3