Data Engineering Data Serialization Formats

Question 1

What is the primary advantage of Apache Parquet over CSV for analytical workloads?

Accepted Answer

It stores data in columnar format enabling faster analytical queries

Answer

Parquet stores data in a columnar format, allowing analytical queries to read only the relevant columns and dramatically reducing I/O.

Question 2

Which serialization format uses binary encoding and requires a schema to serialize and deserialize data?

Accepted Answer

Apache Avro

Answer

Apache Avro uses binary encoding and requires a schema (defined in JSON) to serialize and deserialize data, making it compact and efficient.

Question 3

What type of internal storage organization does ORC (Optimized Row Columnar) use?

Accepted Answer

A hybrid of row stripes containing columnar data

Answer

ORC splits data into stripes (row groups) and within each stripe stores data in columnar format, combining the benefits of both row and columnar approaches.

Question 4

Which serialization format is most appropriate when humans need to inspect and edit data directly without special tools?

Accepted Answer

JSON

Answer

JSON is a human-readable text format that can be opened and understood in any text editor, making it ideal for debugging and manual inspection.

Question 5

What does 'schema evolution' mean in the context of data serialization?

Accepted Answer

The ability to modify a schema over time while maintaining compatibility with existing data

Answer

Schema evolution is the ability to modify a schema (e.g., add or remove fields) over time while still reading data written with older or newer schema versions.

Data Engineering Practice Test

Data Engineering Practice Test

Data Engineering Data Serialization Formats