ACP ACP Data Engineering & Workflow Automation 3

Question 1

Which Python library provides the `Parquet` file format support for high-performance columnar storage of DataFrames?

Accepted Answer

pyarrow

Answer

`pyarrow` (and `fastparquet`) enables pandas to read/write Parquet files via `df.to_parquet()` and `pd.read_parquet()`, offering efficient columnar compression for large datasets.

Question 2

Which conda command exports a complete list of all packages in the current environment to a file for reproducibility?

Accepted Answer

conda env export > environment.yml

Answer

`conda env export > environment.yml` captures all packages, versions, and channels in the active environment into a YAML file that can recreate the environment elsewhere.

Question 3

In a data engineering context, what is the purpose of using Dask instead of pandas for large dataset processing in Python?

Accepted Answer

Dask enables parallel and out-of-core computation on datasets larger than RAM

Answer

Dask partitions large datasets into chunks and processes them in parallel across cores or a cluster, handling data that exceeds available memory with a pandas-compatible API.

Question 4

Which method in pandas handles missing values by filling them with a specified value or strategy (e.g., forward fill)?

Accepted Answer

df.fillna()

Answer

`df.fillna(value)` replaces NaN values with a constant or uses methods like `ffill` (forward fill) and `bfill` (backward fill) to propagate adjacent valid values.

Question 5

When automating a data pipeline on a schedule using cron or a task scheduler, which Python standard library module provides programmatic access to run shell commands and subprocesses?

Accepted Answer

subprocess

Answer

The `subprocess` module provides `subprocess.run()` and `Popen` for launching external processes, capturing output, and handling errors within automated Python pipeline scripts.

(ACP) Anaconda Certified Professional Practice Test

(ACP) Anaconda Certified Professional Practice Test

ACP ACP Data Engineering & Workflow Automation 3