ACP ACP Data Engineering & Workflow Automation 2

Question 1

Which Python library is specifically designed for defining, scheduling, and monitoring data pipeline workflows as Directed Acyclic Graphs (DAGs)?

Accepted Answer

All of the above

Answer

Luigi

Answer

Apache Airflow

Answer

Prefect

Question 2

In pandas, which method is used to apply a custom function to every row or column of a DataFrame?

Accepted Answer

df.apply()

Answer

df.map()

Answer

df.transform()

Answer

df.execute()

Question 3

Which pandas method efficiently removes duplicate rows from a DataFrame, keeping only the first occurrence by default?

Accepted Answer

df.drop_duplicates()

Answer

df.remove_duplicates()

Answer

df.unique_rows()

Answer

df.deduplicate()

Question 4

When building a data pipeline, what does the term 'idempotency' mean in the context of pipeline task execution?

Accepted Answer

Running a task multiple times produces the same result as running it once

Answer

A task runs in parallel with zero overhead

Answer

A task automatically retries on failure

Answer

A task can process data from any source format

Question 5

Which pandas method stacks a DataFrame from wide format (one column per variable) to long format (one row per observation)?

Accepted Answer

df.melt()

Answer

pd.wide_to_long()

Answer

df.stack()

Answer

df.pivot()

Question 6

In pandas, what does `groupby()` followed by `agg()` allow you to do?

Accepted Answer

Apply multiple aggregation functions to groups simultaneously

Answer

Sort a DataFrame by multiple columns

Answer

Filter rows based on group membership

Answer

Join two DataFrames on a common group key