Data Science with Python Data Cleaning and Preparation Questions and Answers

Question 1

You are cleaning a dataset of customer information in a pandas DataFrame named 'df'.
The 'phone_number' column contains numbers in various formats (e.g., '(123) 456-7890', '123.456.7890', '1234567890').
Which of the following is the most effective approach to standardize all phone numbers to the format '1234567890'?

Accepted Answer

Using the `.str.replace()` method with a regular expression to remove all non-digit characters.

Answer

Using a for loop to iterate over each phone number and manually replace characters.

Answer

Applying the `astype(int)` method to the column.

Answer

Using the `fillna()` method to replace malformed numbers with a standard one.

Question 2

A data scientist is preparing a dataset for a machine learning model. They suspect the presence of outliers in a feature column that is approximately normally distributed. Which of the following methods is a common statistical approach to identify these outliers?

Accepted Answer

Calculating the Z-score for each data point and identifying values with an absolute Z-score greater than a threshold (commonly 3).

Answer

Calculating the median absolute deviation (MAD) and identifying values beyond a certain threshold.

Answer

Grouping the data by value counts and identifying the least frequent values.

Answer

Forward-filling the data to smooth out extreme values.

Question 3

While cleaning a pandas DataFrame, you discover that a column 'start_date', which should contain dates, is currently of the `object` dtype. Which pandas function is specifically designed to convert such a column to a proper datetime format?

Accepted Answer

pd.to_datetime(df['start_date'])

Answer

df['start_date'].astype('datetime64[ns]')

Answer

df['start_date'].apply(lambda x: datetime.strptime(x, '%Y-%m-%d'))

Answer

pd.to_numeric(df['start_date'])

Question 4

You have a DataFrame `sales_data` with duplicate rows based on an 'order_id' column. You need to remove these duplicates, keeping only the most recent entry for each 'order_id', which corresponds to the last occurrence in the DataFrame. Which line of code will achieve this?

Accepted Answer

sales_data.drop_duplicates(subset=['order_id'], keep='last')

Answer

sales_data.drop_duplicates(subset=['order_id'], keep='first')

Answer

sales_data.drop_duplicates(subset=['order_id'], keep=False)

Answer

sales_data[~sales_data.duplicated(subset=['order_id'], keep='last')]

Question 5

A DataFrame contains a 'rating' column with some missing values represented as `np.nan`. The goal is to replace these missing values with the average rating of the column. Which of the following pandas operations correctly performs this imputation?

Accepted Answer

df['rating'].fillna(df['rating'].mean())

Answer

df['rating'].dropna()

Answer

df['rating'].replace(np.nan, 'mean')

Answer

df['rating'][df['rating'].isnull()] = df['rating'].median()

Question 6

Which of the following scenarios is the primary reason for converting a column's data type from `object` to a more specific type like `int` or `float` during data cleaning?

Accepted Answer

To enable mathematical operations and improve memory efficiency.

Answer

To make the DataFrame display more aesthetically pleasing.

Answer

To ensure all columns have the same number of unique values.

Answer

To prepare the data for export to a JSON file, which requires numeric types.