Apache Spark Spark SQL and DataFrames

Question 1

What is the entry point for Spark SQL functionality in Spark 2.x and later?

Accepted Answer

SparkSession

Answer

SparkSession is the unified entry point for Spark SQL, DataFrame, and Dataset APIs introduced in Spark 2.0.

Question 2

Which method is used to run a SQL query on a registered temporary view in Spark SQL?

Accepted Answer

spark.sql()

Answer

spark.sql() executes a SQL query and returns the results as a DataFrame.

Question 3

What is a DataFrame in Apache Spark?

Accepted Answer

An RDD of Row objects with a named column schema

Answer

A DataFrame is a distributed collection of data organized into named columns, conceptually equivalent to a database table.

Question 4

How do you register a DataFrame as a temporary SQL view in Spark?

Accepted Answer

df.createOrReplaceTempView("name")

Answer

createOrReplaceTempView() registers a DataFrame as a temporary view scoped to the current SparkSession.

Question 5

Which DataFrame function is used to select specific columns?

Accepted Answer

select()

Answer

select() returns a new DataFrame with only the specified columns.

Apache Spark Practice Test