Azure Real-Time and Batch Analytics Flashcards

Which Azure service is the managed platform for running Apache Spark and Apache Hadoop clusters for large-scale batch processing?

Answer: Azure HDInsight

Azure HDInsight is Microsoft's fully managed cloud distribution of open-source frameworks including Hadoop, Spark, Kafka, and HBase.

What Azure service provides a fully managed, serverless Apache Spark environment integrated with Azure Synapse Analytics?

Answer: Azure Synapse Spark pool

Azure Synapse Spark pools provide on-demand, auto-scaling Apache Spark clusters within the Synapse workspace, billed per second of use.

In Azure Event Hubs, what is a consumer group?

Answer: A logical view of an Event Hub that enables multiple consumers to read the stream independently at their own pace

Consumer groups allow multiple downstream consumers (e.g., Stream Analytics, Databricks, a custom app) to each read the full event stream independently without interfering.

Which Azure service is best suited for orchestrating complex data workflows that include both batch and real-time components with dependencies between steps?

Answer: Azure Data Factory

Azure Data Factory is a cloud-scale data integration service that orchestrates both batch ETL and hybrid data pipelines with rich dependency and scheduling capabilities.

In a lambda architecture built on Azure, what is the role of the batch layer?

Answer: To reprocess all historical data periodically to produce accurate, comprehensive batch views

The batch layer processes the complete historical dataset to produce accurate batch views, compensating for any errors in the speed layer.

What is the primary purpose of Azure Event Hubs Capture?

Answer: To automatically archive streaming events to Azure Blob Storage or ADLS Gen2 for batch analysis

Event Hubs Capture automatically stores the full stream of events to Azure Blob Storage or ADLS Gen2 in Avro format for downstream batch processing.