Kafka Connect Flashcards

What is the Schema Registry's role when used with Kafka Connect?

Answer: Centrally stores and validates Avro/JSON/Protobuf schemas so producers and consumers share a consistent contract

The Schema Registry stores schemas by subject so that Connect converters can serialize and deserialize messages with schema versioning and compatibility enforcement.

What does `value.converter=io.confluent.kafka.serializers.KafkaAvroSerializer` indicate in a Connect configuration?

Answer: Record values will be serialized using Avro format against the Schema Registry

Setting the value converter to KafkaAvroSerializer tells Connect workers to serialize record values as Avro binary, registering schemas with the Schema Registry.

Which Kafka Connect mode is most suitable for development/testing with a single worker?

Answer: Standalone mode

Standalone mode runs a single Connect worker process with connectors defined in property files, ideal for development and simple deployments.

What is the 'status storage topic' in Kafka Connect distributed mode?

Answer: Stores the runtime status of connectors and tasks so all workers have a consistent view of the cluster state

The status storage topic tracks the state transitions (RUNNING, PAUSED, FAILED) of all connectors and tasks, enabling the REST API to report accurate status.

How do Dead Letter Queues (DLQs) work in Kafka Connect sink connectors?

Answer: Records that fail to be processed are routed to a configurable DLQ topic instead of stopping the connector

A DLQ allows a sink connector to continue processing by redirecting problematic records to a separate topic, preventing one bad record from halting the pipeline.

What configuration property controls the maximum number of tasks a Kafka Connect connector can use?

Answer: tasks.max

tasks.max sets the upper bound on the number of tasks the connector framework will create; the connector may create fewer based on available parallelism.