Biography

Experienced software engineer with more than 20 years of experience. Confluent Certified Developer for Apache Kafka.

Talk

Choosing the Right Abstraction Level for Your Kafka Project

To develop Kafka applications using JVM languages we can choose between two levels of abstraction, the low level KConsumer/KProducer or the high level KafkaStreams API. The low level KConsumer/KProducer API is simple and provides simple data access. We can build really powerful applications using this API together with the data manipulation capabilities found in Java 8+, Scala or Kotlin.

The KafkaStreams API is much higher level and provides abstractions that greatly simplify the development of applications. This comes at a price though, using the KafkaStreams API requires that an application follows a standard pattern. How to decide which API to use is not as simple as it seems. There are many factors that need to be considered.

Does the application require dynamic configuration of the pipeline or is the pipeline configuration relatively static? Do we need to interact with more than one kafka cluster? What kind of operations need to be applied to the events? Do we need to interact with external systems? In this presentation, he will go through several scenarios and cases to highlight the key factors that should be considered when deciding which API should be used for a given project.

Session Keywords