🐲 Here be Dragons^H^H Stacktraces β€” Flink SQL for Non-Java Developers

A presentation at Kafka Summit 2024 in in London, UK by Robin Moffatt

Apache Flink might be the belle of the ball at the moment, but that doesn’t stop it from being baffling to learn at times. A platform steeped in its history as a Java project, it can be daunting for the humble data engineer equipped with only some SQL and their wits to navigate. And that’s a shame, because with Flink SQL you can do some rather useful things with streams (and batches) of data just using SQL - no coding required!

Join me as I map out the components of Flink, explore the bits that you doβ€”and don’tβ€”need to be familiar with to begin to work with Flink SQL. We’ll explore together the murky undergrowth of catalogs and connectors, clients and Calciteβ€”and take in a spot of architecture along the way to give us a proper understanding of what happens when we run a SQL statement on Flink.

By the end of this talk, you’ll have learnt how to run Flink locally, submit SQL jobs, integrate with various systems through source and sink connectors, and use Flink SQL for effective data transformation.

Whether you’re a seasoned data engineer or just starting out, you’ll leave with the confidence to hit the ground running with Flink SQL for yourself.


The following resources were mentioned during the presentation or are useful additional information.

Buzz and feedback

Here’s what was said about this presentation on social media.