Apache Flink might be the belle of the ball at the moment, but that doesn’t stop it from being baffling to learn at times. A platform steeped in its history as a Java project, it can be daunting for the humble data engineer equipped with only some SQL and their wits to navigate. And that’s a shame, because with Flink SQL you can do some rather useful things with streams (and batches) of data just using SQL - no coding required!
Join me as I map out the components of Flink, explore the bits that you do—and don’t—need to be familiar with to begin to work with Flink SQL. We’ll explore together the murky undergrowth of catalogs and connectors, clients and Calcite—and take in a spot of architecture along the way to give us a proper understanding of what happens when we run a SQL statement on Flink.
By the end of this talk, you’ll have learnt how to run Flink locally, submit SQL jobs, integrate with various systems through source and sink connectors, and use Flink SQL for effective data transformation.
Whether you’re a seasoned data engineer or just starting out, you’ll leave with the confidence to hit the ground running with Flink SQL for yourself.
