Modern Data Engineering

In the "old" days, data engineering was all about getting data from a transactional data store to a data warehouse, whereby it enabled business analysts to create reports and gain insights. In this track we’ll explore the ways in which data engineering has changed in the last few years and has become key to the success of products and companies - new requirements breed new solutions. Data & data engineering are used to power mission critical production applications, create powerful network effects, predict user behavior and drive cutting-edge AI. Come to this track to learn about recent advances in the theory and practice of data engineering.

TRACK HOST

Sid Anand

Chief Architect @Datazoom, PMC/Committer @Apache Airflow, Co-chair @QCon I currently serve as the Chief Architect for Datazoom. Prior to joining Datazoom, I served as PayPal's Chief Data Engineer, focusing on ways to realize the value of data. Prior to joining PayPal, I held several positions including Agari's Data Architect, a Technical Lead in Search @ LinkedIn, Netflix’s Cloud Data Architect, Etsy’s VP of Engineering, and several technical roles at eBay. I earned my BS and MS degrees in CS from Cornell University, where I focused on Distributed Systems. Outside of work, I'm a maintainer/committer on Apache Airflow and I co-chair QCon, a family of popular practitioner-driven technical conferences, and Data Council, a global Data & AI conference. I also offer technical advisory services to startups and serve on the boards of a few companies. Find Sid Anand at:

TRACK HOST Sid Anand Chief Architect @Datazoom, PMC/Committer @Apache Airflow, Co-chair @QCon

From this track

Session

Serverless Search for My Blog With Java, Quarkus, & AWS Lambda

Tuesday Nov 17 / 10:00AM PST

A Serverless app? With Java?! Absolutely!We’ll discuss when Serverless is a great fit (and when it isn’t!) and why you don’t need to leave the Java platform when going Serverless. Based on the real-world example of a Serverless blog search, you’ll learn how Quarkus and...

Gunnar Morling Open Source Software Engineer @RedHat

Session

Designing IoT Data Pipelines for Deep Observability

Tuesday Nov 17 / 10:50AM PST

Millions of IoT devices emitting trillions of events per day enable us to track the health of the Tesla fleet. From a data engineering perspective, it's a challenging scale, but what makes it unique is how naturally fragile the data pipeline is. The physical world is full of chaos: what if...

Shrijeet Paliwal Sr. Staff Software Engineer @Tesla

Session

Building Latency Sensitive User Facing Analytics via Apache Pinot

Tuesday Nov 17 / 11:40AM PST

Real-time analytics has become the need of the hour for modern Internet companies. The ability to derive internal insights around business metrics, user growth & adoption as well as security incidents from all the raw logs is crucial for day to day operation. Even more critical is enabling...

Chinmay Soman PMC Member/Commiter @SamzaStream

Interactive Session

Modern Data Engineering Panel

Tuesday Nov 17 / 12:30PM PST

Data Engineering is a vast field that concerns itself with efficient access to data based on the needs of a business. Though data is the prized entity from which a company extracts insights, data doesn't exist in a void. It first needs to be stored somewhere and then an API needs to be...

Shrijeet Paliwal Sr. Staff Software Engineer @Tesla Chinmay Soman PMC Member/Commiter @SamzaStream Chris Riccomini Distinguished Engineer @WePay

3 weeks of live software engineering content designed around your schedule.

Don’t miss out! Save your seat now

TOP