You are viewing content from a past/completed QCon Plus - November 2020

The video on-demand of this panel is available to logged in QCon attendees only. Please login to your QCon account to watch the panel.

PANEL DISCUSSION + Live Q&A

Modern Data Engineering Panel

Data Engineering is a vast field that concerns itself with efficient access to data based on the needs of a business. Though data is the prized entity from which a company extracts insights, data doesn't exist in a void. It first needs to be stored somewhere and then an API needs to be provided such that a client can access this data. Therein lie the opportunity and the challenge. We have seen an explosion in technologies in the field of data engineering (OLTP DBs, OLAP DBs, Data Streams, Big Data Processing, Search Engines, Graph Processing and Graph Serving Engines, Caches, Block and Object Stores, etc.... ). When you consider the myriad ways these puzzle pieces can be put together to build a modern data engineering stack, you soon find that no 2 stacks resemble one another. 

What does a data engineer need to know in order to be successful in today's world? What are some best practices, pitfalls, and ways to think about building a low cost-of-ownership, high-quality data platform? What technologies are non-starters and why? What technologies are hidden gems? Finally, what should the industry think about and what is coming next? Join our panel of experts as we explore these questions in order to shed light on these areas. 
 

Speaker

Shrijeet Paliwal

Sr. Staff Software Engineer @Tesla

Shrijeet is a founding member of data platform team at Tesla, tasked to build a scalable, self-served, cost-efficient data platform that caters everything data at Tesla. He has his hands all the way from improving server provisioning & automation to writing multi-petabyte time series storage....

Read more

Speaker

Chinmay Soman

PMC Member/Commiter @SamzaStream

Chinmay Soman has been working in the distributed systems domain for the past 10+ years. He started out in IBM where he worked on distributed file systems and replication technologies. He then joined the Data Infrastructure team in LinkedIn and worked on open source technologies such as Voldemort...

Read more
Find Chinmay Soman at:

Speaker

Chris Riccomini

Distinguished Engineer @WePay

Chris Riccomini is a software engineer, startup investor, and advisor with more than a decade of experience at major tech companies such as PayPal, LinkedIn, and WePay. He has been involved in open source throughout his career and is the author of Apache Samza. He's recently written The...

Read more
Find Chris Riccomini at:

Speaker

Shrijeet Paliwal
Shrijeet Paliwal

Sr. Staff Software Engineer @Tesla

Speaker

Chinmay Soman
Chinmay Soman

PMC Member/Commiter @SamzaStream

Speaker

Chris Riccomini
Chris Riccomini

Distinguished Engineer @WePay

Date

Tuesday Nov 17 / 03:30PM EST (40 minutes)

Track

Modern Data Engineering

Add to Calendar

Add to calendar

Share

Share Share

From the same track

Session + Live Q&A

Building Latency Sensitive User Facing Analytics via Apache Pinot

Tuesday Nov 17 / 02:40PM EST

Real-time analytics has become the need of the hour for modern Internet companies. The ability to derive internal insights around business metrics, user growth & adoption as well as security incidents from all the raw logs is crucial for day to day operation. Even more critical is enabling...

Chinmay Soman

PMC Member/Commiter @SamzaStream

Session + Live Q&A

Serverless Search for My Blog With Java, Quarkus, & AWS Lambda

Tuesday Nov 17 / 01:00PM EST

A Serverless app? With Java?! Absolutely!We’ll discuss when Serverless is a great fit (and when it isn’t!) and why you don’t need to leave the Java platform when going Serverless. Based on the real-world example of a Serverless blog search, you’ll learn how Quarkus and...

Gunnar Morling

Open Source Software Engineer @RedHat

Session + Live Q&A

Designing IoT Data Pipelines for Deep Observability

Tuesday Nov 17 / 01:50PM EST

Millions of IoT devices emitting trillions of events per day enable us to track the health of the Tesla fleet. From a data engineering perspective, it's a challenging scale, but what makes it unique is how naturally fragile the data pipeline is. The physical world is full of chaos: what if...

Shrijeet Paliwal

Sr. Staff Software Engineer @Tesla

View full Schedule