Shrijeet Paliwal

Sr. Staff Software Engineer @Tesla

Shrijeet is a founding member of data platform team at Tesla, tasked to build a scalable, self-served, cost-efficient data platform that caters everything data at Tesla. He has his hands all the way from improving server provisioning & automation to writing multi-petabyte time series storage. Prior to Tesla, Shrijeet has contributed to data infrastructure at Pinterest and Rocket Fuel.


Designing IoT Data Pipelines for Deep Observability

Millions of IoT devices emitting trillions of events per day enable us to track the health of the Tesla fleet. From a data engineering perspective, it's a challenging scale, but what makes it unique is how naturally fragile the data pipeline is. The physical world is full of chaos: what if network partitions lasted days, not seconds? Ever suffered data loss due to sub-zero temperatures? As these faults propagate, they threaten the quality of data-driven insights.

In this talk, we will share recipes that immunize IoT data pipelines against these faults from the get-go. We have built these recipes around a toolchain called dataflow. While designed for IoT, dataflow concepts generalize well. Learn how to apply these techniques to draw insights such as a drop in coverage, anomalous patterns & end to end processing latencies.


Tuesday Nov 17 / 10:50AM PST (40 minutes )

TRACK Modern Data Engineering ADD TO CALENDAR Add to calendar SHARE

3 weeks of live software engineering content designed around your schedule.

Don’t miss out! Save your seat now