Sprinkling eBPF Onto Your Observability

When talking about Observability in 2022 there is no way around eBPF. However, eBPF is often seen as magic dust that can be sprinkled into infrastructure and magically do anything, when the reality is much more complicated. In this talk, Frederic will walk through eBPF's capabilities, and provide a mental framework that can be used when thinking about eBPF's capabilities. Beyond that Frederic will demonstrate the real-world use of eBPF in next-generation Observability tooling using the open source continuous profiling project Parca and how it complements an existing Observability stack.

What is the focus of your work these days?

I work on continuous profiling software. People may be familiar with regular profiling where you look at a single process for a 10 secs period of time. We get to see, for example, where CPU time within that process is being spent. Continuous profiling means that we're always doing this to all processes and infrastructure and storing this data over time. That allows us to do some super interesting analysis on this data. For example, if we had an incident, we can go back and look at what our processes were doing. We can look at not just a 10 secs period of time, we can look at where the CPU time in our entire infrastructure is being spent. So it allows us to do a novel introspection of our running infrastructure that is incredibly useful for a variety of use cases.

What's the motivation behind your talk?

Some of my talk is related to what we do with continuous profiling, because in order to be able to collect this data at low overhead, we needed to look around at what technology was most suitable to do this. eBPF happened to be the right tool for the job. Actually, at the beginning of when we started looking into the continuous profiling space, we very intentionally did not want to concern ourselves with the collection of the data. We were thinking the storage and querying of this stuff is hard enough, but it became painfully obvious at some point that the overhead of collection was going to be vital for adoption of this technology. 

I think the formula is easy to understand, if the overhead outweighs the benefit of optimizations that we can do based on this data, then it's not really worth doing it. So it's a game of getting the overhead down as much as possible while getting the potential outcome as high as possible. And eBPF allows us to grab this data at a super, super low overhead.

 

How would you describe the persona and level of the target audience for your session?

This is interesting because eBPF is so versatile, it can be anyone from software engineers who just want to have meaningful insight into their applications. Like I said, with continuous profiling, for example, we find that sometimes even CFOs are interested in this because they can make calculations like, this is the cost of resources per customer or something like that. They want to be able to drive that down as much as possible. So it is really huge, but typically we find that SREs, infra people or software engineers have the most use out of this technology.

Is there anything that you'd like to highlight that you would like this persona to walk away with after watching your session?

It's a mix that eBPF is already, but will continue, to disrupt the observability space. The nature of how eBPF works is is just amazing because all of these trace points have been in the Linux kernel forever, but they were inaccessible, and eBPF all of the sudden all of these trace points that have been maintained by kernel developers over many, many years, now they're becoming accessible to the common folk. One point I want to warn people to walk away with is that there are incredibly exciting opportunities with eBPF. 

The second point is maybe a little bit more realistic where I want to show that while all of these things are really shiny and there is a lot of opportunity, there is also a lot of work that needs to go into this to actually make it happen. 

My last point is going to be, that there are already projects such as the open source project that I happen to work on that have put in this work so that some use cases using eBPF are already very viable to be used today. So this progression of eBPF is exciting, then it's still hard, but there are projects out there that are very successfully using it.

 


Speaker

Frederic Branczyk

CEO & Founder @PolarSignalsIO, previously Senior Principal Engineer @Redhat

Frederic is the founder and CEO of Polar Signals. Before founding Polar Signals he was a senior principal engineer and the main architect for all things Observability at Red Hat, which he joined through the CoreOS acquisition. Frederic is a Prometheus and Thanos maintainer as well as until recently was the tech lead for the special interest group for instrumentation in Kubernetes. In a previous life, he was a security researcher working on key management solutions as well as intrusion detection systems. When not working on software Frederic enjoys obsessing over brewing a perfect cup of coffee.

Read more

Date

Tuesday Dec 6 / 10:10AM PST ( 50 minutes )

Topics

Modern Infrastructure Languages Observability eBPF Languages

Share

From the same track

Session Modern Infrastructure

Taming Configuration Complexity Made Fun with CUE

Tuesday Dec 6 / 09:00AM PST

Configuration has become the number one complexity problem to solve in infrastructure and beyond. Configuration is in more places than people imagine. Every part of your tech stack—databases, apps, schemas, services, workflows, policy, models, networking—must be configured.

Marcel van Lohuizen

Creator of CUE

Session Modern Infrastructure

Programming Your Policies

Tuesday Dec 6 / 11:20AM PST

Software is eating the world, and this talk is about how it is coming to eat the world of policy. I will talk about why this is happening, what the business drivers are, and how it affects developers and compliance and security departments, and the cultural and communication changes there.

Justin Cormack

CTO @Docker

Session Modern Infrastructure

Infrastructure as Code: Past, Present, Future

Tuesday Dec 6 / 12:30PM PST

Infrastructure as code enables us to automate and manage all sorts of infrastructure, from on-premises virtual machines to cloud resources, and everything in between.

Joe Duffy

Founder and CEO @PulumiCorp