You are viewing content from a past/completed QCon - November 2020

Pitfalls and Patterns in Microservice Dependency Management

What You'll Learn

1Hear stories how a small change can impact a system.

2Learn about the importance of having a broad view of a system to better understand how a change can impact a system and its customers.

A distributed system is one in which the failure of a computer you didn't even know existed can render your own computer unusable." -- Leslie Lamport, May 28, 1987. Building a product based on microservices has many advantages as vertical/horizontal scaling, independent release cycles and logical decoupling. But allowing microservices connections to grow unbounded can also lead to complex architectures. In this talk, we will review the most common traps and tips for better alignment between microservices traffic and their backend dependencies. Have you ever launched a new feature just to discover your database didn't have enough resources to cope with the new requests? Has your service experienced poor performance in one region when a backend had an outage in another one? Can a service offer a higher SLO than its backends? Let's take a deep look into these scenarios and understand what happens and how can we guarantee good service at all times.

Tell me a bit about your talk.

I'm going to be talking about dependency management in an environment of microservices. And I'm going to present three interesting scenarios that changed my way to interpret the environment. They were key moments where I noticed that the binary was not everything, that the traffic between the binaries made a huge difference. The first one was when I was still a junior engineer and it was related to traffic. I think it was my first launch, the feature I first launched it. To summarize, it affected the traffic into one of our back-ends. I didn't give notice to the back-end owners and they noticed the spike. I don't want to spoil the surprise, but they were very surprised about my launch. Let's say that. And then I learned at that point that every time I change the binary, the flow of requests to the back-end will be affected. I have to be considerate of that.

What's the takeaway?

I'm going to be giving some tips from my personal learnings. I will tell you exactly what went wrong, so you can look for the signs. If you notice something is going wrong, maybe it's similar to what I experienced, but then I'm going to let you know what I did to make it better to avoid similar situations.

Who's your core persona you're talking to?

All these scenarios happened when I was working with other roles in engineering. At one point I was working with a PM. At another point, I was a SRE working with devs. And on the last one, I'm working with basically everybody because our goal is to build reliable services. It's not just one person, not one role that will make this happen. I think all three scenarios can be useful for a variety of roles.

What would be the one thing that you think will really stand out?

I hope people start thinking about their systems in a more holistic way. Every time you change your system, this will affect many other parts, many other components of the whole product. It will affect the back-ends, either if they are run by your own company or if you run them in the cloud or whatever. And they will definitely affect the customer. And this is the person that you want to keep in mind at all times. But before your change reaches the customer, it might change just a bit. So, again, be thorough when you are designing a change in a system. Consider all the nuances and keep focus on the customer.


Silvia Esparrachiari

Software Engineer @Google
Silvia has been a software engineer at Google for 10 years, having worked at User Data Privacy, Spam and Abuse Prevention, and most recently in Cloud and Infrastructure/SRE. She has a bachelor in Molecular Science and a masters in Computer Vision and Human-Computer Interaction. Her current focus... Read more

Tuesday Nov 10 / 01:50PM EST (40 minutes)

TRACK Operating Microservices ADD TO CALENDAR Calendar IconAdd to calendar

Build your learning journey and level-up on the skills most in-demand in 2021. Attend QCon Plus (May 17-28, 2021).

Save your spot for $599 before May 28th