Orchestration vs Choreography, A Guide To Composing Your Monolith

Interview:

Can you tell me what the focus of your work is these days?

At the moment I'm leading a team who are building a low-code platform for fintech. So we mainly work with banks like Bank of America, BNY Mellon, BMO, Citibank, etc. and we're looking at ways that we can accelerate software delivery in highly regulated industries. My role is a technology leadership role. I work with four or five squads of engineers, leading engineering governance, best practices and strategies, and a lot of coaching and mentoring of people to try and help us get to the point where we can have a really successful product.

You touched on this a little bit, but what's the motivation behind your talk?

One of the things I've noticed throughout my career in bigger companies is that people don't always consider all of the complexity and coupling of their decisions. And especially with microservices, I think there's a lot of rush to get into them, but there’s perhaps not necessarily the design up-front that helps people to make them a long term success. And so I'm really interested in exploring how things like orchestration and choreography patterns for interactions affect your choices for scaling your software, scaling platforms, but also scaling the organization and making sure you've got a healthy future as a business. 

You see lots of different ways that this manifests itself in businesses. When I was working at Sky Bet, at one point we had this huge wall with all of the squads in the business that had a relationship to each other mapped on a matrix. Lots of dependencies were mapped out on it and we'd have this weekly walk through the board and it would take ages and it's like, “Oh, have you delivered this feature?”, “No, because I'm waiting on this other team. But that team can't deliver that feature because of another team”...Without the proper care and attention up-front, you can end up sleepwalking into that sort of situation, where you're stopping the ability to deliver. Not to mention the times where an outage in a seemingly unrelated service leads to customer facing issues.

The talk highlights the thinking in this area, ways that you can avoid accidental coupling and how you can implement patterns and contracts between teams and services much more effectively. It’s a look at how you can set yourself up for long term success when working with microservices.

How would you describe the persona and level of the target audience for your session?

I'd see it as someone who's an architect, a tech lead, or someone who's progressing through from senior to a Staff+ position. Possibly also talking to people more on the engineering management side like VPs and potentially CTOs. Because there's two facets to it. One is the technical aspects where people need to avoid thinking, “well, I'm just going to put events in my system and it's a slam dunk and everything's going to be rosy”. There are trade offs they have to consider, and they can then work them through and discuss them properly in their design sessions. The second is looking from the organizational side of things, it's very important that the business is engaged and supports the decisions as well, and work through the designs to understand what their implications are from a people perspective. How the teams are going to interact with each other, what the growth plans for the organization might look like and how they're going to support whatever's built in the future. 

There can be some very senior people who perhaps haven't managed to see the benefits of giving more empowerment to individual teams and trying to decouple them in a very positive way. You might have years and years of experience with technology but empowered, autonomous teams may be quite a new thing to embrace. I think it’s Martin Fowler who says, “an architect’s value is inversely proportional to the number of decisions he or she makes”. And I think that's a really good way of framing some of this stuff. By thinking about coupling and the way your teams interact with each other properly, you can level up their maturity and you can also defer the decision making to those teams. And you don't have to have these bottlenecks in your business that will ultimately slow you down, you’ve embraced the first ideal - locality and simplicity.

If you had to choose a couple of highlights that you'd like somebody to walk away with after watching a presentation, what would those be?

I think all senior engineers need to learn that ultimately most of these technology problems are people problems. You've got to balance the needs of the customer, the needs of the business to the needs of all the colleagues that you work with and how you can be working effectively to make them successful. And then work through that to get to the technology. For those who are more technology focused, there are always trade offs to consider when you're designing systems and when you're designing software. I want to make sure people have been given an opportunity to think of some of the trade offs that they may not have considered before and some of the things that perhaps don't get talked about as much. Because with early adopter conferences, you tend to hear a lot about things like eBPF and Service Mesh and all these technologies. They sound really exciting but there's more fundamental things, first principles things that actually you need to consider and be mindful of. And the last thing that I had is that you really need to take the time to do the design, because when people hear microservices, especially if they're trying to be agile in their delivery, they can rush through and they don't necessarily do the governance that they need upfront. At some point, you do need to take the time to be designing deliberately what you're going to end up with, rather than just rushing into things, you could build this thing really quickly, and you find in 6, 12, 18 months+ that it is really bad for you and you're in a worse place than when you started.