You are viewing content from a past/completed QCon Plus - November 2020

Session

From Monolith to Microservices

Due to its low overhead and centralized management, companies often start building products under a monolithic architecture.  But as team sizes grow and product use cases become more complicated, this often slows down software development and adds unnecessary friction in the deployment and troubleshooting processes.  

In this talk, we aim to take a practical approach to software architecture decisions.  We will be taking a deeper look at GitHub’s historical and current state, go over some internal and external factors, and discuss practical consideration points in how we started to tackle this migration.  Then we will walk through some key concepts and best practices of implementing a microservices architecture that you can apply as you think about this transformation for your organization.

Main Takeaways

1 Hear how GitHub plans to evolve their current Ruby on Rails monolith to accommodate future scaling both of the development team and the codebase.

2 Learn some best practices to rely on when introducing a microservices architecture.


What are you going to focus on in this talk?

I'm going to start with a little bit of the history on how GitHub became one of the world's largest Ruby on Rails monoliths. GitHub has been around since 2008. How did a 12+ year old system grow into its current state? What are some telltale signs that indicated a monolithic codebase is no longer the most efficient solution for GitHub?

For example, in the past 18 months, we've grown tremendously as a company. We went from just shy of a thousand employees to now over 2,000. We've grown organically and inorganically, both from hiring as well as acquisitions. We've acquired Semmle, npm, Gitalytics, Dependabot and Pull Panda. And even internally within Microsoft, now that GitHub is owned by Microsoft, we've pulled over some of our sister teams to help with the development of GitHub products such as Actions CI/CD. 

Given all of the different cultures, norms, and technology stacks that each of the teams brings to the table, we needed to fundamentally rethink how we do software development at GitHub. We can’t just have over a thousand engineers working in the same monolithic code base, and learn Ruby on Rails before they can become productive. We had to find solutions to enable development outside of the monolith and to support all these different teams and systems and technologies working harmoniously together. That's the beginning of the talk, setting the context for how we got there.

From that point, do you talk about operating microservices or decomposing microservices? Where do you go from there?

We go through factors such as scale and our exponential growth to over 50 million developers and over 100 million repos on our platform. From that point, we talk about the need for breaking up data, and ensuring modularity across services. We also focus on the role of core services, such as authentication and authorization, in a microservices transition. For example, people should be able to authenticate into Git systems and use git commands without the need of github.com being up and running. Currently, as part of a monolithic architecture, github.com physically has to be up and running for you to do any git operations. This is why it’s so important for some of these fundamental pieces to be extracted, to allow primary functions to still happen without having to be tied into the monolith. 

There’s also a tremendous amount of gravity pull in a large monolithic codebase. People tend to say, hey, I can get this feature up and running in two weeks if I just built it in the monolith. How do we start to peel away some of those tools and functionalities that make working in the monolith so attractive, so that developers can start saying, hey, you know what? Now that these core primitives and building blocks exist outside of the monolith, it will be easier for me to do development outside of it.

Who is the persona you are talking to?

I'm talking to people who are senior architects or engineering leaders who are trying to make the decision of moving towards a services architecture. If you have a monolithic codebase today, should you continue to scale on your current path or do something completely different? As you can see from GitHub, you have quite a bit of runway to scale a monolithic architecture to support a world-class organization with billions of API calls on a daily basis. 

But some follow up questions are: When is that inflection point for starting to split up the monolith?  And how do you go about doing that split once you’ve made the decision to move forward? What are the existing best practices and other important factors you need to consider?  How should you think about the right size and the number of microservices given your current teams and potential factors such as on-call rotations?  How do you handle data read and write in a microservices environment?  

For example, now that things are not hosted on a single server, when your dependencies go down you need to think about how to handle those situations gracefully, and leverage exponential backoffs so that you don't overwhelm the system the minute it comes back online. We will try to cover topics such as designing for failure and other best practices of implementing a microservices architecture in this talk.


Speaker

Sha Ma

VP of Software Engineering @GitHub

Sha Ma is VP of Software Engineering at GitHub, where she is responsible for Core Platform and Ecosystem.  Prior to GitHub, Sha was the VP of Engineering at SendGrid, and was part of the leadership team that took the company public in 2017.  Sha cares passionately about Diversity,...

Read more

Date

Tuesday Nov 10 / 01:00PM EST (40 minutes)

Track

Operating Microservices

Add to Calendar

Add to calendar

Share

From the same track

Session

Solving Mysteries Faster with Observability

Tuesday Nov 10 / 02:40PM EST

Everyone loves a good mystery, but not when it involves operating our services. Investigating production issues in a microservice architecture can make you feel like a detective, combing through evidence and gathering clues to reconstruct the scene of a crime, all while the clock is ticking. You...

Elizabeth Carretto

Senior Software Engineer @Netflix

Session

Pitfalls and Patterns in Microservice Dependency Management

Tuesday Nov 10 / 01:50PM EST

A distributed system is one in which the failure of a computer you didn't even know existed can render your own computer unusable." -- Leslie Lamport, May 28, 1987. Building a product based on microservices has many advantages as vertical/horizontal scaling, independent release cycles...

Silvia Esparrachiari

Software Engineer @Google

Session

The Medieval Census Problem

Tuesday Nov 10 / 03:30PM EST

Delve into the mists of time and the kingdom of Andzania and their progressive queen's journey towards a census of her population. See how the concepts of distributed computing have been known to humanity for hundreds of years and how you can use the medieval census as a model to picture how...

Andy Walker

Founder at Kraken Industries

View full Schedule

Less than

15

weeks until QCon Plus May 2022

Level-up on the emerging software trends and practices you need to know about.

Deep-dive with world-class software leaders at QCon Plus (Nov 1-12, 2021).

Save your spot for $549 before February 7th

Register