GitHub Engineering

Stretching Spokes

GitHub’s Spokes system stores multiple distributed copies of Git repositories. This article discusses how we got Spokes replication to span widely separated datacenters.

Mitigating replication lag and reducing read load with freno

At GitHub, we use MySQL as the main database technology backing our services. We run classic MySQL master-replica setups, where writes go to the master, and replicas replay master’s changes asynchronously. To be able to serve our traffic we read data from the MySQL replicas. To scale our traffic we may add more replica servers. Reading from the master does not scale and we prefer to minimize master reads.

Transit and Peering: How your requests reach GitHub

GitHub is at a scale that provides exposure to interesting aspects of running a major site and are working to mature and level-up many parts of our infrastructure as we grow. One of the areas where this is evident is in how your requests find their destination using DNS and make their way into our sites over transit and peering. Many organizations are either too small to need to tackle these sorts of problems or so large they have groups to maintain existing solutions for each portion of them. It is really compelling to be able to directly work on such projects and closely with great engineers who are solving others still.

Evolution of GitHub's data centers

Over the past 18 months we’ve made a significant investment in GitHub’s physical infrastructure. The goal of this work is to improve the redundancy and global availability of our system. In doing so we’ve solidified the foundation upon which we will expand our compute and storage footprint in support of our growing user base.

GitHub Debug

GitHub is proud to handle thousands of requests per second from our millions of users. The Internet, however, can be a fickle beast of cables and sparks, and sometimes those requests don’t happen very fast (or at all). While we’re happy to help you troubleshoot connection issues to us, we also know our users like swift answers and a hands-on approach.

Older posts