Real World Architectures

Learn from how real-world systems are designed at scale

Studying real-world architectures is one of the best ways to learn system design principles. These articles showcase how major companies have solved complex scalability challenges.

How to Study These Architectures

Don’t focus on nitty gritty details for the following articles, instead:

• Identify shared principles, common technologies, and patterns within these articles
• Study what problems are solved by each component, where it works, where it doesn’t
• Review the lessons learned

Source: Twitter timelines at scale

Data Processing Systems

System	Description	Reference
MapReduce	Distributed data processing from Google	research.google.com
Spark	Distributed data processing from Databricks	slideshare.net
Storm	Distributed data processing from Twitter	slideshare.net

Data Store Systems

System	Description	Reference
Bigtable	Distributed column-oriented database from Google	harvard.edu
HBase	Open source implementation of Bigtable	slideshare.net
Cassandra	Distributed column-oriented database from Facebook	slideshare.net
DynamoDB	Document-oriented database from Amazon	harvard.edu
MongoDB	Document-oriented database	slideshare.net
Spanner	Globally-distributed database from Google	research.google.com
Memcached	Distributed memory caching system	slideshare.net
Redis	Distributed memory caching system with persistence and value types	slideshare.net

File Systems

System	Description	Reference
Google File System (GFS)	Distributed file system	research.google.com
Hadoop File System (HDFS)	Open source implementation of GFS	apache.org

Infrastructure Systems

System	Description	Reference
Chubby	Lock service for loosely-coupled distributed systems from Google	research.google.com
Dapper	Distributed systems tracing infrastructure	research.google.com
Kafka	Pub/sub message queue from LinkedIn	slideshare.net
Zookeeper	Centralized infrastructure and services enabling synchronization	slideshare.net

Company Architectures

Tech Giants

Amazon Architecture

Learn about Amazon’s distributed systems architecture

Google Architecture

Explore Google’s approach to building scalable systems

Facebook Engineering

Scaling memcached, TAO distributed data store, and photo storage

Netflix Stack

A 360 degree view of the entire Netflix stack

Social Media & Communication

Company	Key Topics	Reference
Twitter	Timelines at scale, 150M active users, 300K QPS	Making Twitter 10000% faster Timelines at scale
Instagram	14 million users, terabytes of photos	Architecture
WhatsApp	Architecture Facebook bought for $19 billion	Architecture
Pinterest	From 0 to 10s of billions of page views	Scaling Pinterest

Other Notable Architectures

Company	Key Topics	Reference
Uber	Real-time market platform, scaling to 2000 engineers	Scaling Uber
Dropbox	How Dropbox scaled their infrastructure	Video
YouTube	YouTube scalability and architecture	Scalability
Stack Overflow	Stack Overflow architecture	Architecture
Salesforce	Handling 1.3 billion transactions a day	Architecture

Specialized Systems

Company	Specialty	Reference
ESPN	Operating at 100,000 duh nuh nuhs per second	Architecture
Flickr	Photo sharing architecture	Architecture
Tumblr	15 billion page views a month	Architecture
TripAdvisor	40M visitors, 200M dynamic page views, 30TB data	Architecture
Cinchcast	Producing 1,500 hours of audio every day	Architecture
DataSift	Realtime datamining at 120,000 tweets per second	Architecture