System Design Notes

A complete, doubt-free journey from absolute basics to advanced distributed systems. Explained in simple, clear language with real-world examples.

107Topics
14Parts
0Doubts left

#1What is System Design?

One-line definition

System Design is the process of deciding how to build a software application so that it works correctly, stays fast, never breaks, and keeps working even when millions of people use it at the same time.

Let us understand this slowly, because this single idea is the foundation of everything we will learn.

When you build a very small program (like a calculator that runs only on your laptop), you don't need to think much. Only one person uses it. There is no internet, no millions of users, no money lost if it crashes.

But real apps like YouTube, Instagram, Swiggy, Uber, Google Search, Facebook, or even AI tools like Cursor and Claude are used by crores of people at the same time, from all over the world, on slow and fast internet, day and night. If such an app becomes slow or goes down even for 5 minutes, the company can lose crores of rupees and millions of users get angry.

System Design is the planning and decision-making that makes such big apps possible. It answers questions like:

  • How do we handle 1 crore users at the same time without the app crashing?
  • How do we make sure a video on YouTube starts playing in 1-2 seconds, not 30 seconds?
  • Where do we store billions of photos (Instagram) or messages (WhatsApp)?
  • What happens if one computer (server) suddenly dies? Does the whole app go down, or does it keep running?
  • How do we make the app fast for a user in India AND a user in America at the same time?

Real-life Analogy: Building a Hut vs Building a City

Think of it like construction

Building a small program is like building a single hut — one person can do it in a day, no planning needed. But building a large app like Swiggy is like building an entire city for lakhs of people.

A city needs careful design before construction:

  • Roads so traffic does not jam (in apps: how data travels).
  • Water & electricity supply that never stops (in apps: databases and servers that stay up).
  • Multiple hospitals so if one is full, people go to another (in apps: backup servers).
  • Traffic police to direct cars to less crowded roads (in apps: load balancers).
  • Local shops in every area so people don't travel far (in apps: caching and CDNs).

If you build a city without planning, it collapses the moment too many people arrive. The same happens to apps built without system design — they crash the moment they become popular. System Design is the city-planning of software.

A Concrete Example: What Happens When You Press "Play" on YouTube?

This looks simple to you — you tap a video, it plays. But behind that one tap, a huge designed system works in milliseconds:

Youphone / laptop
→
InternetDNS finds YouTube
→
Load Balancerpicks a free server
→
Servershandle your request
↓
CDNvideo stored near you
→
Cachefast temporary memory
→
Databasevideo info, likes, comments
One tap on "Play" triggers this entire designed journey — and it finishes in about 1-2 seconds.

Every box in that picture is a design decision. Why a load balancer? Why a CDN? Why a cache? Each one solves a real problem (speed, crowd, failure). We will learn every single box in detail in the coming topics. For now, just understand: system design = choosing and connecting these boxes the right way.

Real Apps and the Hard Problems They Solve

Every famous app you use is a masterpiece of system design. Here is what makes each one hard to build:

AppThe Hard System Design Problem
Google SearchSearch the entire internet (billions of pages) and return answers in under half a second, for billions of searches every day.
YouTubeStore and stream billions of videos. Play instantly anywhere in the world without buffering, on slow and fast internet.
Instagram / FacebookStore billions of photos and posts. Build a personalised feed for each of 2+ billion users in real time.
Swiggy / ZomatoMatch lakhs of live orders with restaurants and delivery partners, track them on a live map, and update prices and ETAs every second.
Uber / OlaFind the nearest free driver out of millions in real time, calculate routes and fares, and handle surge pricing instantly.
WhatsAppDeliver 100+ billion messages a day instantly, in order, even if the receiver's phone is switched off.
Cursor / Claude (AI tools)Send your request to massive AI models running on expensive GPUs, stream the answer back word-by-word, and serve millions of users without huge delays or huge cost.

Notice the common themes in every row: huge scale (many users), speed (fast response), reliability (never goes down), and cost (do it without wasting money). These four goals are the heart of system design.

Why Should YOU Care About System Design?

  • For interviews: Big companies (Google, Amazon, Flipkart, startups) have a dedicated "System Design Interview" round for any developer with even 1-2 years of experience. You cannot get a good job at scale without it.
  • For real work: When your app grows from 100 users to 10 lakh users, only system design knowledge will save it from crashing.
  • For becoming a senior engineer: Writing code is a junior skill. Deciding how the whole system fits together is what makes you a senior engineer or architect (and gets you paid more).

Two Levels of System Design (Quick Preview)

High-Level Design (HLD)

The big picture. Which major pieces exist and how they connect — servers, databases, load balancers, caches.

Like an architect's map of a whole city.

Low-Level Design (LLD)

The fine details inside one piece — classes, functions, database tables, exact logic.

Like the detailed blueprint of one single building.

Don't worry

We have a full dedicated topic (#2) on HLD vs LLD next. For now, just know that "System Design" usually means the big-picture HLD, which is what interviews focus on most.

Common Misconceptions (Cleared Right Now)

Myth 1: "System Design is only about coding."

Reality: System design is mostly about thinking and deciding, not writing code. It is about trade-offs — choosing option A over option B and explaining why. You draw boxes and arrows, not always code.

Myth 2: "There is one correct answer."

Reality: There is no single right answer in system design. Every choice has pros and cons. A good engineer picks a reasonable option and clearly explains the trade-offs. We will learn to think this way.

Myth 3: "It is too advanced for beginners."

Reality: If you understand simple analogies (city, restaurant, post office), you can understand system design. We start from absolute zero and build up. No step is skipped.

Interview Tips

  • Interviewers don't expect a perfect answer. They want to see how you think and whether you discuss trade-offs.
  • Always ask clarifying questions first ("How many users? What features?") before designing. This shows maturity.
  • Start simple, then improve. First design for 100 users, then scale up to 1 crore. This step-by-step approach impresses interviewers.
  • A very common opening question is literally: "What do you understand by system design?" — now you can answer it confidently.

Key Takeaways

  • System Design = planning how to build software that is fast, reliable, and works for millions of users at once.
  • It is like city-planning for software — without it, popular apps crash.
  • The four core goals are: Scale, Speed, Reliability, and Cost.
  • Every big app (YouTube, Swiggy, Uber, Instagram, Cursor, Claude) is solving these same goals in its own way.
  • There is no single correct answer — it is all about understanding and explaining trade-offs.
  • HLD = big picture; LLD = fine internal details (full topic coming next).

#2HLD vs LLD vs System Design

One-line definition

System Design is the full work of planning a software system before and while building it. It has two zoom levels: HLD (High-Level Design) is the bird's-eye "big picture", and LLD (Low-Level Design) is the close-up "inside details" of each piece.

These three words confuse almost every beginner. People think HLD, LLD, and System Design are three separate subjects you study from three different books. That is wrong. Let us remove this confusion forever with one simple idea:

The core idea (remember this one line)

HLD and LLD are not different things. They are the same design seen at two different zoom levels, and "System Design" is the name for the whole thing together.

Think about Google Maps. It is one single map, but you can zoom it:

  • Zoom all the way out → you see the whole country: states, big highways connecting cities. You do not see individual houses. → This view is HLD.
  • Zoom all the way in → you see one single street, one house, even the front gate. But now you cannot see the whole country. → This view is LLD.
  • The complete map (which lets you do both) → that is System Design.

It is the same map the whole time. You just changed how close you are looking. HLD and LLD work exactly the same way on a software system.

Interactive Demo — Zoom between HLD and LLD

This is the same system. Click the buttons to zoom out (see the big picture) or zoom in (see what is inside one box). Watch how the view changes — this is exactly the difference between HLD and LLD.

Mobile Appclient
→
Load Balancer
→
Backend Serverclick "Zoom In"
→
Database
HLD view: only the major boxes and how they connect. We see that a Backend Server exists, but not what is inside it.

Real-life Analogy: Building a House

Architect vs Electrician

When you build a house, two completely different kinds of drawings are made by two different experts:

1. The Architect's plan (HLD): Shows the big picture — where the bedrooms, kitchen, bathroom, and stairs go, and how the rooms connect. It does NOT show where each individual electrical wire or water pipe goes.

2. The Electrician's & Plumber's blueprint (LLD): Zooms into one room and shows the exact wiring path, every switch position, every pipe joint, and exact measurements in centimetres.

You need both to build the house. The architect's plan alone is not enough to actually wire the house; the wiring blueprint alone makes no sense without the room layout. Both drawings together = the complete plan = System Design.

High-Level Design (HLD) — The Big Picture

HLD shows the major building blocks of the system and how they talk to each other. It answers the question "What are the main pieces, and how do they connect?" — not "How is each piece coded inside?".

When you do HLD, you are making decisions like these:

  • How many servers do we need, and in which regions of the world?
  • Which type of database (SQL or NoSQL), and how many copies of it?
  • Do we need a load balancer? A cache? A CDN? A message queue?
  • How does data flow from the user, through the system, to the database, and back?
  • How will the system grow (scale) when users increase, and how will it survive if one server dies?

Notice that none of these questions mention class names or function logic. HLD stays at the level of boxes and arrows.

Animated Demo — Follow one request, step by step

Press Play. Each box will light up one by one as your request reaches it, and the line below explains in plain words what is happening at that moment. This "boxes, arrows, and flow" thinking is exactly what HLD is about. Use Step to move manually at your own pace.

Appyour phone
→
Load Balancertraffic police
→
Serverbusiness logic
→
Cachefast memory
→
Databasereal data
Press  Play  to send a request through the system…

What an HLD document usually contains

When a senior engineer writes an HLD, it normally includes these things. Do not worry if some words are new — we have a full topic on each one later. For now, just see the kind of decisions HLD covers:

  • Architecture diagram — the boxes-and-arrows picture of the whole system.
  • Major components — load balancers, servers, databases, caches, queues, CDNs.
  • Data flow — how a request travels in, and how the response comes back.
  • Database choice — SQL or NoSQL, and roughly what data is stored where.
  • Scaling plan — how the system handles more users (add more servers, etc.).
  • Reliability plan — what happens if a server or database fails.
  • APIs between services — which service talks to which, at a high level.

Low-Level Design (LLD) — The Inside Details

LLD zooms into one single box from the HLD (for example, the "Backend Server" box) and designs exactly how it works inside, in real code. It answers the question "How is this one piece actually built?".

When you do LLD, you are making decisions like these:

  • Which classes and functions to create, and how they call each other.
  • The exact database tables: their columns, data types, and relationships.
  • The precise step-by-step logic inside each feature.
  • Which design patterns to use (Factory, Observer, Strategy, etc.).
  • How to keep the code clean, reusable, and easy to test.
Example: A "User" in LLD

For the same app, the HLD just said "we need a database". The LLD goes much deeper and defines the exact table:

TABLE users (
  user_id      BIGINT       PRIMARY KEY,   -- unique id for each user
  name         VARCHAR(100),               -- user's full name
  email        VARCHAR(150) UNIQUE,        -- no two users share an email
  created_at   TIMESTAMP                   -- when the account was made
)

This level of exact detail — column names, data types, and keys — is LLD. The HLD did not care about any of this; it only cared that a database exists.

How HLD and LLD Work Together (The Real Workflow)

In a real company, the two are done in order. You almost always go HLD first, then LLD:

1. HLDdraw the big boxes
→
2. Pick a boxe.g. "Order Service"
→
3. LLDdesign its classes & tables
→
4. Code itdevelopers build it
First the architect draws the city (HLD). Then, building by building, the detailed blueprints are made (LLD). Then construction (coding) begins.

Why Split the Work into Two Levels at All?

A fair question: why not just start coding directly? Why waste time on HLD and LLD? Here is why every serious team does it:

  • You catch expensive mistakes early. Fixing a wrong decision on paper takes 5 minutes. Fixing it after writing 50,000 lines of code takes months. HLD lets you find big problems before they cost money.
  • Many people can work together. Once the HLD shows 5 boxes, 5 different teams can each take one box and design its LLD in parallel. Without a shared big picture, everyone would clash.
  • It separates two kinds of thinking. HLD needs "will this scale and survive failures?" thinking. LLD needs "is this code clean and correct?" thinking. Mixing both at once overwhelms the brain. Splitting them keeps each step clear.
  • It becomes documentation. New engineers who join later can read the HLD to understand the whole system in 10 minutes, instead of reading lakhs of lines of code.
Simple way to remember

HLD answers "Will this system work at scale?" — LLD answers "Is this piece of code correct and clean?" Two different questions, so two different levels.

What Kind of Diagrams Are Used at Each Level?

This is a detail many beginners miss, and it instantly makes things clearer. HLD and LLD literally use different types of diagrams:

HLD diagrams

  • Block / architecture diagrams (boxes and arrows).
  • Data flow diagrams (how data moves around).
  • Deployment diagrams (which server sits where).

LLD diagrams

  • Class diagrams (classes and how they relate — UML).
  • ER diagrams (database tables and their links).
  • Sequence diagrams (exact order of function calls).

HLD vs LLD: Side-by-Side Comparison

PointHigh-Level Design (HLD)Low-Level Design (LLD)
Zoom levelBird's-eye / big pictureClose-up / fine detail
Question it answersWhat are the parts & how connectedHow is each part built inside
ShowsServers, DBs, load balancers, cachesClasses, functions, tables, logic
Level of detailBoxes and arrowsActual code structure
AudienceArchitects, team leads, managersDevelopers who write the code
Usually made bySenior engineers / architectsDevelopers
Also calledMacro / architecture designMicro / detailed design
Interview round name"System Design" round (scalability)"LLD / OOP / Machine Coding" round

So Where Does "System Design" Fit?

System Design is the umbrella term that covers both HLD and LLD together. But here is the important real-world fact: in everyday talk and in most interviews, when people say "System Design", they usually mean HLD (the big-picture, scalability-focused design), because that is the harder and more discussed part.

System Designthe full umbrella
↓
HLDbig picture
LLDinside details
System Design = HLD + LLD together. But "System Design interview" almost always means HLD.

Real Example: Designing Instagram

Let us take one real app and clearly split which decisions are HLD and which are LLD. Click the tabs to switch.

  • Use a CDN so photos load fast for users in every country.
  • Use object storage (like Amazon S3) to hold billions of images.
  • Use a cache for the home feed so it appears instantly.
  • Split the app into separate services: feed, upload, notifications, search.
  • Use a message queue so notifications are sent in the background.

See the pattern? The HLD tab never mentions a class or a table. The LLD tab never mentions a CDN or scaling. Each lives at its own zoom level — but both describe the same Instagram.

Full Walkthrough: "Place Order" on Swiggy

Let us trace one single feature — tapping "Place Order" on Swiggy — through both levels, so you see exactly where HLD ends and LLD begins.

Step 1 — The HLD (big picture)

At the HLD level, we only decide the major pieces involved and how the request flows. We do not write any code yet:

Swiggy App
→
API Gateway
→
Order Service
→
Database
↓
Payment Service
Notification Servicetells the restaurant
HLD: which services handle "Place Order" and how they connect. Still just boxes and arrows.

Step 2 — The LLD (inside the Order Service)

Now we zoom into the single Order Service box and design its actual code and data:

// The classes inside the Order Service
class OrderController {
    placeOrder(request) { ... }     // entry point for the web request
}
class OrderService {
    validateCart(cart) { ... }      // are items still available?
    createOrder(cart, user) { ... } // build the order object
}
class OrderRepository {
    save(order) { ... }             // write the order to the database
}

// The exact database table it writes to
TABLE orders (
    order_id     BIGINT      PRIMARY KEY,
    user_id      BIGINT,
    restaurant_id BIGINT,
    total_amount DECIMAL,
    status       VARCHAR(20),   -- PLACED, COOKING, DELIVERED
    created_at   TIMESTAMP
)

Notice the clean split: the HLD said "there is an Order Service". The LLD said "here are the exact classes and the exact table inside it". Same feature, two zoom levels.

Who Works on What in a Real Team?

RoleMostly works onTypical question they ask
Architect / Staff EngineerHLD"Will this handle 10x more users next year?"
Senior EngineerHLD + reviews LLD"Is this the right database for this data?"
Software DeveloperLLD + writing code"How should these classes be structured?"
The whole teamBoth, together"Does our detailed design match the big picture?"

Common Misconceptions (Cleared)

Myth 1: "HLD and LLD are completely separate skills."

Reality: They are two zoom levels of the same design work. A great engineer moves smoothly between them — first the big picture, then the details.

Myth 2: "LLD is only for juniors, HLD is only for seniors."

Reality: Both matter at every level. Juniors should understand HLD too, and seniors must still get LLD right. A bad LLD can ruin a perfect HLD — just like bad wiring can ruin a beautifully designed house.

Myth 3: "You must finish all HLD before any LLD."

Reality: In real projects it is iterative. You may do HLD, start LLD on one service, learn something, and go back to adjust the HLD. It is a back-and-forth, not a one-way street.

Interview Tips

  • If the round is called "System Design" or "Design Twitter / Uber / Netflix", they almost always mean HLD — focus on scale, databases, caching, and load balancing.
  • If the round is called "LLD", "Machine Coding", or "OOP Design" (e.g. "Design a parking lot" or "Design a chess game"), they want classes, objects, and clean code.
  • In an HLD interview, start with the big boxes first. Do NOT jump into table columns early — that is LLD and will waste your limited time.
  • If you are unsure which type the round is, just ask the interviewer: "Should I focus on the high-level architecture or the detailed class design?" This is a smart, professional question.

Key Takeaways

  • System Design = HLD + LLD. They are zoom levels of the same thing, not three separate subjects.
  • HLD = big picture (servers, DBs, caches, and how they connect). Answers "what & how connected".
  • LLD = inside details of one box (classes, tables, logic). Answers "how built in code".
  • Real workflow: HLD first, then LLD, then coding — and it is iterative (you go back and forth).
  • When people say "System Design interview", they almost always mean HLD.
  • This whole course mostly teaches HLD, which is what scaling and interviews need most.

#3Functional vs Non-Functional Requirements

One-line definition

Functional Requirements describe what the system should do (its features and actions). Non-Functional Requirements describe how well the system should do it (its speed, reliability, security, and overall quality).

Before you design or build anything, you must first know what you are being asked to build. That list of "what is needed" is called the requirements. Every requirement falls into one of two buckets, and understanding the difference is the true starting point of all system design.

Let us learn both with one simple sentence to keep in mind:

The one-line trick to never get confused

Functional = the "WHAT" (what features the system has).
Non-Functional = the "HOW WELL" (how fast, how safe, how reliable those features are).

Real-life Analogy: Ordering Food at a Restaurant

The menu vs the experience

Imagine you walk into a restaurant.

Functional requirements are the things on the menu — what the restaurant can do for you: serve pizza, serve cold drinks, take your order, accept card payment. These are the actions and features.

Non-functional requirements are about the quality of the experience — the food arrives in 10 minutes (fast), the plates are clean (safe), the restaurant is open 24/7 (always available), and it can seat 200 people at once (handles load).

A restaurant can have a great menu (functional) but still be terrible if the food takes 2 hours and the kitchen is dirty (bad non-functional). Both matter.

Functional Requirements — The "WHAT"

A functional requirement is a specific action, feature, or behaviour that the system must provide to its users. In other words, it answers the question: "What should this system be able to do?" Every button you can click, every action you can take, and every result the system gives back is described by a functional requirement.

Think of functional requirements as a promise list. Each item is a promise the software makes to the user: "If you do this, I will do that." For example, WhatsApp promises: "If you type a message and press send, I will deliver it to your friend." That single promise is one functional requirement.

The simple test to spot a functional requirement

Whenever you are unsure, try to fit the requirement into this sentence: "The system should allow the user to ______" or "The system should be able to ______." If the sentence makes sense, it is almost certainly functional. For example, "The system should allow the user to reset their password" fits perfectly — so it is functional. But "The system should allow the user to load pages in 2 seconds" sounds wrong — because speed is a quality, not an action. That tells you it is non-functional instead.

Functional requirements usually fall into three groups

To make sure you never miss any, it helps to think of them in three buckets:

  • User actions — things the user directly does, like "post a photo", "search for a product", or "send a message". These are the most obvious ones.
  • System actions — things the system does on its own, often in the background, like "send an OTP email", "automatically log out after 30 minutes", or "calculate the delivery fee".
  • Data handling — what data the system stores and manages, like "store the user's order history" or "keep a record of all transactions".

Examples across real apps (and why each is functional)

Below, notice that every single one describes an action or behaviour — something the system does. That is the signature of a functional requirement:

  • Instagram: "A user can upload a photo" is an action the user performs. "A user can like a post" and "follow another user" are also clear actions. Each one is a feature the app must build.
  • Swiggy: "A user can search for restaurants", "add items to a cart", and "pay for an order" — each is a concrete step in the ordering journey, so each is functional.
  • WhatsApp: "Send a text message", "create a group", "make a voice call" — these are the core things the app lets you do.
  • YouTube: "Upload a video", "search videos", "subscribe to a channel" — again, all clear actions and features.
Key insight: functional requirements define the "scope"

The list of functional requirements decides how big the project is — its scope. If you add "video calling" to WhatsApp's list, you have just added months of work. This is why, in interviews, you first agree on a small core set of functional requirements, so you do not try to design an impossibly large system in 45 minutes.

Non-Functional Requirements — The "HOW WELL"

A non-functional requirement does not add any new feature. Instead, it describes a quality — how good, how fast, how safe, or how reliable the existing features must be. If functional requirements are what the system does, non-functional requirements are the standard to which it must do it.

Here is the crucial point that beginners miss: two apps can have the exact same features (functional) but feel completely different because of their non-functional quality. Imagine two messaging apps that both let you send messages. In one, messages arrive instantly and never get lost. In the other, messages take 10 seconds and sometimes vanish. Same feature, totally different quality. The difference is entirely non-functional — and it is what decides whether users love or hate the app.

These qualities are often nicknamed the "-ilities" because so many of them end in "-ility" (scalability, availability, reliability, and so on). Let us understand each one properly, because these are the true subject matter of system design.

1. Performance / Latency — "How fast does it respond?"

Latency is the time between you doing something and the system responding. When you tap a video and it starts in 1 second, that 1 second is the latency. Good performance means low latency — the system feels instant. A real target might be: "the search results page must appear in under 200 milliseconds." Users are extremely impatient: studies show that even a 1-second delay can make people abandon a website. So performance directly affects whether a business makes money.

2. Scalability — "Can it grow without breaking?"

Scalability is the system's ability to keep working well as the load increases — more users, more data, more requests. A system that runs perfectly for 1,000 users but crashes at 1,00,000 users is not scalable. Think of a small tea stall: it serves 10 customers easily, but if 1,000 people arrive at once, it collapses. A scalable design is one you can grow (by adding more servers) to handle the crowd. This is one of the most discussed topics in all of system design, and we have entire chapters on it later.

3. Availability — "Is it always up and running?"

Availability is the percentage of time the system is working and reachable. We measure it in "nines": 99.9% availability means the system can be down for about 8.7 hours a year, while 99.99% means only about 52 minutes a year. For something like a bank or a hospital system, even a few minutes of downtime is unacceptable. High availability usually means having backup servers ready, so if one fails, another instantly takes over and users never notice.

4. Reliability — "Does it work correctly every time?"

Reliability means the system gives the correct result and does not lose data, even when things go wrong. Availability says "the system is up"; reliability says "and it is behaving correctly." For example, when you transfer money, reliability guarantees that the money is never lost or duplicated — even if a server crashes halfway through. An app can be available (online) but unreliable (giving wrong balances). You need both.

5. Security — "Is user data safe?"

Security is about protecting the system and its data from attackers, leaks, and misuse. This includes making sure only the right people can access data (authentication and authorization), keeping data encrypted so hackers cannot read it even if they steal it, and defending against attacks. For example, WhatsApp uses end-to-end encryption so that not even WhatsApp itself can read your private messages. We will cover the full security topics later in the course.

6. Maintainability — "Is it easy to fix and improve later?"

Software is never "finished" — engineers constantly fix bugs and add features. Maintainability is how easy and cheap it is to do that. A well-designed, clean, modular system can be updated quickly by any engineer. A messy one becomes a nightmare where changing one thing breaks five others. Good maintainability saves enormous time and money over the years a product lives.

7. Consistency — "Do all users see the same correct data?"

In big systems, data is copied across many servers around the world. Consistency means that everyone sees the same, up-to-date value. For example, if you change your profile photo, consistency decides whether your friend in another country sees the new photo immediately or a few seconds later. Sometimes systems deliberately trade a little consistency to gain more speed and availability — a famous trade-off we will study in depth in the CAP Theorem topic.

Here is a quick reference table summarising all seven, so you can revise them at a glance:

QualityThe question it answersExample target
PerformanceHow fast does it respond?Page loads in under 2 seconds
ScalabilityCan it grow without breaking?Handle 1 crore users
AvailabilityIs it always up?Up 99.99% of the time
ReliabilityIs it always correct?Never lose a payment
SecurityIs data safe?All data encrypted
MaintainabilityEasy to change later?Add a feature in days, not months
ConsistencySame data everywhere?All users see the latest value
Why these matter so much

In a system design interview, almost all the hard and interesting work is about non-functional requirements. Anyone can say "users can send messages" (functional — one line). The real challenge that takes real engineering skill is "send 100 billion messages a day, instantly, without ever losing one, securely, even if servers fail" (that is performance + scalability + reliability + security + availability, all at once). This is exactly where system design lives, and why we will spend most of this course here.

Interactive Quiz — Can you classify them?

Read each requirement and decide: is it Functional (a feature/action) or Non-Functional (a quality)? Click your answer to get instant feedback and an explanation. This is exactly the skill interviewers test.

Functional vs Non-Functional: Side-by-Side

PointFunctionalNon-Functional
Answers the questionWhat does it do?How well does it do it?
DescribesFeatures and actionsQualities and constraints
Example"User can send a message""Message is delivered in under 1 second"
If missingThe feature simply does not existThe feature exists but works badly
Easy to test?Yes (does it work: yes/no)Harder (needs measuring: how fast, how many)
Interview focusListed quickly at the startDiscussed deeply (the main event)

Real Example: Requirements for WhatsApp

Let us split a real app's requirements into the two buckets. Click the tabs.

  • A user can send and receive text messages.
  • A user can send photos, videos, and voice notes.
  • A user can create a group chat.
  • A user can see "last seen" and "online" status.
  • A user can make voice and video calls.

How to Gather Requirements in an Interview

When an interviewer says "Design Instagram", they are deliberately being vague. There is no way to design a system if you do not know exactly what it must do and how well it must do it. So the interviewer is secretly testing whether you have the maturity to ask questions first instead of blindly jumping into a solution. Gathering requirements is therefore the very first phase of any design, and it usually goes like this:

Step 1 — Pin down the functional scope. Start by agreeing on which features you will actually design. Real apps have hundreds of features, but you only have 45 minutes, so you must narrow it down. You might ask: "Which features should I focus on — just posting photos and viewing the feed, or also stories, reels, and direct messages?" This prevents you from accidentally trying to build the entire app and running out of time.

Step 2 — Understand the scale. Scale completely changes the design. A system for 1,000 users is simple; a system for 100 crore users needs caching, sharding, and load balancing. So ask about numbers: "Roughly how many total users? How many are active daily? How many photos are uploaded per second?" These numbers drive almost every later decision, which is why we have a whole topic on "back-of-the-envelope calculations" near the end.

Step 3 — Clarify the quality priorities. You usually cannot make everything perfect at once, so find out what matters most. For example: "Is it more important that the feed is always available, or always perfectly up to date?" The answer tells you which trade-offs to make. Knowing the top 2-3 non-functional priorities lets you design intelligently instead of trying to optimise everything.

Step 4 — Write it all down as two clear lists. Summarise everything into a Functional list and a Non-Functional list, and say them out loud to confirm with the interviewer. This shows structure, makes sure you both agree, and gives you a checklist to design against for the rest of the interview.

Critical interview habit

Never start drawing boxes before clarifying requirements. Jumping straight into a solution without asking questions is the number one reason candidates fail system design interviews.

Common Misconceptions (Cleared)

Myth 1: "Non-functional requirements are less important."

Reality: The word "non-functional" makes them sound optional, but they are usually the hardest and most important part. A slow, insecure, or crashing app fails no matter how many features it has.

Myth 2: "Requirements are fixed and never change."

Reality: Requirements often change as a product grows. Good design leaves room to adjust. That is exactly why maintainability is itself a non-functional requirement.

Myth 3: "I should list every possible requirement."

Reality: Especially in an interview, focus on the core few functional features and the key non-functional qualities. Trying to list everything wastes time and shows poor prioritisation.

Interview Tips

  • Always begin a system design answer by stating the functional and non-functional requirements out loud. It shows structure and maturity.
  • For non-functional requirements, try to put numbers on them: "under 200 milliseconds", "99.99% available", "10 million daily users". Numbers impress interviewers far more than vague words.
  • Pick the 2-3 most important non-functional requirements and design around them. You cannot perfectly optimise everything at once — choosing priorities shows real engineering judgement.
  • A classic question: "What is the difference between functional and non-functional requirements?" — you can now answer it in one clean sentence.

Key Takeaways

  • Functional = WHAT the system does (features/actions). Non-Functional = HOW WELL it does it (qualities).
  • Functional test: "The system should let the user ___". If it fits, it is functional.
  • Key non-functional qualities: performance, scalability, availability, reliability, security, maintainability, consistency.
  • System design interviews focus mostly on the non-functional side — that is where the hard problems are.
  • Always clarify requirements first (ask questions, put numbers on them) before designing anything.

#4Client-Server Architecture

One-line definition

Client-Server Architecture is a way of building software where one computer (the client) asks for something, and another computer (the server) provides it. The client sends a request, the server sends back a response. Almost every app and website you use works this way.

This is the most fundamental pattern in all of system design. Before we can talk about load balancers, databases, or scaling, we must deeply understand this one simple relationship: someone asks, and someone answers. Let us build that understanding from absolute zero.

What Problem Does This Solve?

Imagine you want to watch a YouTube video. The video file is huge — maybe hundreds of megabytes. It would be impossible to store every video in the world on your phone. So instead, the videos live on YouTube's powerful computers, and your phone simply asks for the one video you want, when you want it. Your phone is the client; YouTube's computer is the server.

This split exists because of a simple truth: the device in your hand is small and limited, but the data and computing power needed are enormous. So we keep the heavy stuff (data, processing) on big shared computers (servers) and let many small devices (clients) request only what they need. This is the core idea behind client-server architecture.

Real-life Analogy: A Restaurant

You, the waiter, and the kitchen

Think of eating at a restaurant:

You are the client. You decide what you want and place an order. You do not go into the kitchen or cook anything yourself — you just ask.

The kitchen is the server. It has all the ingredients, the chefs, and the equipment (the "resources"). It receives your order, does the hard work of cooking, and produces the dish.

The waiter and the path between tables and kitchen is the network. The waiter carries your request to the kitchen and carries the finished food back to you.

Notice: many customers (clients) can be served by one kitchen (server). You do not need your own kitchen at home for every meal — you share the restaurant's. That sharing is exactly why client-server is so powerful and efficient.

Who Is the "Client"?

The client is the device or program that starts the conversation by asking for something. It is usually the thing in front of the user. Common examples of clients include:

  • A web browser (Chrome, Safari) when you open a website.
  • A mobile app (the Instagram or Swiggy app on your phone).
  • A desktop application (the Spotify app on your laptop).

The client's main jobs are: take input from the user, send a request to the server, wait for the response, and then display the result nicely on the screen. The client usually does not store the main data or do the heavy processing — it relies on the server for that.

Who Is the "Server"?

The server is a powerful computer (often many computers working together) that waits for requests and responds to them. It holds the important data and does the heavy work. When you hear "server", do not imagine something magical — it is just a computer, usually sitting in a large building called a data center, that is always switched on and connected to the internet, ready to answer requests 24/7.

The server's main jobs are: listen for incoming requests, process each one (which may involve reading or writing to a database, running calculations, checking permissions), and send back a response. One server can handle requests from thousands of clients, which is why this model scales so well.

The Request-Response Cycle (The Heart of It All)

The entire interaction is built on one repeating loop called the request-response cycle. It always follows the same four steps:

  1. The client sends a request. For example, your browser sends "Please give me the home page of youtube.com."
  2. The request travels over the network (the internet) to reach the correct server.
  3. The server processes the request. It figures out what is being asked, fetches the needed data (maybe from a database), and prepares an answer.
  4. The server sends back a response, which travels back over the network to the client, and the client displays it.

One very important detail: in classic client-server, the client always speaks first. The server never randomly sends data to a client out of nowhere — it only answers when asked. (There are special technologies like WebSockets that change this, and we will study them later, but the basic model is "client asks, server answers.")

Interactive Demo — Send a real request

Press "Send a Request" and watch the full cycle happen: the request packet travels from the client to the server, the server processes it, and the response travels back. Read the live log below to follow each step in plain words.

๐Ÿ’ปClientyour browser
๐Ÿ–ฅ๏ธServerYouTube data center
Click the button to begin…

Full Walkthrough: What Happens When You Open youtube.com

Let us connect this to something you do every day. When you type youtube.com and press Enter, here is the client-server story behind it:

  1. Your browser (the client) needs to find YouTube's server. It first asks a special "phone book of the internet" called DNS to convert youtube.com into a numerical address (an IP address). We cover DNS fully in a later topic.
  2. Now knowing the address, the client sends an HTTP request to YouTube's server: "Give me the home page."
  3. YouTube's server processes the request — it checks who you are, fetches video recommendations from its database, and builds a personalised page.
  4. The server sends back an HTTP response containing the page (HTML, images, video links).
  5. Your browser receives this and renders the YouTube home page you see. Every thumbnail you then click starts a brand new request-response cycle.
Connecting the dots

Notice how many future topics already appear here: DNS, IP addresses, HTTP, databases. Client-server is the skeleton; everything else in this course hangs off it. That is why we learn it first.

Client-Server vs Peer-to-Peer (P2P)

There is another, different model worth knowing so you appreciate the contrast. In Peer-to-Peer (P2P), there is no central server — every computer is both a client and a server at the same time, sharing directly with each other. Old file-sharing apps like BitTorrent and some crypto networks work this way.

PointClient-ServerPeer-to-Peer (P2P)
Who has the data?A central serverEvery participant shares it
ControlCentral, easy to manageNo central control
If the center failsEveryone is affectedOthers keep working
ExamplesYouTube, Instagram, banking appsBitTorrent, blockchains

For almost all modern apps (and everything in this course), the client-server model is what we use, because the central control makes data, security, and updates far easier to manage.

Strengths and Weaknesses

Strengths

  • Central control: data lives in one place, so it is easy to secure, back up, and update.
  • Sharing: one server serves many clients efficiently.
  • Lightweight clients: your phone does not need huge storage or power.
  • Easy updates: fix something on the server and every client instantly benefits.

Weaknesses

  • Single point of failure: if the one server goes down, every client is stuck.
  • Bottleneck: too many clients at once can overload a single server.
  • Cost: running powerful, always-on servers is expensive.
This weakness is the doorway to the rest of the course

The big weakness — "one server can be overloaded or can fail" — is exactly the problem that load balancers, multiple servers, caching, and replication exist to solve. Every advanced topic you will learn is, in some way, fixing a limitation of the basic client-server model.

Common Misconceptions (Cleared)

Myth 1: "A server is a special, magical machine."

Reality: A server is just an ordinary computer running software that listens for and answers requests. Any computer — even your laptop — can act as a server if it runs server software. The word "server" describes a role, not a magical device.

Myth 2: "One server means one physical computer."

Reality: Behind a single website, "the server" is often hundreds or thousands of computers working together, hidden behind one address. From the client's point of view it looks like one server, but it is usually many.

Myth 3: "The client is always a phone or browser."

Reality: A client is anything that makes a request. A server can itself be a client when it asks another server for something. The roles are about who is asking versus who is answering in a given conversation.

Interview Tips

  • Be able to clearly explain the request-response cycle in your own words — it is a common warm-up question.
  • When you draw any system diagram, the client (user) on the left and the server on the right is the standard starting point. Interviewers expect you to begin here.
  • If asked about the weakness of a single server, immediately mention "single point of failure" and "bottleneck" — then you can naturally lead into load balancing and replication.

Key Takeaways

  • Client = asks (sends a request). Server = answers (sends a response). The network carries messages between them.
  • It exists because small devices should not hold all the data and power — big shared servers do the heavy lifting.
  • The request-response cycle is the core loop: client requests → network → server processes → response travels back.
  • In classic client-server, the client always speaks first; the server only answers when asked.
  • Its main weakness (overload and single point of failure) is the reason most advanced system design topics exist.

#5IP Address, DNS & Domain Name

One-line definition

An IP Address is the unique number that identifies a computer on the internet. A Domain Name (like youtube.com) is the easy, human-friendly name for that computer. DNS is the system that translates the human-friendly domain name into the machine-friendly IP address.

In the last topic we said the client sends a request to "the server". But how does your phone actually find the right server out of the billions of computers on the internet? That is the question this topic answers. These three ideas — IP, Domain Name, and DNS — work together to make that possible.

The Big Idea: Computers Use Numbers, Humans Use Names

Here is the core problem. Computers on the internet find each other using numbers called IP addresses, such as 142.250.72.206. Numbers are perfect for machines, but terrible for humans — nobody wants to memorise a string of numbers for every website. Humans prefer names like youtube.com.

So we have a mismatch: humans type names, but computers need numbers. Something must translate between the two. That translator is DNS. This single idea explains the whole topic — everything else is just detail about how the translation works.

Real-life Analogy: The Contacts App on Your Phone

Saving "Mummy" instead of a 10-digit number

Think about your phone's contacts. To call your mother, you do not remember and dial her 10-digit number every time. Instead, you saved her as "Mummy" and just tap that name. Behind the scenes, your phone looks up "Mummy" in your contacts and finds the actual number to dial.

The internet works the same way:

  • "Mummy" (the saved name) = the Domain Name (youtube.com).
  • The phone number = the IP Address (142.250.72.206).
  • Your contacts app that does the lookup = DNS.

IP Address — The Computer's Number

Every device connected to the internet — your phone, a server, a smart TV — gets a unique IP (Internet Protocol) address. It works like a postal address: it tells data exactly where to be delivered. When the server sends back a response, it uses your IP address to know where to send it.

IPv4 and IPv6 (two versions)

There are two formats of IP address you should know:

  • IPv4: the older, most common format. It looks like 142.250.72.206 — four numbers (0 to 255) separated by dots. The problem: IPv4 allows only about 4.3 billion unique addresses, and the world has far more devices than that now. We are literally running out.
  • IPv6: the newer format, created to solve the shortage. It is much longer, like 2401:4900:1c30:8b9d::1, and provides a practically unlimited number of addresses (enough for every grain of sand on Earth to have many).

Public vs Private IP (a quick but important distinction)

A public IP is visible to the whole internet (like your home's street address that anyone can post mail to). A private IP is used only inside a local network, like your home Wi-Fi, where your phone, laptop, and TV each get a private IP (such as 192.168.1.5) that only makes sense within that home. Your router has one public IP that the whole house shares to reach the outside world.

Domain Name — The Human-Friendly Name

A domain name is the readable address you type, like www.youtube.com. It has a structure, and reading it from right to left reveals its parts:

wwwsubdomain
.
youtubedomain name
.
comTLD (top-level domain)
Read right to left: the TLD is the broadest category, then the specific name, then the subdomain.
  • TLD (Top-Level Domain): the ending, like .com, .org, .in, .gov. It is the broadest grouping.
  • Domain Name: the unique name a company registers, like youtube or google.
  • Subdomain: an optional prefix to point to a specific part, like www, mail (as in mail.google.com), or maps.

DNS — The Phone Book of the Internet

DNS (Domain Name System) is the giant, distributed system that stores the mapping between domain names and IP addresses, and answers lookup questions. When you type a domain name, DNS is what finds the matching IP so your browser can connect. It is called the "phone book of the internet" because, just like a phone book turns a name into a number, DNS turns a domain into an IP.

But DNS is not one single computer — that would be impossible to manage for the entire planet. Instead, the work is split across a hierarchy of different servers, each responsible for one part of the answer. Let us see how they cooperate.

How a DNS Lookup Actually Works (Step by Step)

When you request a domain and your computer does not already know its IP, the lookup travels through a chain of helpers. Here are the players:

  • DNS Resolver: the helper (usually run by your internet provider like Jio or Airtel) whose job is to do the searching for you and return the final answer.
  • Root Server: the top of the hierarchy. It does not know the IP, but it knows who handles each TLD (who handles .com, .in, etc.).
  • TLD Server: handles one ending, e.g. the .com server. It knows which server is the final authority for each .com domain.
  • Authoritative Server: the final source of truth for a specific domain. It actually holds the real IP address for youtube.com.

Use the demo below to watch them work together. Then read the recap underneath.

Interactive Demo — Resolve a domain into an IP

Press Play to watch youtube.com get turned into an IP address, hop by hop, through the DNS hierarchy. Use Step to go at your own pace.

Browseryou
→
ResolverISP helper
→
Roottop level
→
TLD.com server
→
Authoritativeowns youtube.com
Press  Play  to resolve youtube.com…

DNS Caching — Why It Is Not Slow Every Time

You might worry: does this whole multi-step journey happen every single time you open a website? That would be slow. The answer is no, thanks to caching. Once an IP is found, it is remembered (cached) at several levels — in your browser, in your operating system, and in the resolver — for a set period of time called the TTL (Time To Live).

So the very first lookup does the full journey, but for a while afterwards, your browser already knows the IP and connects instantly without asking anyone. This is why a website you visit often feels faster to connect to. When the TTL expires, the cache is cleared and the next lookup refreshes it — this lets websites change their IP when needed.

Common DNS Record Types (Good to Know)

DNS stores different kinds of entries called records. You do not need to master these now, but recognising them is useful:

RecordWhat it maps
A recordA domain name to an IPv4 address.
AAAA recordA domain name to an IPv6 address.
CNAMEOne domain name to another domain name (an alias).
MX recordA domain to its mail (email) server.

Common Misconceptions (Cleared)

Myth 1: "DNS stores or delivers the website."

Reality: DNS only gives you the address (the IP). It does not store the website's content at all. After DNS hands over the IP, your browser connects to that server separately to get the actual page. DNS is the phone book, not the website.

Myth 2: "An IP address never changes."

Reality: A website can change its IP address (for example, when it moves servers). DNS is exactly what makes this painless — the domain name stays the same, only the IP it points to is updated, so users notice nothing.

Myth 3: "DNS is one big central server."

Reality: DNS is a distributed hierarchy of many servers (root, TLD, authoritative) spread worldwide. No single machine holds the whole internet's phone book — that would be a massive single point of failure.

Interview Tips

  • Be ready to explain the famous question "What happens when you type a URL and press Enter?" — the DNS lookup is the first major step of that answer.
  • Mention DNS caching and TTL — it shows you understand why the internet is fast and how load is reduced.
  • Knowing that DNS is a distributed hierarchy (not one server) demonstrates that you think about reliability and avoiding single points of failure.

Key Takeaways

  • IP address = the computer's number (e.g. 142.250.72.206). Domain name = the human-friendly name (e.g. youtube.com).
  • DNS is the "phone book" that translates a domain name into an IP address.
  • A DNS lookup travels a hierarchy: Resolver → Root → TLD → Authoritative server, which finally gives the IP.
  • Caching with TTL means most lookups are instant; only the first one does the full journey.
  • DNS gives only the address — your browser then connects to that server separately to fetch the actual website.

#6HTTP / HTTPS Protocol

One-line definition

HTTP is the set of rules (a "language") that a client and a server use to talk to each other on the web — how to ask for something and how to reply. HTTPS is the exact same thing but secure: the conversation is encrypted so no one in between can read or tamper with it.

In the previous topics, the client sent a "request" and the server sent a "response". But what language do they speak? What exact format does a request take? The answer is HTTP. It is the foundation of the entire web, so let us understand it thoroughly.

First, What Is a "Protocol"?

A protocol is simply an agreed set of rules that two sides follow so they can understand each other. Humans use protocols all the time without thinking. For example, when you answer a phone call, there is an unspoken protocol: the caller waits, you say "Hello?", then they speak. If both people spoke at once or stayed silent, communication would fail.

Computers need the same kind of agreed rules, but written out very precisely. HTTP (HyperText Transfer Protocol) is the specific protocol for the web. Because both the browser and the server follow the exact same HTTP rules, a browser made by Google (Chrome) can talk perfectly to a server running software written by anyone, anywhere. That shared agreement is what makes the web universal.

Real-life Analogy: Sending a Formal Letter

A letter with a fixed format

Think of an HTTP request like posting a formal letter that must follow a fixed format:

  • The action you want ("I want to read" or "I want to submit") = the HTTP method.
  • The address on the envelope = the URL (which resource on which server).
  • Extra notes on the envelope (your name, language, "fragile") = the headers.
  • The actual contents inside = the body (the data you are sending).

The server reads your "letter", does what you asked, and posts a reply letter back — also in a fixed format. That reply is the HTTP response.

The Anatomy of an HTTP Request

Every HTTP request your browser sends has the same parts. Here is a real example of what one looks like behind the scenes when you log in:

POST /login HTTP/1.1            โ† method + path + HTTP version
Host: www.example.com           โ† which server
Content-Type: application/json  โ† what kind of data is in the body
Authorization: Bearer abc123    โ† (a header, e.g. who you are)

{                               โ† the body (the actual data)
  "email": "raj@gmail.com",
  "password": "secret123"
}
  • Method (POST): what action you want (explained below).
  • Path (/login): which resource on the server.
  • Headers: extra information about the request (who you are, what format, etc.).
  • Body: the actual data being sent (not all requests have one).

HTTP Methods — The "Verbs"

The method tells the server what kind of action you want to perform. The four most important ones map neatly to everyday actions:

MethodMeansEveryday example
GETRead / fetch dataOpening your Instagram feed
POSTCreate new dataPosting a new photo
PUT / PATCHUpdate existing dataEditing your bio
DELETERemove dataDeleting a post

We will go deeper into how these are used to build clean APIs in the REST API topic. For now, just know that the method communicates intent.

HTTP Status Codes — The Server's Reply Code

When the server replies, it always includes a 3-digit status code that summarises what happened. You have definitely seen 404 before. The first digit tells you the category:

  • 1xx — Informational (rarely seen).
  • 2xx — Success (everything worked).
  • 3xx — Redirection (go look somewhere else).
  • 4xx — Client error (you, the client, did something wrong).
  • 5xx — Server error (the server messed up).
Interactive — Tap a status code to learn it

Click any code to see exactly what it means and when you would see it.

๐Ÿ‘† Click a code above to see what it means.

An Important Property: HTTP Is "Stateless"

This is a concept that confuses many beginners, so let us be very clear. Stateless means the server does not remember anything about previous requests. Each HTTP request is treated as brand new, as if it is the first time it has ever seen you.

Imagine a shopkeeper with zero memory: every time you walk up to the counter — even if it is the tenth time in five minutes — he greets you like a total stranger and asks "Who are you?". That is HTTP. The server forgets you the moment it sends the response.

So how do websites "remember" that you are logged in? Because the client re-sends proof of identity (like a token or cookie) with every single request. The server does not remember you; you keep reminding it. We will explore exactly how this works in the topics on sessions and authentication. For now, just lock in this fact: plain HTTP has no memory.

HTTPS — The Secure Version

Plain HTTP has one giant problem: everything is sent as plain, readable text. Anyone who can intercept the traffic — a hacker on the same public Wi-Fi, your internet provider, etc. — can read it all: your passwords, your messages, your card numbers. This is obviously dangerous.

HTTPS (HTTP Secure) fixes this. The "S" stands for Secure. It is the same HTTP, but the entire conversation is encrypted using a technology called TLS (you may also hear the older name SSL). Encryption scrambles the data into meaningless gibberish that only the intended server can unscramble. Even if a hacker captures it, they see useless noise.

Interactive — See what a hacker sees

Imagine you send your password over public Wi-Fi. Toggle between HTTP and HTTPS to see what a hacker watching the network would actually capture.

What travels across the network:

email=raj@gmail.com&password=secret123
๐Ÿฆน Hacker: "Perfect, I can read your password in plain text!"

How HTTPS Works (The Simple Version)

You do not need deep cryptography yet, but here is the simple flow when your browser connects to an HTTPS website:

  1. Your browser asks the server to prove its identity. The server sends an SSL/TLS certificate — like a verified ID card issued by a trusted authority — proving it really is, say, the real youtube.com and not an imposter.
  2. Your browser checks the certificate is valid and trusted. (This is why you see a padlock icon in the address bar.)
  3. The two sides then agree on a secret key (this exchange is called the TLS handshake).
  4. From then on, all data is encrypted with that key. Only these two parties can read it.
The padlock means two things

The padlock icon in your browser confirms (1) the data is encrypted, and (2) the website's identity is verified. That is why you should never enter a password on a site without it.

HTTP vs HTTPS: Side-by-Side

PointHTTPHTTPS
SecurityPlain text, readable by anyoneEncrypted, unreadable to outsiders
Identity checkNoneVerified by a certificate
Default port80443
Address barhttp://, often "Not secure"https:// with a padlock ๐Ÿ”’
Use todayUnsafe, being phased outThe standard for all sites

A Note on HTTP Versions

HTTP has improved over the years, mainly to get faster. You do not need details now, but recognise the names: HTTP/1.1 (the old reliable one), HTTP/2 (allows many requests over one connection at once, much faster), and HTTP/3 (the newest, built for even lower latency). The rules of requests and responses stay the same; these versions just change how efficiently the data travels.

Common Misconceptions (Cleared)

Myth 1: "HTTPS makes my whole website 100% safe."

Reality: HTTPS only secures data while it travels between client and server. It does not protect against weak passwords, bugs in your code, or a hacked server. It is essential, but it is one layer of security, not all of it.

Myth 2: "The server remembers me between requests."

Reality: HTTP is stateless — the server forgets you after each response. The feeling of "being remembered" comes from the client re-sending a token or cookie with every request.

Myth 3: "HTTP and HTTPS are completely different protocols."

Reality: They are the same HTTP. HTTPS is just HTTP wrapped in an encryption layer (TLS). The request/response rules are identical underneath.

Interview Tips

  • Always remember and state that HTTP is stateless — it comes up constantly and leads naturally into discussions of sessions, cookies, and tokens.
  • Know the status code categories (2xx, 4xx, 5xx). Being able to say "that should return a 404" or "a 500 means our server crashed" sounds professional.
  • Be able to explain in one line why HTTPS matters: "it encrypts data in transit and verifies server identity."
  • If asked to design any web system, assume HTTPS everywhere — it is the modern default.

Key Takeaways

  • HTTP is the agreed language (protocol) of rules for client-server communication on the web.
  • A request has a method, path, headers, and body; a response includes a status code (2xx success, 4xx your error, 5xx server error).
  • HTTP methods: GET (read), POST (create), PUT/PATCH (update), DELETE (remove).
  • HTTP is stateless — the server remembers nothing between requests; the client re-sends identity each time.
  • HTTPS = HTTP + encryption (TLS). It scrambles data so outsiders cannot read it and verifies the server's identity (the padlock ๐Ÿ”’).

#7TCP vs UDP

One-line definition

TCP and UDP are two different methods for sending data across a network. TCP is reliable but slower — it guarantees every piece arrives, correctly and in order. UDP is fast but unreliable — it just fires the data off quickly without checking if it all arrived.

This topic sounds technical, but the idea behind it is actually very simple. Let us start with an everyday situation and build up slowly. By the end, TCP and UDP will feel obvious.

Start Here: Two Ways to Tell Your Friend Something

Imagine you want to tell your friend a 5-word sentence, one word at a time, from across a noisy room. There are two possible styles you could use:

Style 1 — The careful way

After every word, you wait for your friend to shout back "got it!". If they did not hear a word, you say it again. You even check "can you hear me?" before you start.

Result: the full sentence always reaches them, correct and complete — but it takes more time.

Style 2 — The fast way

You just say all 5 words quickly, one after another, without waiting for any reply. If your friend misses a word, oh well — you do not repeat it.

Result: super fast — but a word or two might get missed.

That is the whole idea

The careful way is TCP. The fast way is UDP. Everything else in this topic is just detail added on top of this one simple picture. Keep this in your head and you cannot get confused.

Quick context: what are these two, really?

TCP and UDP are the two main ways data actually travels across the internet (their formal name is "transport protocols" — the things that carry your data). One nice fact: the web (HTTP) uses TCP, because web pages must arrive complete. So TCP is what delivered this very page to your screen.

TCP — The Careful Way (Reliable)

TCP is the "careful friend" from our example. Its one job is to make sure every piece of data arrives, correctly, and in the right order. It does four simple things to guarantee this. You do not need to memorise the technical names — just understand the plain meaning:

What TCP doesWhat it actually means (plain words)
Connects first
(the "handshake")
Before sending anything, both sides quickly say "ready?" / "ready!" to confirm they are connected. Like saying "hello, can you hear me?" before you start talking.
Waits for "got it!"
(an "acknowledgement")
After each piece, the receiver replies "got it!". This is exactly like the double blue ticks on WhatsApp telling you the message was received.
Resends lost pieces
(a "retransmission")
If a "got it!" never comes back, TCP knows that piece was lost — so it simply sends it again. This is why nothing is ever lost.
Fixes the orderEach piece is numbered, so even if they arrive jumbled up, TCP arranges them back into the correct order before handing them over.

All these checks make TCP completely reliable — nothing is ever lost or out of order. But the checks take a little extra time. That is simply the price of being careful.

UDP — The Fast Way (Speedy)

UDP is the "fast friend". It just sends the data as quickly as possible and skips all of TCP's careful checks:

  • It does not connect first — it starts sending right away.
  • It does not wait for any "got it!" reply.
  • If a piece gets lost, UDP does not resend it — that piece is simply gone.
  • It does not bother fixing the order either.

This might sound careless, but skipping all those checks makes UDP much faster. And for some apps, that speed matters far more than catching every single piece — as you will see in the demo below.

The one question that decides which to use

Whenever you must choose, just ask yourself: "If one piece gets lost, is it worth stopping to resend it?"

• If YES (a file, a web page, a payment — every piece matters) → use TCP.
• If NO (a live video call — an old lost frame is useless by the time it could arrive) → use UDP.

Interactive Demo — The same problem, two real apps

Press Start. On the left, UDP is powering a live video call (like a WhatsApp call). On the right, TCP is powering a file download. In both, packet 3 gets lost on the way — watch how each one reacts, and why that reaction is perfect for its job.

๐ŸŽฅ UDP — powering a LIVE VIDEO CALL (video frames)
๐Ÿ“ฅ TCP — powering a FILE DOWNLOAD (file chunks)

TCP vs UDP: Side-by-Side

PointTCPUDP
ReliabilityGuaranteed deliveryNo guarantee (can drop)
OrderingAlways in orderMay arrive out of order
SpeedSlower (more checks)Faster (no checks)
ConnectionNeeds a handshake firstNo setup, sends instantly
ConfirmationYes (acknowledgements)None
Best forAccuracy mattersSpeed matters

When to Use Which?

Use TCP when accuracy is essential

If losing even one byte would break things, use TCP. Examples:

  • Web pages (HTTP/HTTPS) — a half-loaded page is useless.
  • Email — you cannot drop words from a message.
  • File downloads — a missing chunk corrupts the file.
  • Banking / payments — every bit must be correct.

Use UDP when speed beats perfection

If being fast and live matters more than catching every packet, use UDP. Examples:

  • Video / voice calls (WhatsApp, Zoom) — a tiny glitch is better than a delay.
  • Live streaming — keep the stream moving.
  • Online games — the latest position matters, not an old lost one.
  • DNS lookups — small, quick, just retry if needed.

Common Misconceptions (Cleared)

Myth 1: "UDP is broken or bad because it loses data."

Reality: UDP is not broken — it is designed to skip reliability for speed. For live video and gaming, that is a feature, not a bug. The "right" choice depends entirely on the use case.

Myth 2: "TCP is always the better, safer choice."

Reality: TCP's guarantees add delay. For real-time applications, that delay is worse than a little data loss. There is no universally "better" one — only the better fit for the job.

Myth 3: "Video calls use TCP because video is important."

Reality: Most real-time video and voice calls use UDP. Resending a lost video frame from a second ago is pointless — the call has already moved on. Staying live matters more than perfection.

Interview Tips

  • Frame your answer around the trade-off: TCP = reliability over speed, UDP = speed over reliability. Interviewers love hearing trade-off thinking.
  • Have ready examples for each: "HTTP and file transfer use TCP; video calls, gaming, and DNS use UDP."
  • Mention that HTTP runs on TCP — it connects this topic to the previous one and shows depth.
  • If asked to design something real-time (like a video call system), proposing UDP and explaining why instantly shows strong understanding.

Key Takeaways

  • TCP = reliable but slower: handshake, acknowledgements, retransmission, and ordering guarantee perfect delivery.
  • UDP = fast but unreliable: no setup, no confirmation, no resending — lost data stays lost.
  • It is a classic trade-off: reliability (TCP) versus speed (UDP).
  • Use TCP for web, email, files, payments. Use UDP for video/voice calls, live streaming, gaming, DNS.
  • HTTP rides on TCP, which is why web pages always arrive complete.

#8REST API Basics

One-line definition

An API is a way for two pieces of software to talk to each other. A REST API is the most popular style of building that conversation over the web — using clear web addresses (URLs) for things, and standard HTTP methods (GET, POST, etc.) to act on them.

This topic ties together everything from the last few chapters. The client and server (Topic 4) talk using HTTP (Topic 6). But how should we organise that talking so it is clean, predictable, and easy for any developer to use? The answer the whole industry settled on is REST. Let us build it up from the very beginning.

First, What Is an API?

API stands for Application Programming Interface. That sounds complex, but the idea is simple: it is a defined way to ask one program to do something for you, without needing to know how it works inside.

You use APIs in real life constantly. A power socket is an API: you plug in any device using the standard shape, and you get electricity. You do not need to understand the power plant. The socket is a clean, agreed "interface" between you and the electricity system. A software API is the same: a clean, agreed way for your app to request something (data, an action) from another system.

Analogy: The waiter is the API

At a restaurant, you (the client) do not walk into the kitchen (the server) and start cooking. Instead, you talk to the waiter. The waiter is the API: a defined messenger who knows how to take your order, carry it to the kitchen, and bring back the food. You only need to know how to talk to the waiter — not how the kitchen works. The menu is like the API documentation: it tells you exactly what you are allowed to ask for.

What Does "REST" Actually Mean? (Kept Simple)

Forget the scary full form ("Representational State Transfer") — honestly, the name does not matter at all. Here is all REST really is: a set of simple, tidy rules for naming things and asking for them, so that every developer builds their API the same, predictable way.

Think of it like a well-organised library. In a good library, every book has a clear address (a shelf and a number), and there are clear actions you can take: borrow, return, reserve. Because everything is tidy and consistent, anyone can walk in and use it without confusion. REST does exactly this, but for data on the web.

And the best news: REST is built on just two simple ideas. Genuinely, that is all you need to get started:

Idea 1: Name the "thing" in the URL

Every thing your app deals with (a user, a photo, an order) gets its own web address. That address is always a noun — the name of a thing.

/users = "all users".
/users/2 = "the user with id 2".

Idea 2: Say the "action" with the method

What you want to do with that thing comes from the HTTP method (from Topic 6), not from the URL.

GET = read it.
POST = create.
DELETE = remove.

Now put the two ideas together and you get a complete instruction. GET /users/2 simply reads as: "GET (read) the user with id 2." Notice how it reads almost like plain English — action + thing. That readability is the entire point of REST. An API built this way is called RESTful.

Two smaller rules, just so you have heard them

REST also likes the conversation to be stateless (the server remembers nothing between requests — exactly like HTTP in Topic 6, so the client simply re-sends its identity each time), and data is usually sent as JSON (a simple text format we cover in the very next topic). Do not worry about these yet — the two ideas above are the real heart of REST.

Resources and Endpoints

A resource is a thing. An endpoint is the URL (address) that points to that resource. The beauty of REST is how logically these are organised. Look at this pattern for "users":

EndpointWhat it refers to
/usersThe whole collection of all users
/users/2One specific user (the one with id 2)
/users/2/postsAll posts belonging to user 2

Notice these are all nouns. You never see /getUser or /deleteUser in good REST design. The URL only names the thing; the action comes from the HTTP method. This separation is the heart of REST.

Mapping Methods to Actions (CRUD)

Almost everything an app does to data falls into four actions, nicknamed CRUD: Create, Read, Update, Delete. REST maps each to an HTTP method. This is the single most important table in this topic:

Action (CRUD)HTTP MethodExampleMeaning
ReadGETGET /users/2Fetch user 2's details
CreatePOSTPOST /usersAdd a new user
UpdatePUT / PATCHPUT /users/2Update user 2
DeleteDELETEDELETE /users/2Remove user 2

See the elegance? The same URL /users/2 can be read, updated, or deleted — just by changing the method. The address stays the same; the verb changes the action.

Interactive Demo — Try a REST API

Click any API call below to see the exact request the client sends and the response the server returns (with its status code). This is a realistic "users" API.

Request
Click an API call above →
Response
…

Anatomy of a REST Call

Putting it together, here is a full REST request and response for creating a new user. Notice how it uses the HTTP knowledge from Topic 6:

Request:
POST /users HTTP/1.1
Host: api.example.com
Content-Type: application/json

{ "name": "Amit", "email": "amit@gmail.com" }

Response:
201 Created
Content-Type: application/json

{ "id": 3, "name": "Amit", "email": "amit@gmail.com" }

The client said "create a user" (POST to the /users collection) with the data in the body. The server created it and replied 201 Created along with the new user, including the id it assigned. Clean and predictable.

Good REST Design Practices

These small rules are what separate a clean API from a messy one, and interviewers notice them:

  • Use nouns, not verbs, in URLs: /users, not /getUsers. The method already says the action.
  • Use plural nouns for collections: /users and /users/2, consistently.
  • Use the right status codes: 200/201 for success, 404 when not found, 400 for bad input, 500 for server errors.
  • Version your API: put /v1/ in the path (like /v1/users) so you can change things later without breaking old apps.
  • Keep it stateless: every request carries its own authentication (like a token), since the server remembers nothing.

Why REST Became So Popular

  • Simple and predictable: once you learn the style, every REST API feels familiar.
  • Uses plain HTTP: no special tools needed — it works in any browser, language, or device.
  • Stateless and scalable: because the server holds no memory of clients, you can add more servers easily (a big deal for scaling, as we will see later).
  • Human-readable: the URLs and JSON are easy to read and debug.

Common Misconceptions (Cleared)

Myth 1: "REST is a protocol or a technology."

Reality: REST is a style / set of guidelines, not a protocol. It uses the existing HTTP protocol. You cannot "install REST" — you just design your API to follow REST principles.

Myth 2: "REST must always use JSON."

Reality: JSON is the most common format, but REST can technically use others (like XML). JSON just won because it is simple and lightweight (more on this next topic).

Myth 3: "The URL should describe the action, like /createUser."

Reality: In REST, the URL names the resource (a noun), and the HTTP method describes the action. So it is POST /users, never /createUser.

Interview Tips

  • Be able to instantly map CRUD to HTTP methods (Create=POST, Read=GET, Update=PUT/PATCH, Delete=DELETE). This comes up very often.
  • Emphasise that REST is stateless — this directly enables horizontal scaling, which interviewers love to connect.
  • When designing any system, you will define the API endpoints early. Showing clean, noun-based, RESTful endpoints signals strong fundamentals.
  • Know one alternative exists (GraphQL, gRPC) — we compare them in a later topic — so you can mention REST is one choice among a few.

Key Takeaways

  • An API is a defined way for software to talk to software; the waiter between client and server.
  • REST is a popular style of web API: resources have noun-based URLs, and HTTP methods act on them.
  • CRUD → HTTP: Create=POST, Read=GET, Update=PUT/PATCH, Delete=DELETE.
  • Same URL (/users/2), different method = different action. URLs are nouns; methods are verbs.
  • REST is stateless, uses plain HTTP, usually exchanges JSON, and scales easily — which is why it dominates the web.

#9JSON vs XML

One-line definition

JSON and XML are two different ways of writing down data as text so that it can be sent between a client and a server. They both carry the same information — they just look different. JSON is short and clean; XML is longer and uses tags.

In the last topic, our REST API sent back data like { "name": "Priya" }. That format is called JSON. But why that format? Could it be written another way? Yes — and the main alternative is XML. Let us understand both in the simplest possible way.

First: Why Do We Even Need a "Data Format"?

Here is the basic problem. A server wants to send some data — say, a user's name, age, and city — to a phone. But data can only travel across the internet as plain text. So the server must write the data down as text in a way that the phone can read back and understand perfectly.

This is exactly like filling a form. If everyone writes their details in a clear, agreed structure, anyone can read it back correctly. A data format is just that agreed structure for writing data as text. JSON and XML are two such agreed structures.

Analogy: Writing down an address

Imagine telling someone your address so they can write it down. One person writes it as a short, neat note. Another writes it as a full labelled form ("House Name: ...", "Street: ...", "City: ..."). Both contain the same address — one is just shorter, the other more labelled and longer. JSON is the short neat note; XML is the fully labelled form.

JSON — The Short, Clean One

JSON stands for JavaScript Object Notation, but you do not need to know JavaScript to use it. The idea is beautifully simple: data is written as key-value pairs — a label, then its value. A "key" is the name of the thing (like name), and the "value" is the actual data (like "Priya").

{
  "name": "Priya",
  "age": 24,
  "city": "Mumbai"
}

Read it like plain language: the name is "Priya", the age is 24, the city is "Mumbai". The curly braces { } simply mean "here is one object (one thing) with these details". It is easy for both humans and computers to read — which is exactly why it became so popular.

XML — The Tag-Based One

XML stands for eXtensible Markup Language. If you have seen HTML, XML looks very familiar: every piece of data is wrapped inside an opening tag and a closing tag, like <name>Priya</name>. The same user data looks like this in XML:

<user>
  <name>Priya</name>
  <age>24</age>
  <city>Mumbai</city>
</user>

It carries the exact same information as the JSON above. But notice the difference: each piece of data is written twice in tags (once to open <name>, once to close </name>). This makes XML more descriptive, but also much longer and heavier.

Interactive — Same data, two formats

This is the exact same information about one user. Toggle between the two formats and watch how much shorter and cleaner JSON is compared to XML.

{
  "name": "Priya",
  "age": 24,
  "city": "Mumbai",
  "premium": true
}
๐Ÿ“ About 70 characters — short, clean, easy to read.

Side-by-Side Comparison

PointJSONXML
StyleKey-value pairs { }Opening & closing tags
SizeShorter, lighterLonger, heavier
Easy to read?Very easyHarder (more clutter)
Speed to processFasterSlower
Best friendWeb & mobile apps, REST APIsOlder / enterprise systems
Popularity todayThe default choiceLess common, still around

Why JSON Won (Mostly)

Today, JSON is the default for almost all modern web and mobile APIs. Here is why it took over:

  • It is smaller: less text means less data to send, so it travels faster and is cheaper — a big deal when serving millions of users.
  • It is easier to read: for both humans (debugging) and computers (processing).
  • It fits the web naturally: JSON came from JavaScript, the language of web browsers, so it plugs in effortlessly.

So Is XML Dead? No.

XML is older, but it is still used in plenty of places, and it has some genuine strengths. It is more descriptive and can carry extra information (like attributes and comments), and many large, older "enterprise" systems — banks, government systems, and a style of API called SOAP — were built on XML and still run on it. Configuration files for some tools are also written in XML. So you will still meet it; it is just no longer the first choice for new web apps.

Common Misconceptions (Cleared)

Myth 1: "JSON and XML are programming languages."

Reality: Neither is a programming language. You cannot write logic or loops in them. They are just formats for writing down data as text — nothing more.

Myth 2: "You need to know JavaScript to use JSON."

Reality: JSON only borrowed its look from JavaScript. Every programming language (Python, Java, C++, etc.) can read and write JSON easily. The name is just history.

Myth 3: "JSON is always better than XML."

Reality: For modern web apps, JSON is usually the better fit. But XML is not "worse" — it is more descriptive and is the right choice in certain enterprise and document-heavy systems. It depends on the use case.

Interview Tips

  • If asked "which format would your API use?", the safe modern answer is JSON — and you can explain why (lighter, faster, easier, web-native).
  • Be able to write a small example of both from memory. It shows you genuinely understand the difference, not just the buzzwords.
  • Mention that the smaller size of JSON matters at scale — saving a few bytes per response adds up massively across millions of requests. This connects to performance, which interviewers love.

Key Takeaways

  • JSON and XML are both just ways to write data as text for sending between client and server — not programming languages.
  • JSON uses simple key-value pairs — short, clean, fast, and the modern default.
  • XML uses opening and closing tags (like HTML) — more descriptive but longer and heavier.
  • JSON won for the web because it is smaller, easier, and faster; XML still lives on in older and enterprise systems.
  • They carry the same information — the choice is about size, readability, and fit, not about what data they can hold.

#10Latency, Throughput & Bandwidth

One-line definition

Latency = the delay before data starts arriving (how long you wait). Bandwidth = the maximum amount of data that could flow at once (the capacity). Throughput = the actual amount of data that does flow per second (real-world speed).

These three words measure the "speed" of a system, and they are constantly mixed up — even by experienced people. By the end of this topic you will tell them apart instantly. The secret is one simple analogy that makes all three obvious at once.

The Master Analogy: A Highway

Cars travelling on a highway

Imagine cars (your data) driving from City A to City B on a highway:

  • Latency = how long one car takes to travel from A to B. It is a time (measured in milliseconds). Less is better.
  • Bandwidth = how many lanes the highway has. More lanes means more cars can travel side by side. It is the maximum capacity.
  • Throughput = how many cars actually reach City B each second in reality (after traffic jams, slow cars, etc.). It is the real achieved rate.

Hold this highway picture in your mind. Now each term becomes easy.

Latency — The Waiting Time

Latency is the delay between asking for something and starting to get it. It answers: "How long do I have to wait?" It is measured in milliseconds (ms), and lower is better.

You feel latency every day. When a video call has "lag" and the other person's reply comes a second late — that is high latency. When an online game feels laggy — high latency. The famous "ping" number gamers care about is latency. A big cause of latency is simply distance: data sent to a server on the other side of the planet takes longer than to one in your own city (which is exactly why we later use CDNs to place servers near users).

Bandwidth — The Maximum Capacity

Bandwidth is the most data that could travel through the connection in one second, if conditions were perfect. It is the highway's number of lanes — the theoretical maximum. It is measured in things like Mbps (megabits per second). When your internet plan says "100 Mbps", that is bandwidth: the maximum it promises.

Throughput — The Real Speed You Actually Get

Throughput is how much data actually gets through per second in the real world. Bandwidth is the promise; throughput is the reality. Just like a 4-lane highway rarely carries the maximum number of cars (because of traffic, accidents, slow vehicles), your real throughput is usually less than your bandwidth, because of network congestion, errors, and other slowdowns.

The key relationship

Bandwidth is the maximum possible; throughput is what you really get. Throughput can never be higher than bandwidth, and is usually a bit lower. Think: bandwidth = the size of the pipe, throughput = the water actually flowing through it right now.

Interactive — Download on slow vs fast internet

Click a connection to download the same 10 MB file. Watch the latency (the short wait before anything happens) and the throughput (how fast the bar then fills). Notice the fast connection has both lower latency and higher throughput.

Latency (wait)
—
Throughput (speed)
—
Total time
—
Pick a connection above to start the download…

The Big Insight: High Bandwidth Does NOT Mean Low Latency

This is the point that confuses everyone, so let us nail it. Bandwidth and latency are completely separate things. You can have one without the other:

  • High bandwidth but high latency: imagine a 100-lane highway, but City B is on another continent. Tons of cars can flow (high bandwidth), but each one still takes a long time to arrive (high latency). Like a satellite internet connection: lots of capacity, but a noticeable delay.
  • Low bandwidth but low latency: a 1-lane road between two neighbouring houses. Few cars at a time (low bandwidth), but each arrives almost instantly (low latency).
A famous way to picture it

A truck full of hard drives driving across the country has enormous bandwidth (it carries a massive amount of data) but terrible latency (it takes two days to arrive). A single text message over fibre has tiny bandwidth but amazing latency (it arrives in milliseconds). Different jobs need different priorities.

Which One Matters When?

When latency matters most

  • Online gaming — you need instant reactions.
  • Video/voice calls — delay makes talking awkward.
  • Stock trading — milliseconds mean money.
  • Any "click and it should respond now" action.

When bandwidth/throughput matters most

  • Streaming HD/4K video — lots of data per second.
  • Downloading big files — you want it all fast.
  • Backing up data to the cloud.
  • Any "move a lot of data" task.

Side-by-Side Comparison

TermMeasuresUnitHighway meaning
LatencyDelay / waiting timemilliseconds (ms)Time for one car A→B
BandwidthMaximum capacityMbpsNumber of lanes
ThroughputActual data per secondMbpsCars actually arriving/sec

Common Misconceptions (Cleared)

Myth 1: "More bandwidth always means a faster experience."

Reality: If your problem is latency (delay), adding bandwidth will not help. A video call does not get smoother by buying a bigger internet plan if the lag comes from distance. Match the fix to the real problem.

Myth 2: "Bandwidth and throughput are the same thing."

Reality: Bandwidth is the maximum possible; throughput is the actual achieved. Real throughput is almost always lower than the advertised bandwidth due to congestion and overhead.

Myth 3: "Latency can be reduced to zero."

Reality: Latency can be made very small, but never zero — data cannot travel faster than the speed of light. That physical limit is why placing servers closer to users (via CDNs) is the main way to cut latency.

Interview Tips

  • Keep the one-liners ready: latency = delay, bandwidth = max capacity, throughput = actual rate. Crisp definitions impress.
  • Always state your targets with numbers: "we want latency under 100 ms" sounds far more professional than "we want it fast".
  • If a system feels slow, diagnose which of the three is the bottleneck before proposing a fix — this shows real engineering thinking.
  • Connect latency to CDNs and caching (upcoming topics): putting data closer to users is the classic way to reduce it.

Key Takeaways

  • Latency = the waiting delay (in ms). Lower is better. Caused largely by distance.
  • Bandwidth = the maximum data capacity (the number of highway lanes).
  • Throughput = the actual data delivered per second (cars that really arrive). Usually a bit less than bandwidth.
  • High bandwidth does not mean low latency — they are separate. A 100-lane road to another continent is still slow per car.
  • Latency matters for real-time things (gaming, calls); bandwidth matters for large data things (streaming, downloads).

#11Monolithic Architecture

One-line definition

A Monolithic Architecture means the entire application is built as one single, large unit. All the features — login, payments, search, everything — live together in one codebase and run as one program.

The word "monolith" means "one giant stone". A monolithic app is exactly that: one big block of software where all the parts are tightly joined together and shipped as a single deployable program. This is how almost every successful application starts — including giants like Amazon, Netflix, Twitter, and Instagram, which all began life as monoliths — because it is by far the simplest and fastest way to build something that works.

Let us slow down and really understand what "one single unit" means, because this is the foundation for the next two topics. In a monolith, your entire product — every feature your users can see and use — lives inside one project, compiled or packaged into one artifact, and run as one process. When you start the app, all of it starts together. When you stop it, all of it stops together.

Real-life Analogy: A One-Room Shop

Everything under one roof

Imagine a small shop where one person handles everything in a single room — billing, storage, customer service, and cleaning, all in the same space. When the shop is small, this is wonderful: there is no overhead, no coordination, and the owner can do anything instantly. But two problems appear as it grows. First, if that one room catches fire, the entire shop stops — billing, storage, everything goes down at once. Second, if you hire 5 people, they all crowd into the same little room, bumping into each other and arguing over the same shelves. That crowding-and-fragility is exactly what happens to a monolithic codebase as the product and the team grow.

What Actually Lives Inside a Monolith

To make this concrete, picture a typical e-commerce app built as a monolith. Inside that one codebase you would find all of these, calling each other directly as ordinary function calls:

  • User module — signup, login, profiles.
  • Product catalog module — listings, search, categories.
  • Cart & checkout module — adding items, calculating totals.
  • Payment module — charging cards.
  • Order module — placing and tracking orders.
  • Notification module — sending emails and SMS.

Crucially, all of these talk to one shared database, and they communicate with each other simply by calling functions in memory — for example, the checkout code just calls paymentModule.charge(order) directly. There is no network between them, which is why a monolith is so fast internally.

How It Works: The Build and Deploy Cycle

In a monolith, all the code is written, built, tested, and deployed as one piece. This single fact drives almost every pro and con. When you make even a tiny change — say, fixing the colour of the login button — you must rebuild and redeploy the entire application, including payments, search, and everything else, even though none of those changed.

Users
→
Single Applicationlogin + search + payments + orders all together
→
One Database
One big unit holds every feature and talks to one shared database.

Why a Single Bug Can Take Everything Down

Because everything runs in one process and shares the same memory, a serious problem in any one part can crash the whole thing. If the search feature has a memory leak, or the notification code goes into an infinite loop, it consumes the resources of the single process — and your payments and checkout go down with it, even though they were written perfectly. There is no wall between the modules. This is the flip side of the speed advantage: tight coupling means tight blast radius.

Signs Your Monolith Is Starting to Hurt

A monolith is not "bad" — but there are clear warning signs that tell you it has outgrown its usefulness:

  • Deploys become scary and slow — a 30-second change requires redeploying a huge app, and everyone is afraid to ship.
  • Teams block each other — 50 engineers editing one codebase causes constant merge conflicts and coordination overhead.
  • You can't scale the hot part — only search is overloaded, but you are forced to run more copies of the entire app to cope.
  • One module's failure keeps taking down unrelated features.
  • The codebase is so large that builds and tests take forever.

These exact pains are what pushed companies like Amazon and Netflix to eventually split into microservices — which we cover next.

Bonus concept: the "Modular Monolith"

There is a smart middle path. A modular monolith is still deployed as one unit (keeping the simplicity), but the code inside is organised into clean, well-separated modules with clear boundaries. This gives you most of the organisational benefits of microservices without the operational complexity — and makes it much easier to split into microservices later if you ever need to. Many experienced teams now recommend starting here.

Strengths and Weaknesses

Strengths

  • Simple to build at the start — one codebase, one place.
  • Easy to test and deploy — just one app to ship.
  • Fast internally — features call each other directly, no network in between.
  • Great for small teams and new products.

Weaknesses

  • Hard to scale parts — you must scale the whole thing, even if only search is busy.
  • One bug can crash everything.
  • Slow to update as it grows — every change redeploys the whole app.
  • Teams step on each other in one giant codebase.
Important: monolith is NOT "bad"

Starting with a monolith is often the smartest choice for a new app. It is simple and fast to build, easy to reason about, and lets a small team move incredibly quickly. The problems only appear when the app and team grow very large — which is when people consider microservices (next topic). Reaching for microservices on day one is one of the most common beginner mistakes.

Common misconceptions, cleared

"Monolith means messy/spaghetti code." False — a monolith can be beautifully organised (a modular monolith). "Monolith" describes how it is deployed (one unit), not how clean the code is.

"Big companies don't use monoliths." False — many still run large, successful monoliths, and almost all started as one. Even where microservices exist, individual services are often small monoliths internally.

"Monolith can't scale." Misleading — a monolith scales fine horizontally (run many copies behind a load balancer); what it can't do is scale individual features independently.

Interview tip

If asked "monolith or microservices for a new startup?", the strong answer is: "Start with a monolith (or modular monolith) for speed and simplicity, and only split into microservices when you feel real pain — team size, deploy friction, or independent scaling needs." This shows mature judgement instead of chasing trends.

Key Takeaways

  • A monolith = the whole app as one single unit: one codebase, one build, one deployment, one shared database.
  • Modules talk via direct in-memory function calls — no network — which is why it is fast internally.
  • Pros: simple to build, test, and deploy; fast internally; great for small teams and early-stage products.
  • Cons: can't scale parts independently, one bug can take down everything, deploys get slow and risky, teams collide as it grows.
  • It is the right starting point for most projects — not a mistake. A modular monolith is a great middle ground.

#12Microservices Architecture

One-line definition

Microservices Architecture means breaking the application into many small, independent services, where each service handles one specific job (like payments, or search) and runs on its own. They talk to each other over the network (usually via APIs).

If a monolith is "one giant stone", microservices are "many small bricks". Instead of one huge program, you build lots of small programs, each focused on a single business capability, each able to be developed, deployed, and scaled on its own. This is not just a coding style — it is an architectural philosophy that reshapes how teams are organised and how the whole system behaves under load and failure.

The core promise of microservices is independence. Each service is owned by a team, lives in its own repository, has its own database, runs as its own process (or many copies), and can be released without waiting for or affecting any other service. That independence is the prize — and almost everything good and bad about microservices flows from it.

Real-life Analogy: A Food Court

Many independent stalls

A monolith is one giant kitchen cooking every cuisine. Microservices are a food court: separate stalls for pizza, dosa, Chinese, and juice. Each stall runs independently — if the pizza stall closes for cleaning, you can still buy dosa. If the juice stall gets crowded in summer, you can add more juice staff without touching the pizza stall. Each stall has its own counter, its own staff, and its own cash box (its own database). But there is a cost: the stalls now need a shared map and signage so customers can find them, and if the dosa stall needs an ingredient from the pizza stall, they have to physically walk over and ask — slower than reaching across one kitchen. That "walking over" is the network call between services.

The Golden Rule: Database Per Service

This is the single most important — and most misunderstood — principle of microservices. Each service owns its own private database, and no other service is allowed to touch it directly. If the Order service needs user data, it must ask the User service through its API; it can never reach into the User service's database tables.

Why be so strict? Because if many services shared one database, they would become secretly coupled through it — changing one table could break five services, and you would lose the independence that was the whole point. A private database keeps each service a true black box that can evolve freely behind its API.

How Services Talk to Each Other

Since services are separate programs (often on separate machines), they communicate over the network, in two main styles:

  • Synchronous (request/response): one service directly calls another and waits for the reply — using REST or gRPC (Topics 8 and 45). Simple, but creates a temporary dependency: if the called service is slow or down, the caller is stuck.
  • Asynchronous (events/messages): a service publishes an event to a queue or message bus and moves on; interested services react later (Topics 38–40). More resilient and decoupled, but harder to trace.
Users
→
API Gateway
↓
User Serviceown DB
Order Serviceown DB
Payment Serviceown DB
Notification Serviceown DB
Many small services, each owning its private database, talking over the network through an API gateway.

Full Walkthrough: Placing an Order

Let us trace one real request through a microservices e-commerce app, so the abstract idea becomes concrete:

  • 1. You tap "Place Order". The request hits the API Gateway (Topic 50), the single front door.
  • 2. The gateway routes it to the Order service, which creates an order record in its own database.
  • 3. The Order service calls the Payment service over the network to charge your card.
  • 4. On success, the Order service publishes an "order_placed" event. The Notification service reacts by emailing you, and the Inventory service reacts by reducing stock — all independently.
  • 5. Each of these services can be scaled, deployed, and even rewritten separately without the others knowing.

Notice how a single user action becomes a small orchestra of independent services cooperating over the network.

How the Real Giants Use Microservices

Netflix, Amazon, Uber

Netflix famously runs hundreds of microservices — separate services for recommendations, playback, billing, search, and more — so that, for example, if the recommendations service has a bad day, you can still hit play on a movie. Amazon moved from a monolith to microservices so its thousands of engineers could ship independently; the rule "every team exposes its data only through service APIs" became legendary. Uber split its early monolith into services for trips, pricing, matching, and payments to scale each independently as it grew across cities. In every case, the move happened because of scale and team size — not because microservices are inherently "better".

The Hidden Cost: You Are Now Building a Distributed System

This is the part beginners underestimate. The moment your features talk over a network instead of in-memory function calls, you inherit the hard problems of distributed systems (which Part 8 covers in depth):

  • The network is unreliable — calls can fail, time out, or arrive twice. You need retries, timeouts, and circuit breakers (Part 7).
  • No more easy transactions — you can't wrap a change across services in one database transaction; you need patterns like Saga (Topic 63).
  • Debugging is harder — one request spans many services, so you need distributed tracing (Topic 83).
  • Operational overhead — many services to deploy, monitor, and secure, which demands strong DevOps, containers, and orchestration (Part 12).

Strengths and Weaknesses

Strengths

  • Scale each part separately — add more Payment servers without touching others.
  • One service failing does not crash the whole app.
  • Teams work independently on their own service.
  • Each service can use a different technology if it suits.

Weaknesses

  • Much more complex — many moving parts to manage.
  • Network calls between services add latency and can fail.
  • Harder to test and debug across services.
  • Needs strong DevOps (monitoring, deployment automation).
Common misconceptions, cleared

"Microservices are always better than monoliths." False — for a small app or team, they add huge complexity for little benefit and will slow you down. They pay off at scale.

"Microservices means tiny services." Not exactly — "micro" refers to a focused business capability, not lines of code. A service should be as small as makes sense, not artificially tiny.

"Each service must have its own database server." It needs its own data ownership (its own schema/tables that others can't touch) — it doesn't necessarily need a separate physical server.

Interview tip

When proposing microservices, always justify it with scale and team independence, and immediately acknowledge the costs you will pay (network failures, distributed transactions, observability). Then mention the mitigations: API gateway, async events, sagas, circuit breakers, and tracing. Showing you know the downsides is what separates a senior answer from a buzzword answer.

Key Takeaways

  • Microservices = the app split into many small, independent services, each owning one business capability and its own private database.
  • Services communicate over the network — synchronously (REST/gRPC) or asynchronously (events).
  • Pros: independent scaling, fault isolation, independent teams, freedom to choose tech per service.
  • Cons: you inherit all distributed-system problems — network failures, no easy transactions, harder debugging, heavy ops.
  • Best for large apps and large teams (Netflix, Amazon, Uber) — and only after a monolith starts to hurt.

#13Monolith vs Microservices

One-line definition

This topic directly compares the two architectures so you know which to choose and when. In short: start with a monolith for simplicity, and move to microservices only when scale and team size make the monolith painful.

Now that you understand both (Topics 11 and 12), let us put them head to head. This is one of the most common system design discussion points, so it is worth being crystal clear.

Interactive — See both at a glance

Toggle to compare how each one handles the same situations.

  • Deploy: ship the whole app at once.
  • Scaling: scale the entire app together.
  • A bug: can crash the whole application.
  • Team: everyone works in one codebase.
  • Best when: small app, small team, early stage.

Full Comparison Table

PointMonolithMicroservices
StructureOne single unitMany small services
ComplexityLow (simple)High (many parts)
DeploymentWhole app togetherEach service alone
ScalingScale everythingScale parts independently
Fault isolationOne bug can crash allFailures stay contained
Speed to build (early)FastSlow (lots of setup)
Best team sizeSmallLarge

How to Actually Decide (a practical checklist)

Instead of guessing, ask yourself these concrete questions. The more you answer "yes", the more microservices make sense:

  • Is your team large? If 50+ engineers keep colliding in one codebase, splitting into service-owning teams helps. If you are a team of 5, a monolith is almost certainly right.
  • Do different parts have very different scaling needs? If search gets 100× the traffic of billing, being able to scale them separately is valuable.
  • Are deploys slow and scary? If shipping a one-line fix means a risky redeploy of everything, independent deployment is a real win.
  • Do you have strong DevOps maturity? Microservices require automation, monitoring, and orchestration. Without it, you will drown.
  • Is fault isolation critical? If one feature crashing must not take down the rest, separate processes help.

If most answers are "no", a monolith (or modular monolith) is the better engineering choice — not the "lesser" one.

The Worst of Both Worlds: The "Distributed Monolith"

A trap to avoid

If you split into microservices but they are still tightly coupled — they must all be deployed together, share a database, or every request fans out across many of them — you get a distributed monolith: all the complexity and network pain of microservices, with none of the independence. This is a very common and painful failure. The fix is clean boundaries: each service truly independent, with its own data and the ability to deploy alone.

How Companies Migrate: The Strangler Fig Pattern

You rarely rewrite a monolith into microservices in one big bang — that is far too risky. Instead, teams use the Strangler Fig pattern (named after a vine that slowly grows around a tree until it replaces it):

  • 1. Keep the monolith running and serving traffic.
  • 2. Pick one capability (say, notifications) and build it as a new, separate microservice.
  • 3. Route just that capability's traffic to the new service (often via the API gateway), leaving everything else in the monolith.
  • 4. Repeat, peeling off one piece at a time, until the monolith is small or gone.

This lets you migrate safely and incrementally, with the ability to roll back any single piece. It is exactly how Amazon and others did it.

The Golden Rule

Start monolith, split later

Most successful companies (including Amazon and Netflix) started as monoliths and moved to microservices only when growth demanded it — gradually, using the strangler pattern. Do not start with microservices for a brand-new small app — the complexity will slow you down for no benefit. Add complexity only when you actually feel the pain.

Common interview trap

If asked "monolith or microservices?", never blurt "microservices because it is modern". The smart answer is: "It depends on scale and team size" — then explain the trade-offs. That shows real judgement.

Key Takeaways

  • Monolith: simple, fast to start, but hard to scale and risky as it grows.
  • Microservices: scalable and resilient, but complex to build and operate.
  • The right answer depends on scale and team size — there is no universal winner.
  • Golden rule: start with a monolith, split into microservices only when the pain is real.

#14Vertical vs Horizontal Scaling

One-line definition

Scaling means making your system handle more users/load. Vertical scaling = make one server more powerful (bigger machine). Horizontal scaling = add more servers that share the work.

When your app gets popular and one server can no longer cope, you have exactly two ways to grow. Understanding both — and why horizontal usually wins at large scale — is core to system design.

Real-life Analogy: A Busy Restaurant Kitchen

Bigger chef vs more chefs

Your kitchen cannot keep up with orders. You have two choices:

Vertical scaling: hire one super-chef with more hands, faster knives, a bigger stove. One person, more powerful. But there is a limit — no chef can have 50 hands.

Horizontal scaling: hire more normal chefs and split the orders among them. Need more capacity? Just add another chef. There is almost no limit.

Vertical Scaling (Scale Up)

You upgrade the single server — add more CPU, more RAM, faster disks. The app does not change at all; it just runs on a beefier machine.

  • Pro: very simple — no code changes, nothing to coordinate.
  • Con: there is a hard ceiling (a machine can only get so big), and a powerful machine is very expensive.
  • Con: it is still one server — if it dies, everything goes down (single point of failure).

Horizontal Scaling (Scale Out)

You add more servers and put a load balancer (next topic) in front to share the work between them.

  • Pro: almost unlimited growth — just keep adding servers.
  • Pro: if one server dies, the others keep serving (no single point of failure).
  • Con: more complex — you need load balancing and your app should be "stateless" (covered later).
Vertical:
Small Server
→
One BIG Server
 
Horizontal:
Server 1
Server 2
Server 3
+ more…
Vertical = one bigger machine. Horizontal = many machines working together.
PointVertical (Scale Up)Horizontal (Scale Out)
HowBigger single machineMore machines
LimitHard ceilingAlmost unlimited
Failure riskSingle point of failureOthers survive
ComplexitySimpleMore complex
Cost curveGets very expensiveUses cheap commodity servers

The Hidden Requirement of Horizontal Scaling: Statelessness

Here is the catch that trips people up. When you have many servers behind a load balancer, a single user's requests may land on different servers each time. Request 1 hits Server A, request 2 hits Server B. If Server A stored "this user is logged in" in its own memory, then Server B has no idea who the user is — and the user appears randomly logged out.

So horizontal scaling requires your servers to be stateless: they must not keep important per-user information in their own memory. Instead, that state is pushed to a shared store (a database or Redis) that all servers can read (this is Topics 47–48). Vertical scaling never has this problem — with one server, all state is naturally in one place. This is a big part of why horizontal scaling is "more complex".

When Each One Wins (with numbers)

Imagine one server can handle 1,000 requests/sec, and you suddenly need 10,000.

  • Vertical: buy a machine ~10× more powerful. Easy — until you need 1,000,000 req/sec, where no single machine on Earth can cope, and the top-end machines cost a fortune. The cost curve is steep and the ceiling is hard.
  • Horizontal: run 10 ordinary servers, each handling 1,000. Need a million? Run 1,000 servers. The growth is essentially unlimited, using cheap "commodity" machines, and you get fault tolerance for free.

A Key Nuance: Databases Often Scale Vertically First

Stateless app servers vs stateful databases

Stateless app servers are easy to scale horizontally. But databases hold state (the data!), so scaling them out is much harder — you need replication and sharding (Topics 26–27). For this reason, teams frequently scale the database vertically (a bigger DB machine) for as long as possible, while scaling the app tier horizontally. Recognising this split is a sign of real understanding.

Which do big companies use?

Horizontal scaling is how all large systems (Google, Netflix, Amazon) handle massive scale, because it has no real ceiling and survives failures. Vertical scaling is a quick early fix, but you eventually hit its limits. In practice, real systems use both: scale up to a sensible machine size, then scale out with many of them.

Common misconceptions, cleared

"Horizontal is always better." Not for everything — for stateful systems like databases, scaling out is genuinely hard, and vertical may be the right first move.

"Scaling means just adding servers." Adding servers does nothing without a load balancer to distribute traffic and stateless design so any server can serve any request.

Interview tip

When asked to scale a system, say: "First I'd scale vertically as a quick win, but the long-term answer is horizontal scaling behind a load balancer, which means making the app tier stateless and pushing session/state to a shared store like Redis. The database I'd scale with replicas and, if needed, sharding." That single sentence ties five concepts together.

Key Takeaways

  • Vertical scaling = one more powerful server. Simple (no code change), but has a hard ceiling, gets expensive, and is a single point of failure.
  • Horizontal scaling = more servers sharing load. Nearly unlimited and fault-tolerant, but more complex.
  • Horizontal scaling requires stateless servers + a load balancer; state moves to a shared store.
  • Databases are stateful, so they often scale vertically first, then via replication/sharding.
  • Real systems use both: scale up to a point, then scale out.

#15Load Balancer

One-line definition

A Load Balancer is a "traffic police" that sits in front of your servers and spreads incoming requests evenly across them, so no single server gets overloaded and the system stays fast and available.

The moment you have more than one server (horizontal scaling), you need something to decide which server each request should go to. That something is the load balancer. It is one of the most important building blocks in all of system design.

Real-life Analogy: Queues at a Supermarket

The staff member directing you to a free counter

At a busy supermarket, imagine a staff member standing at the front who watches all the billing counters and tells each customer "Counter 3 is free, go there!". This keeps all counters evenly busy — no huge line at one while another sits empty. A load balancer does exactly this for servers.

What a Load Balancer Does

  • Distributes requests across multiple servers so the load is shared.
  • Health checks: it constantly pings servers, and if one is dead, it stops sending traffic there — users never hit a broken server.
  • Enables scaling: you can add or remove servers behind it without users noticing.
  • Improves availability: if one server fails, traffic just goes to the healthy ones.
Users
→
Load Balancer
→
Server 1
→
Server 2 & 3
All requests hit the load balancer first, which forwards each to a healthy, free server.

What Actually Happens on a Single Request

Let us trace one request to make the role crystal clear. You type myapp.com and hit enter:

  • 1. DNS resolves myapp.com to the load balancer's public IP (not to any individual server — users never know the real servers exist).
  • 2. Your request arrives at the load balancer.
  • 3. The load balancer picks a healthy server using its algorithm (Topic 16) — say Server 2.
  • 4. It forwards your request to Server 2, gets the response, and sends it back to you.
  • 5. Your next request might go to Server 5 instead — which is exactly why the servers must be stateless (Topic 47).

Health Checks in Detail

The "magic" that keeps users away from broken servers is the health check. The load balancer repeatedly asks each server "are you OK?" (e.g. it calls a /health endpoint every few seconds, Topic 59):

  • If a server responds healthy → it stays in the rotation and receives traffic.
  • If a server fails several checks in a row → it is temporarily removed, so no user request is sent to it.
  • When it recovers and passes checks again → it is automatically added back.

This is automatic and invisible to users — a server can crash and most people never notice.

Hardware vs Software Load Balancers

Load balancers come in a few forms you should be able to name:

  • Software: Nginx and HAProxy are hugely popular, running on ordinary servers. Flexible and cost-effective.
  • Cloud / managed: AWS ELB/ALB, Google Cloud Load Balancer — the cloud runs and scales it for you.
  • Hardware: dedicated physical appliances (e.g. F5) — very fast, used in large enterprises, but expensive.

Layer 4 vs Layer 7 Load Balancers

There are two types you should know by name, based on how deeply they look at traffic (the "layer" refers to the OSI networking model):

  • Layer 4 (L4 — transport level): forwards traffic based only on IP address and port, without looking inside the request. It is like a postal sorter who routes by the address on the envelope without opening it. Extremely fast and efficient, but "dumb" — it cannot make content-based decisions.
  • Layer 7 (L7 — application level): can look inside the request (the URL path, headers, cookies) and make smarter decisions — e.g. send /images/* to image servers and /api/* to API servers, or route by user. It is like a receptionist who reads your letter and sends it to exactly the right department. Slightly slower, but far more capable. Most modern web load balancers (Nginx, ALB) are L7.

The Load Balancer Must Not Become the Single Point of Failure

Who balances the balancer?

You added a load balancer for reliability — but now if it dies, your whole app dies, because all traffic goes through it. So the load balancer itself must be made redundant. The common solution is active-passive: run two load balancers, with a standby that instantly takes over the shared "floating" IP if the active one fails (a failover, Topic 54). Cloud load balancers handle this redundancy for you automatically.

Why it also boosts reliability

The load balancer is not just about speed — it is a key part of high availability. Because it routes around dead servers, your app can lose a server and keep running with zero user impact. It is the component that turns "many servers" into "one reliable, scalable service".

Interview tip

Whenever you draw a system with multiple app servers, always put a load balancer in front of them — and mention health checks and that the LB itself is made redundant. Forgetting the load balancer (or making it a single point of failure) is a classic interview slip.

Key Takeaways

  • A load balancer is the "traffic police" that spreads requests across many servers.
  • It does health checks and routes traffic away from dead servers — boosting availability.
  • It is what makes horizontal scaling actually work.
  • L4 = fast, routes by IP/port. L7 = smarter, can read the request content.
  • The load balancer itself should be redundant so it is not a single point of failure.

#16Load Balancing Algorithms

One-line definition

A Load Balancing Algorithm is the rule a load balancer uses to decide which server gets the next request. Different rules suit different situations.

We know the load balancer spreads requests across servers (Topic 15). But how exactly does it pick which server? That choice is made by an algorithm. Here are the main ones, in simple terms.

The Main Algorithms

AlgorithmHow it decidesGood for
Round RobinSend to each server in turn: 1, 2, 3, 1, 2, 3…Servers of equal power; simple and fair
Weighted Round RobinStronger servers get more requests (a bigger "weight")When some servers are more powerful
Least ConnectionsSend to the server currently handling the fewest active requestsWhen requests take different amounts of time
Least Response TimeSend to the server that is responding fastest right nowSqueezing out best performance
IP HashUse the user's IP to always send them to the same serverWhen a user should "stick" to one server

Understanding the Two You Will Use Most

Round Robin is the simplest: just go around the servers in a circle, one after another. It works great when all servers are equally powerful and all requests are similar. Its weakness: it does not care if one server is already busy with a slow request.

Least Connections is smarter: it looks at how busy each server currently is and sends the new request to the least busy one. This is better when some requests take much longer than others, because it avoids piling work onto an already-busy server.

Analogy: assigning customers to bank tellers

Round Robin = send each new customer to the next teller in order, regardless of how busy they are. Least Connections = look at all tellers and send the customer to whichever one has the shortest line right now. The second is obviously smarter when some customers take a long time.

Weighted & Least-Response-Time, Explained

Two more you should understand beyond the basics:

  • Weighted Round Robin: servers are not always equal. If Server A is twice as powerful as Server B, you give A a weight of 2 and B a weight of 1, so A receives roughly twice the requests. It is round robin that respects each server's capacity — useful when your fleet is a mix of machine sizes.
  • Least Response Time: goes a step beyond Least Connections by also considering how fast each server is currently responding. It sends the next request to the server that is both lightly loaded and answering quickly. This squeezes out the best real-world performance, at the cost of more monitoring overhead.

Why IP Hash Is Special (and Its Trap)

IP Hash always sends the same user to the same server (by hashing their IP address). This is useful when a server needs to "remember" a user between requests — a sticky session. For example, if Server 3 is holding your shopping cart in its own memory, IP Hash keeps sending you back to Server 3 so your cart is there.

The sticky-session problem

Sticky sessions are fragile: if "your" server dies, you lose your session (your cart vanishes). They also cause uneven load — if a few heavy users all stick to one server, it gets overloaded while others sit idle. The modern fix is to make servers stateless (Topic 47) and store sessions in a shared store like Redis (Topic 48). Then any server can serve any user, and you can use a simpler algorithm like round robin or least connections without stickiness at all.

How to Choose in Practice

  • Equal servers, similar requests → Round Robin (simple, fair).
  • Mixed machine sizes → Weighted Round Robin.
  • Requests vary a lot in duration → Least Connections (avoids overloading a busy server).
  • Performance-critical → Least Response Time.
  • Must keep a user on one server → IP Hash — but prefer fixing it with stateless design instead.
Interview tip

A great answer is: "I'd start with Round Robin or Least Connections, keep the servers stateless so I don't need sticky sessions, and use Weighted if the machines differ in size." Mentioning that you avoid sticky sessions by design shows maturity.

Key Takeaways

  • The algorithm is the rule for choosing which server gets the next request.
  • Round Robin = take turns; simple and fair for equal servers.
  • Least Connections = pick the least busy server; better for uneven request times.
  • Weighted = give stronger servers more traffic. IP Hash = stick each user to one server.
  • The best algorithm depends on your situation — another classic trade-off.

#17Forward Proxy vs Reverse Proxy

One-line definition

A proxy is a middleman that sits between a client and a server. A Forward Proxy sits in front of the client (it represents you). A Reverse Proxy sits in front of the server (it represents the website).

The word "proxy" just means "stand-in" or "middleman". The only thing to learn here is which side the middleman stands on. That single difference decides whether it is "forward" or "reverse".

Forward Proxy — Stands in Front of You (the Client)

A forward proxy is a middleman you send your requests through. The destination server sees the proxy, not you, so your identity is hidden behind it.

  • Example uses: a company proxy that filters which websites employees can visit; a VPN that hides your real location; bypassing regional blocks.
  • Who it serves: the client. It acts on your behalf.

Reverse Proxy — Stands in Front of the Server

A reverse proxy is a middleman that receives requests on behalf of the servers. The client thinks it is talking to the real server, but it is actually talking to the reverse proxy, which then forwards the request to the right server behind it.

  • Example uses: load balancing, caching, hiding/protecting the real servers, handling HTTPS encryption. A load balancer is a type of reverse proxy.
  • Who it serves: the server. It protects and represents the backend.
You
→
Forward Proxyrepresents you
→
Reverse Proxyrepresents server
→
Server
Forward proxy hides the client side; reverse proxy hides and protects the server side.
PointForward ProxyReverse Proxy
Sits in front ofThe client (you)The server
HidesThe client's identityThe servers behind it
Works forThe userThe website owner
Common usesVPN, content filteringLoad balancing, caching, security

What a Reverse Proxy Actually Does for You

The reverse proxy is the more important one in system design, because so many key components are reverse proxies (load balancers, API gateways, CDNs). Sitting in front of your servers, it can take over many jobs so your application code does not have to:

  • Load balancing — spread requests across servers (Topic 15).
  • SSL/TLS termination — handle the expensive HTTPS encryption/decryption at the proxy, so your backend servers do simpler, faster plain work behind it.
  • Caching — store common responses and serve them directly, sparing the servers.
  • Compression — shrink responses (gzip) before sending to clients.
  • Security & hiding — the real servers' addresses are never exposed; the proxy can block bad traffic and absorb attacks.
  • Routing — send /api to one set of servers and /images to another.

How They Relate to Things You Already Know

It all connects

A load balancer is a reverse proxy focused on distributing load. An API gateway (Topic 50) is a reverse proxy focused on routing + auth + rate limiting. A CDN (next topic) is a globally-distributed reverse proxy + cache. A VPN is essentially a forward proxy. So this one simple idea — a middleman, and which side it serves — underlies a huge amount of system design.

Easy way to remember

Forward proxy = for the client (hides you from the server). Reverse proxy = for the server (hides the servers from you). Just ask: "whose side is the middleman on?" If it protects the user, it is forward; if it protects the website, it is reverse.

Common misconception

"A proxy and a VPN are different things entirely." Not really — a VPN is a kind of forward proxy with encryption. And "a reverse proxy and a load balancer are different categories" — actually a load balancer is one type of reverse proxy. The labels describe roles, and one piece of software (like Nginx) can play several of them at once.

Key Takeaways

  • A proxy is a middleman between client and server; the only question is which side it represents.
  • Forward proxy sits in front of the client and hides the user (VPN, content filtering, bypassing blocks).
  • Reverse proxy sits in front of the server and does load balancing, SSL termination, caching, compression, and security.
  • Load balancers, API gateways, and CDNs are all reverse proxies. Remember: forward = for client, reverse = for server.

#18CDN (Content Delivery Network)

One-line definition

A CDN is a network of servers spread around the world that store copies of your content (images, videos, files) close to users, so it loads fast no matter where they are.

Remember from Topic 10 that distance causes latency — data from a faraway server is slow. A CDN solves this by keeping copies of your content in many locations worldwide, so users get it from a server near them instead of from one distant origin server.

Real-life Analogy: Local Warehouses

Amazon delivery from a nearby warehouse

When you order on Amazon, it does not ship from one giant warehouse on the other side of the country. It ships from a local warehouse near your city, so it arrives in a day. A CDN does the same with data: instead of every user fetching a video from one distant server, copies are kept in local "warehouses" (called edge servers) around the world, so it arrives fast.

How It Works

Your main server is called the origin. The CDN copies your static content (images, CSS, videos) onto many edge servers placed in cities worldwide. When a user requests an image, the CDN serves it from the nearest edge server. The user gets it quickly, and your origin server is spared the work.

User in India
→
Edge Server (Mumbai)nearby copy
↓
Origin Server (USA)only contacted if edge has no copy
The user is served from a nearby edge server, not the distant origin.

Edge Cache Hit vs Miss (how content gets there)

Edge servers do not magically have your content — they fill up on demand, just like any cache (Topic 31):

  • First request in a region (a "miss"): the edge server in, say, Mumbai does not have the file yet, so it fetches it once from the origin and stores a copy.
  • Every later request (a "hit"): the Mumbai edge already has the file and serves it instantly — the origin is never touched again.

So the very first user "warms" the cache, and millions after them get a fast, local copy.

Push vs Pull CDNs

There are two ways content ends up on the edge servers:

  • Pull CDN (most common): the CDN automatically pulls content from your origin the first time it is requested, then caches it. You do almost nothing — just point your site at the CDN. Best for frequently-changing or large catalogs.
  • Push CDN: you actively upload (push) content to the CDN ahead of time. You control exactly what is stored and when. Best for large files that you know will be popular (like a big software release or a launch video).

Controlling Freshness: TTL and Invalidation

Since edges hold copies, those copies can go stale. Two controls manage this (covered more in Topic 35):

  • TTL (Time To Live): each cached file has an expiry. After it expires, the edge fetches a fresh copy from the origin.
  • Invalidation / purge: if you change a file before its TTL (e.g. fix an image), you tell the CDN to clear it so users get the new version. A common trick is cache busting — renaming the file (logo.v2.png) so it is treated as brand-new.

Beyond Speed: CDNs as a Shield

  • Massive traffic absorption: CDNs are enormous global networks that soak up traffic spikes (a viral video) before they ever reach your origin.
  • DDoS protection: because attack traffic hits the huge CDN first, it can be absorbed and filtered (Topic 74).
  • Security features: many CDNs include a Web Application Firewall (WAF) and bot protection.
  • Modern CDNs even accelerate dynamic content by keeping fast, optimized connections between edges and the origin — so it is not only for static files anymore.

This is exactly how Netflix and YouTube stream video smoothly worldwide — the video chunks are cached on edge servers near you (Topic 94). It is also why your favourite websites load their images and scripts so quickly. Popular CDNs include Cloudflare, Akamai, AWS CloudFront, and Google Cloud CDN.

What CDNs are best for

CDNs are ideal for static content that does not change per user — images, videos, CSS, JavaScript files. They are traditionally not used for highly personalised, always-changing data (like your live bank balance), which must come fresh from the origin — though modern CDNs and edge computing are increasingly handling some dynamic content too.

Interview tip

In almost any design that serves images, video, or static assets globally, mention a CDN early. State that it cuts latency (content is near the user), offloads the origin, absorbs spikes, and adds DDoS protection. It is one of the highest-impact, lowest-effort additions you can propose.

Key Takeaways

  • A CDN keeps copies of your content close to users on edge servers worldwide.
  • It cuts latency (content travels a shorter distance) and reduces load on your origin server.
  • Best for static content: images, videos, CSS, JS.
  • It is how Netflix, YouTube, and most big sites deliver content fast and handle huge traffic.
  • Bonus: CDNs add reliability and security (absorbing spikes and attacks).

#19SQL vs NoSQL Databases

One-line definition

A database is where an app stores its data permanently. SQL databases store data in neat tables (like spreadsheets) with a strict structure. NoSQL databases store data in flexible formats (documents, key-values, etc.) without a fixed structure.

Every app needs to store data — users, orders, messages. The big first decision is which type of database to use. The two families are SQL and NoSQL, and each fits different needs.

SQL (Relational) Databases

SQL databases store data in tables made of rows and columns, exactly like an Excel sheet. Before storing anything, you must define a schema — the exact columns and their types. Tables can be linked ("related") to each other, which is why they are called relational. Examples: MySQL, PostgreSQL, Oracle.

  • Strict structure — every row must follow the defined columns.
  • Powerful queries using the SQL language (joins across tables).
  • Strong consistency & ACID (next topic) — great for data that must be exactly correct, like money.

NoSQL (Non-Relational) Databases

NoSQL databases skip the rigid table structure. Data can be stored as flexible documents (like JSON), key-value pairs, and more. There is usually no fixed schema, so different records can have different fields. Examples: MongoDB, Cassandra, Redis, DynamoDB.

  • Flexible structure — add new fields anytime, no migration needed.
  • Scales horizontally very easily — built to spread across many servers.
  • Great for huge volumes and fast-changing data, but often trades some strict consistency for speed.
Analogy: A strict form vs a blank notebook

SQL is like a printed government form: every box is fixed, you must fill exactly the right fields in the right format. Very organised and reliable. NoSQL is like a blank notebook: you can write anything, in any shape, on any page. Very flexible and fast, but less strictly organised.

PointSQLNoSQL
StructureFixed tables (schema)Flexible (no fixed schema)
ScalingUsually verticalEasily horizontal
ConsistencyStrong (ACID)Often eventual (BASE)
Best forStructured data, transactions (banking)Huge scale, flexible data (feeds, logs)
ExamplesMySQL, PostgreSQLMongoDB, Cassandra, Redis

The Same Data, Two Different Worlds

Let us make the difference concrete. Suppose we store a user and their orders.

In SQL, you would normally split this into two tables — a users table and an orders table — and link them by a user_id. To get "Priya and all her orders", you run a JOIN that stitches the two tables together. This avoids duplicating Priya's details and keeps everything consistent (this is normalization, Topic 24).

In a document NoSQL database (like MongoDB), you would often store the whole thing as one document — Priya's profile with her orders nested right inside it. One read gives you everything, no join needed. The trade-off: if Priya's address is duplicated across documents, updating it is harder.

This single difference — relate-and-join vs nest-it-together — explains most of the practical contrast between the two families.

Why NoSQL Scales Horizontally So Easily

This is the deeper "why" behind the scaling row in the table. In SQL, the power comes from relationships and joins across tables. But joins are very hard to perform when the data is split across many machines (sharded) — the database would have to gather rows from many servers to answer one query. So traditional SQL prefers to live on one strong machine (vertical scaling).

NoSQL databases deliberately give up complex joins and relationships. Each record is self-contained, so it can live on any machine independently. That makes spreading data across hundreds of servers (sharding, Topic 27) natural — which is exactly why NoSQL handles internet-scale data so well. The flexibility is not free; you trade away joins and often strict consistency to get it.

Schema-on-Write vs Schema-on-Read

  • SQL is "schema-on-write": you define the structure first, and the database rejects any data that does not fit. This guarantees clean, predictable data — great when correctness matters.
  • NoSQL is often "schema-on-read": you can store almost anything, and your application makes sense of it when reading. This gives flexibility and fast iteration — great when your data shape is still changing — but the discipline now lives in your code, not the database.
Real-world: who uses what

Banks & payment systems lean SQL (PostgreSQL/Oracle) because money demands ACID correctness. Instagram's feed, Netflix's viewing history, and logging pipelines lean NoSQL (Cassandra/DynamoDB) for massive scale and flexibility. Most large companies practice polyglot persistence — using SQL for orders/payments and NoSQL for feeds/sessions/search within the same product.

How to choose

Need strict correctness, relationships, and complex queries (money, orders, reporting)? Lean SQL. Need massive scale, flexible/changing data, and simple access patterns (social feeds, logs, real-time, sessions)? Lean NoSQL. Many big systems use both, each for what it is best at.

Common misconceptions, cleared

"NoSQL is newer and therefore better." False — it is a different tool with different trade-offs, not an upgrade. SQL databases are battle-tested and often the right default.

"SQL can't scale." Misleading — SQL scales very well vertically and with read replicas; modern "NewSQL" databases (like CockroachDB, Spanner) even scale horizontally with SQL and ACID.

"NoSQL means no consistency." False — many NoSQL stores can be configured for strong consistency (e.g. DynamoDB strong reads); they simply default to favouring availability and speed.

Interview tip

Never start with "I'll use NoSQL because it scales". Instead reason from the data: "This data is highly relational and needs transactions, so SQL. This part is huge-volume, simple-access, and write-heavy, so a NoSQL store like Cassandra." Justifying from access patterns and consistency needs is what interviewers want.

Key Takeaways

  • SQL = structured tables, strict schema (schema-on-write), strong consistency, powerful joins (MySQL, PostgreSQL).
  • NoSQL = flexible formats, no fixed schema, self-contained records that shard easily (MongoDB, Cassandra, Redis).
  • NoSQL scales horizontally because it gives up joins and relationships — a deliberate trade-off.
  • SQL suits transactions and correctness; NoSQL suits scale and flexibility. Choose from access patterns.
  • Big systems use both (polyglot persistence); "NewSQL" even offers SQL + horizontal scale.

#20Types of NoSQL Databases

One-line definition

"NoSQL" is not one thing — it is a family with four main types: Key-Value, Document, Column-family, and Graph. Each stores data in a different shape and is good for different jobs.

Saying "NoSQL" is like saying "vehicle" — there are many kinds for different purposes. Let us look at the four main types, simply.

1. Key-Value Stores

The simplest type. Data is stored as a key and its value, exactly like a dictionary: you look up a key, you get its value. Super fast. Examples: Redis, DynamoDB.

Best for: caching, user sessions, shopping carts — anything you fetch by a single key.

2. Document Databases

Store data as documents (usually JSON-like), where each document can hold nested, flexible fields. Examples: MongoDB, CouchDB.

Best for: flexible data like user profiles, product catalogs, content — where each item may have different fields.

3. Column-Family Databases

Store data in columns grouped together rather than rows, which makes them very fast for reading huge amounts of specific columns. Examples: Cassandra, HBase.

Best for: massive write-heavy workloads and analytics — like time-series data, logs, and big-data systems.

4. Graph Databases

Store data as nodes and the connections (relationships) between them. Built to answer "how are these things connected?" extremely fast. Examples: Neo4j, Amazon Neptune.

Best for: social networks (friends-of-friends), recommendation engines, fraud detection — anything about relationships.

TypeStores data asGreat forExamples
Key-Valuekey → value pairsCaching, sessionsRedis, DynamoDB
DocumentJSON-like documentsProfiles, catalogsMongoDB
Column-familyGrouped columnsBig data, logsCassandra
GraphNodes & relationshipsSocial networksNeo4j

Seeing Each Type in Action

To make the four types stick, here is a concrete picture of each, plus when not to use it:

  • Key-Value: imagine a giant dictionary — session:abc123 → {userId: 42}. You can only look things up by the key; you cannot ask "find all sessions for user 42" efficiently. Avoid when you need to query by the contents of the value.
  • Document: a whole user profile (name, address, list of orders) stored as one JSON document you fetch in a single read. You can query inside documents. Avoid when data is highly relational with many cross-references — you'd duplicate a lot.
  • Column-family: think of a massive table optimised for writing and reading specific columns across billions of rows — perfect for "all sensor readings for device X over time". Avoid when you need flexible ad-hoc queries or joins.
  • Graph: data is people (nodes) connected by "follows"/"friends" (edges). Asking "friends of friends who like cricket" is one fast graph traversal — in SQL it would be a nightmare of joins. Avoid when your data is not really about relationships.

The Deeper Point: NoSQL Means "Designed Around Your Queries"

Query-first, not data-first

In SQL you model the data cleanly first, then query it however you like later. In NoSQL it is the opposite: you decide your access patterns first, then choose the type and shape your data so those exact queries are blazing fast. That is why "which NoSQL type?" always comes back to "how will you read and write this data?"

Common misconceptions, cleared

"NoSQL is one kind of database." No — these four families are as different from each other as a bicycle is from a boat. "NoSQL" just means "not the traditional relational table model".

"Redis is only a cache." Redis is a key-value store often used as a cache, but it also does queues, leaderboards (sorted sets), pub/sub, and can persist to disk.

"A graph DB is just for social networks." It shines anywhere relationships matter — fraud detection, recommendations, knowledge graphs, network/IT maps.

Interview tip

If you propose NoSQL, name the specific type and justify it by the access pattern: "Sessions are simple lookups by key, so a key-value store like Redis; the social graph needs friends-of-friends traversal, so a graph database." Saying just "NoSQL" is vague; naming the right family shows depth.

Key Takeaways

  • Key-Value (Redis): simplest, fastest, fetch by key — caching, sessions. Can't query by value.
  • Document (MongoDB): flexible JSON-like records — profiles, catalogs. Can query inside docs.
  • Column-family (Cassandra): huge write-heavy and analytics workloads across billions of rows.
  • Graph (Neo4j): relationships and connections — social networks, fraud, recommendations.
  • NoSQL is query-first: choose the type and data shape from how you will read and write.

#21ACID Properties

One-line definition

ACID is a set of four guarantees (Atomicity, Consistency, Isolation, Durability) that make database transactions safe and reliable — especially important for things like money, where mistakes are unacceptable.

A "transaction" is a group of database operations that should happen together as one unit. ACID is the promise that this unit behaves correctly even if things go wrong. SQL databases are famous for strong ACID guarantees.

The Classic Example: Bank Transfer

You transfer โ‚น500 from your account to a friend. This involves two steps: (1) subtract โ‚น500 from you, (2) add โ‚น500 to your friend. ACID makes sure this is safe.

The Four Letters

  • A — Atomicity: "all or nothing". Either both steps happen, or neither does. If the system crashes after step 1, the โ‚น500 is not lost — the whole transaction is undone.
  • C — Consistency: the database always moves from one valid state to another. Rules are never broken (e.g. total money in the system stays correct; no negative balances if not allowed).
  • I — Isolation: if many transactions happen at once, they do not mess each other up. Each behaves as if it ran alone. Two people transferring at the same time will not corrupt each other's data.
  • D — Durability: once a transaction is confirmed, it is saved permanently — even if the power goes out one second later, the data survives.
Atomicity in one picture

Imagine the bank deducts โ‚น500 from you, then crashes before adding it to your friend. Without atomicity, your โ‚น500 vanishes! With atomicity, the system rolls back the deduction — as if nothing happened. You try again safely.

Going Deeper on Isolation (the trickiest letter)

Atomicity and durability are fairly intuitive, but isolation is where most real bugs hide, because many transactions run at the same time. If isolation is weak, concurrent transactions can interfere and produce classic anomalies:

  • Dirty read: you read data another transaction wrote but has not committed yet — and then it rolls back, so you read something that "never happened".
  • Non-repeatable read: you read a row twice in one transaction and get different values, because someone changed it in between.
  • Phantom read: you run the same query twice and get extra rows the second time, because someone inserted new matching rows.

To control this trade-off between safety and speed, databases offer isolation levels:

Isolation levelPreventsTrade-off
Read UncommittedAlmost nothingFastest, least safe
Read CommittedDirty readsCommon default
Repeatable Read+ non-repeatable readsStronger
SerializableEverything (acts as if one-at-a-time)Safest, slowest

Higher isolation = more correctness but more locking and lower throughput. This is itself a tunable trade-off, much like consistency choices elsewhere.

How Durability Actually Works

How can a database promise your data survives a crash one millisecond after "success"? The trick is a write-ahead log (WAL): before applying a change, the database first writes it to an append-only log on durable disk. Even if the server crashes immediately after, on restart it replays the log to recover every confirmed transaction. So "confirmed" really means "safely written to the log on disk".

Common misconceptions, cleared

"The C in ACID is the same as the C in CAP." No! ACID's Consistency means "the database's rules/constraints are never violated". CAP's Consistency means "all nodes see the same latest data". They are different concepts that unfortunately share a letter.

"NoSQL has no ACID." Increasingly false — MongoDB now supports multi-document ACID transactions, and many modern databases offer it. ACID vs BASE is a spectrum, not a wall.

Interview tip

When money or inventory is involved, explicitly say "this needs ACID transactions" and mention you'd be careful about the isolation level (e.g. serializable for a critical balance check). Naming isolation levels signals genuine database depth.

Key Takeaways

  • ACID = four guarantees that make transactions safe: Atomicity, Consistency, Isolation, Durability.
  • Atomicity: all-or-nothing. Consistency: the DB's rules are never broken.
  • Isolation: concurrent transactions don't corrupt each other — tuned via isolation levels (Read Committed → Serializable).
  • Durability: confirmed data survives crashes, thanks to a write-ahead log.
  • Essential for money, orders, and critical data. (Note: ACID's "C" ≠ CAP's "C".)

#22BASE Properties

One-line definition

BASE is the relaxed alternative to ACID, used by many NoSQL databases. It trades strict, instant correctness for high availability and scale. The data becomes correct eventually, not necessarily the same instant.

ACID is strict and safe, but being that strict is hard to do across thousands of servers worldwide. So many large NoSQL systems use a more relaxed model called BASE, which favours staying available and fast over being perfectly consistent every millisecond.

What BASE Stands For

  • BA — Basically Available: the system stays up and responds, even if some parts are having trouble.
  • S — Soft state: the data might be "in flux" for a short time — different copies may temporarily disagree.
  • E — Eventually consistent: given a little time, all copies of the data will sync up and become the same.

Real Example: A Like Count

Instagram likes

When a post gets a like, you might see "1,000 likes" while your friend briefly sees "999". For a moment the copies disagree (soft state), but within a second or two they sync to the same number (eventually consistent). And the app never goes down to fix it (basically available). For a like count, this tiny delay is totally fine — and it lets the system serve billions of users.

ACID vs BASE

PointACIDBASE
PriorityCorrectness & safetyAvailability & scale
ConsistencyImmediate & strictEventual
Typical DBSQLNoSQL
Best forMoney, ordersFeeds, likes, logs, huge scale
The trade-off behind it

You usually cannot have perfect instant consistency AND huge scale AND constant availability all at once (the CAP theorem, Topic 29, explains why). ACID picks correctness; BASE picks availability and scale. Choose based on whether a brief disagreement in data is acceptable for your use case.

How "Eventually Consistent" Actually Catches Up

"Eventually" sounds vague, so here is what really happens under the hood. When you write data, it is accepted by one node and the system says "done" immediately (basically available). That node then copies the change to the other replicas in the background. During that gap, replicas disagree (soft state). They reconcile through mechanisms like:

  • Read repair: when a read notices two replicas disagree, it fixes the stale one on the spot.
  • Anti-entropy / gossip: nodes periodically compare data and patch differences in the background.
  • Hinted handoff: if a node was down during a write, another node holds a "hint" and delivers it once that node returns.

"Eventually" in healthy systems usually means milliseconds to a couple of seconds — not minutes.

The Question to Ask: "Is a Brief Disagreement Dangerous?"

This is the heart of choosing ACID vs BASE. Ask: if two users saw slightly different values for a second, would anything bad happen?

  • Like counts, view counts, follower numbers, feeds: a brief mismatch is harmless → BASE is perfect.
  • Bank balance, seat booking, inventory, payments: a brief mismatch could mean double-spending or overselling → you need ACID.
Common misconceptions, cleared

"Eventually consistent means unreliable / data can be lost." No — the data is safe and will become consistent; it is just not instantaneous across all copies. It is a timing property, not a "might lose your data" property.

"BASE is always faster and better for scale, so always use it." Only when your use case tolerates brief inconsistency. Using BASE for money is a classic, costly mistake.

Interview tip

Show you can mix both in one system: "I'll use ACID (SQL) for payments and order state, and BASE (Cassandra/DynamoDB) for the activity feed and like counts." Demonstrating that consistency is a per-feature decision is exactly the senior-level instinct interviewers reward.

Key Takeaways

  • BASE = a relaxed model favouring availability and scale over strict instant correctness.
  • Basically Available, Soft state, Eventually consistent.
  • Replicas sync via read repair, anti-entropy, and hinted handoff — usually within milliseconds.
  • Decide with one question: is a brief disagreement dangerous? If yes → ACID; if no → BASE.
  • ACID (SQL) = strict correctness; BASE (NoSQL) = scale and availability. Mix per feature.

#23Database Indexing

One-line definition

A database index is a special lookup structure that makes searching for data dramatically faster — just like the index at the back of a book helps you find a topic without reading every page.

As tables grow to millions of rows, finding a specific row by scanning every single one becomes painfully slow. An index solves this by keeping a sorted "shortcut" to the data.

Real-life Analogy: The Index of a Book

Finding a topic in a 1,000-page book

Without an index, finding "Photosynthesis" means flipping through all 1,000 pages one by one. With the index at the back, you jump straight to "Photosynthesis — page 423". A database index works exactly the same way: instead of scanning every row, the database uses the index to jump straight to the matching rows.

How It Works (Simply)

When you create an index on a column (say email), the database builds a separate, sorted structure of all the emails, each pointing to its row. Because it is sorted, the database can find any email almost instantly (using a fast search), instead of checking every row one at a time.

Without indexscan all 1,000,000 rows
vs
With indexjump straight to the row
An index turns a slow full scan into a fast direct lookup.

The Catch: Indexes Are Not Free

Indexes make reads faster, but they have costs:

  • They take up extra storage space.
  • They make writes slower — every time you add or change a row, the index must be updated too.
  • Too many indexes can hurt more than help.

So you index the columns you search by often (like email or user_id), not every column. This is a classic read-vs-write trade-off.

What's Inside an Index: the B-Tree

Most database indexes are built as a B-tree (balanced tree). Picture a tree where you start at the top and, at each step, decide "go left or right" to narrow down. Because each step roughly halves the search space, finding any value among a million rows takes only ~20 steps instead of a million. This is why an indexed lookup feels instant — the cost grows logarithmically, not linearly, as data grows. (Some databases also use hash indexes for exact-match lookups and inverted indexes for text search — Topic 80.)

Why Sort Order Matters

Because an index is sorted, it accelerates more than just exact matches. It also speeds up:

  • Range queries: "orders between Jan and March" — the sorted structure lets the DB jump to the start and read in order.
  • Sorting: ORDER BY created_at can read straight from the index instead of sorting everything.
  • Prefix matches: "names starting with 'Pri'".

Composite and Covering Indexes

  • Composite index: an index on multiple columns together, e.g. (user_id, created_at) — perfect for "this user's posts, newest first". Order matters: it helps queries that filter by user_id first.
  • Covering index: if an index contains all the columns a query needs, the database answers entirely from the index and never touches the table — extremely fast.
Real-world: the missing index outage

A very common production incident: a query was fine on 10,000 rows but the table grew to 50 million, and without an index every request triggered a full table scan. The database CPU pegged at 100%, queries timed out, and the whole app slowed to a crawl — fixed instantly by adding one index. "Add an index" is one of the most common real-world performance fixes.

Common misconceptions, cleared

"More indexes = faster database." Only for reads. Every index slows down every INSERT/UPDATE/DELETE and uses storage. Unused indexes are pure overhead.

"Indexes always get used." No — if a query uses a function on the column, or the optimiser decides a scan is cheaper, your index is ignored. Use EXPLAIN to check.

"The primary key has no index." It does — the primary key is automatically indexed.

Interview tip

When asked to make a slow query fast, your first instinct should be: "Which columns does it filter and sort by? Add an index on those (a composite index in the right order)." Then mention the write trade-off. Bonus points for naming B-trees and EXPLAIN.

Key Takeaways

  • An index is a sorted shortcut (usually a B-tree) that makes searching a column much faster.
  • It turns a slow "scan every row" into a fast logarithmic "jump straight to it".
  • Sorted order also speeds up range queries, sorting, and prefix matches.
  • Composite indexes cover multi-column queries; covering indexes answer entirely from the index.
  • Trade-off: faster reads, but extra storage and slower writes — index only what you query often.

#24Database Normalization

One-line definition

Normalization is the process of organising data into separate tables to remove duplication, so each piece of information is stored in only one place. This keeps data clean, consistent, and easy to update.

If you store the same information in many places, it gets messy fast — update one copy and forget another, and now your data disagrees with itself. Normalization prevents this by giving every fact a single home.

The Problem It Solves

Repeating data is dangerous

Suppose every order row stores the full customer name and address. A customer with 50 orders has their address repeated 50 times. If they move house, you must update all 50 rows — miss one, and your data is now wrong and inconsistent. Normalization fixes this: store the customer's details once in a Customers table, and have orders just refer to the customer by id.

How It Works

You split data into multiple related tables, each about one thing, and link them with ids. For example: a customers table, an orders table, and the order simply stores a customer_id pointing to the customer. The customer's address now lives in exactly one place.

The Normal Forms (Brief)

Normalization happens in steps called "normal forms". You do not need to memorise the rules deeply now, just the idea of progressively removing duplication:

  • 1NF (First Normal Form): each cell holds a single value (no lists crammed into one column).
  • 2NF: every column depends on the whole primary key, not just part of it.
  • 3NF: columns depend only on the key, not on other non-key columns.
  • BCNF: a stricter version of 3NF that handles some edge cases.

In practice, most well-designed databases aim for 3NF.

A Worked Example: Before and After

Imagine one giant orders table where every row repeats the customer's details:

order_idcustomer_namecustomer_cityproduct
1PriyaPuneShoes
2PriyaPuneBag
3PriyaMumbaiWatch

Notice row 3: someone updated Priya's city in one order but not the others. Now the data contradicts itself — which city is correct? This is called an update anomaly. After normalizing, Priya lives once in a customers table, and orders just store customer_id = 7. Change her city in one place and every order instantly reflects it. The contradiction becomes impossible.

The Three Anomalies Normalization Prevents

  • Update anomaly: changing a fact in some copies but not others (the Priya case above).
  • Insert anomaly: you can't add a new product because you don't yet have an order for it (the data is tangled together).
  • Delete anomaly: deleting the last order for a customer accidentally erases the customer's details entirely.

Pros and Cons

  • Pro: no duplicate data, so updates are easy and data stays consistent.
  • Pro: saves storage space and prevents the three anomalies above.
  • Con: reading data may require joining several tables, which can be slower.
When to deliberately break the rules

Normalization optimises for clean writes. But at huge read scale, all those joins get expensive. That is when teams intentionally denormalize (Topic 25) — reintroducing some duplication to avoid joins and make reads fast. Normalize first, then denormalize selectively where read performance demands it.

Common misconceptions, cleared

"More normalization is always better." No — over-normalizing forces too many joins and can hurt performance. 3NF is the usual sweet spot.

"Normalization applies to NoSQL too." Mostly the opposite — document/NoSQL designs often embed (denormalize) data on purpose so one read returns everything. Normalization is primarily a relational/SQL discipline.

Interview tip

Explain the trade-off in one breath: "Normalize to keep writes consistent and avoid anomalies; denormalize selectively for read-heavy paths." Mentioning the three anomalies (update/insert/delete) by name shows you understand why normalization exists, not just that it does.

Key Takeaways

  • Normalization = organising data into related tables so each fact is stored once.
  • It prevents the three anomalies: update, insert, and delete anomalies.
  • Done in steps called normal forms (1NF, 2NF, 3NF, BCNF); most aim for 3NF.
  • Trade-off: cleaner, consistent writes, but reads may need to join multiple tables (slower).
  • Normalize first, then denormalize selectively for read-heavy paths.

#25Database Denormalization

One-line definition

Denormalization is the opposite of normalization: you deliberately add some duplicate data back into your tables to make reads faster, by avoiding expensive joins.

Normalization removes duplication but can make reads slow (because you must join many tables). When read speed matters more than perfect tidiness, you do the reverse — denormalize — trading some duplication for speed.

Why Break the "No Duplication" Rule?

Showing a post with its author name

Normalized: to show "Post by Priya", the app reads the posts table, then joins the users table to get "Priya". Two tables, every time, for millions of posts. Denormalized: store the author's name directly inside the post row. Now showing the post needs no join — it is instant. The cost: if Priya changes her name, you must update it in many places.

The Core Trade-off

PointNormalizationDenormalization
DuplicationNoneSome, on purpose
ReadsSlower (joins)Faster (no joins)
Writes/updatesEasy & safeHarder (update many copies)
Best whenData changes oftenReads vastly outnumber writes
When to denormalize

Denormalize when your system is read-heavy (far more reads than writes), like a social feed that is viewed millions of times but rarely edited. The faster reads are worth the extra effort on the rare writes.

The Big Question: How Do You Keep the Copies in Sync?

The danger of duplication is data drifting out of sync. Real systems handle this in disciplined ways:

  • Application logic: when Priya changes her name, the code updates every place that copy lives.
  • Background jobs: a periodic job re-syncs denormalized copies from the source of truth.
  • Event-driven updates: a "user updated" event is published, and every service holding a copy listens and updates itself (very common in microservices).
  • Accept staleness: sometimes the copy is allowed to be slightly old — perfectly fine for things like a cached author name.

The golden rule: keep one source of truth, and treat the duplicates as fast-read copies that are kept up to date from it.

Real-world: the precomputed feed

When you open Instagram or Twitter, the app does not join dozens of tables live. Instead, your feed was precomputed and stored ready-to-serve (a denormalized "materialized" feed). Read = one fast lookup. The heavy work happened earlier, when posts were created and "fanned out" into followers' feeds. This is denormalization at massive scale — trading write-time work for instant reads.

Common misconceptions, cleared

"Denormalization is bad / a hack." No — it is a legitimate, deliberate optimization used by every large read-heavy system. The "mistake" is duplicating data accidentally without a plan to keep it in sync.

"Caching and denormalization are the same." Related but different: a cache is a temporary, separate fast copy; denormalization changes your actual stored schema to avoid joins.

Interview tip

For read-heavy features (feeds, profiles, product pages), proactively say: "I'd denormalize this to avoid joins on the hot read path, and keep the copies in sync via events / a background job." Always pair "I'll denormalize" with "here's how I keep it consistent" — that completeness is what impresses.

Key Takeaways

  • Denormalization = adding duplicate data on purpose to make reads faster (fewer joins).
  • It is the opposite trade-off of normalization: faster reads, harder writes.
  • Keep one source of truth and sync copies via app logic, background jobs, or events.
  • Powers precomputed feeds (Instagram/Twitter) — write-time work for instant reads.
  • Real systems balance both: normalize for safety, denormalize hot paths for speed.

#26Database Replication

One-line definition

Replication means keeping multiple copies of the same database on different servers. If one copy fails, others take over (reliability), and reads can be spread across copies (speed).

Having only one database is risky — if it dies, your whole app loses its data access. Replication makes copies so you are never dependent on a single database.

Real-life Analogy: Backup Singers

Never rely on one

If a concert had only one singer and they lost their voice, the show stops. With backup singers who know all the songs, the show goes on. Database replicas are backup copies that can step in — or sing along to share the load.

Master-Slave (Primary-Replica) Replication

The most common setup. One database is the master (also called primary) — all writes go to it. The master then copies its changes to one or more slave (replica) databases, which handle reads.

  • Benefit: reads are spread across many replicas — great for read-heavy apps.
  • Benefit: if the master fails, a replica can be promoted to become the new master.
  • Catch: replicas may lag slightly behind the master (eventual consistency).
Masterhandles writes
↓ copies changes
Replica 1reads
Replica 2reads
Writes go to the master; it copies data to replicas that serve reads.

Master-Master Replication

Here multiple databases can accept writes, and they sync with each other. This boosts write availability, but introduces a hard problem: if two masters change the same data at once, you get conflicts that must be resolved. It is more powerful but more complex.

Synchronous vs Asynchronous Replication (the key trade-off)

How quickly the master copies data to replicas decides your safety-vs-speed balance:

  • Synchronous: the master waits until the replica confirms it received the write before telling the client "done". Safe — no data loss if the master dies — but slower, because every write waits for replicas.
  • Asynchronous: the master replies "done" immediately and copies to replicas in the background. Fast, but if the master crashes before copying, those last writes are lost, and replicas serve slightly stale data (replication lag).
  • Semi-synchronous: a middle ground — wait for at least one replica to confirm, not all.

What Happens When the Master Dies: Failover

When the master fails, the system performs failover: it detects the failure (via health checks), promotes a replica to be the new master, and redirects writes to it. The catch:

  • With async replication, the promoted replica might be missing the very last writes — potential data loss.
  • If the old master comes back thinking it's still master, you can get split-brain (two masters) — dangerous (see Leader Election, Topic 65).
The read-your-own-write problem

A classic real bug from replication lag: a user updates their profile (write → master), then immediately reloads (read → a replica that hasn't caught up) and sees the old data — it looks like their change vanished! Fixes include routing a user's reads to the master for a short window, or reading from the master for critical "read-after-write" cases.

Common misconceptions, cleared

"Replication is a backup." No! Replication copies everything instantly — including a mistaken DELETE, which is replicated to all copies. You still need real, time-stamped backups for recovery.

"More replicas = faster writes." The opposite — replicas help read scaling. Writes still all go to the master (in master-slave) and synchronous replicas can even slow writes down.

Interview tip

When you propose read replicas, immediately mention replication lag and the read-your-own-write issue, and how you'd handle it. Distinguishing replication (copies, for reads/HA) from sharding (splits, for write scale) and from backups (recovery) shows real maturity.

Key Takeaways

  • Replication = multiple copies of the database for reliability and read scaling.
  • Master-Slave: writes to one master, reads from many replicas; a replica is promoted on failover.
  • Master-Master: multiple write nodes — higher availability but risk of conflicts and split-brain.
  • Sync = safe but slower; async = fast but risks lag and lost writes.
  • Watch for replication lag / read-your-own-write. Replication is not a backup.

#27Database Sharding

One-line definition

Sharding means splitting one huge database into smaller pieces (called shards), each stored on a different server. Each shard holds part of the data, so together they handle far more than one server ever could.

Replication copies the whole database. But what if the data is too big to fit or be handled by one server at all? Then you split it — that is sharding. It is how databases scale horizontally.

Real-life Analogy: Splitting a Phone Book

A-M in one book, N-Z in another

One giant phone book for a whole country is impossible to handle. So you split it: names A–M in Book 1, N–Z in Book 2. To find "Rahul", you know to grab Book 2. Each book is smaller and faster to search, and two people can search both at once. Sharding splits a database the same way.

How It Works

You pick a shard key (for example, user id) and use it to decide which shard a piece of data goes to. For instance, users 1–1,000,000 on Shard A, users 1,000,001–2,000,000 on Shard B, and so on. When you need a user, the system uses the shard key to go straight to the right shard.

Huge DB
→
Shard Ausers 1–1M
Shard Busers 1M–2M
Shard Cusers 2M–3M
Data is split across shards by a shard key, each on its own server.

Benefits and Challenges

  • Benefit: handles enormous data and traffic — just add more shards.
  • Benefit: each shard is smaller, so queries on it are faster.
  • Challenge: queries that need data from many shards become complex.
  • Challenge: choosing a bad shard key can make one shard much busier than others (a "hot shard").
Replication vs Sharding (do not confuse them)

Replication = full copies of the same data (for safety & read scaling). Sharding = splitting different data across servers (for size & write scaling). Big systems often use both together.

Choosing the Shard Key Is Everything

The shard key is the single most important decision in sharding, because it decides how evenly data and traffic spread. A good shard key has two properties:

  • High cardinality & even distribution: it spreads data roughly equally across shards. Hashing the user_id usually does this well.
  • Matches your queries: your most common queries should be answerable from a single shard. If you shard by user_id and most queries are "get this user's data", they hit one shard — perfect.

The Hot Shard Problem

When one shard melts

Suppose you shard by country. India has far more users than most, so the India shard gets overwhelmed while others sit idle — a hot shard. Same thing if you shard a social app by celebrity: the shard holding a megastar with 200M followers is hammered. The fix is a shard key that spreads load evenly (often a hash of the id), so no single shard becomes a bottleneck.

Why Resharding Is Painful (and How to Avoid It)

If you naively assign shards with user_id % number_of_shards, then adding one shard changes the answer for almost every key — meaning you must move enormous amounts of data. This is why large systems use consistent hashing (Topic 66), which lets you add/remove shards while moving only a small fraction of data. Plan capacity early; resharding a live system is one of the hardest operations in practice.

Real-world: who shards

Instagram shards its PostgreSQL by user id across many machines. Discord shards trillions of messages in Cassandra by channel + time bucket. Uber, Slack, and virtually every internet-scale company shard their core data. Sharding is the standard answer when a single database can no longer hold or serve the load.

Common misconceptions, cleared

"Shard early to be safe." Usually wrong — sharding adds big complexity (cross-shard queries, no easy joins, harder transactions). Scale up vertically and use replicas first; shard only when you truly must.

"Sharding and partitioning are the same." Sharding is partitioning across separate servers; partitioning (Topic 28) can also happen within one server.

Interview tip

When you say "I'll shard", always state the shard key and why it avoids hot shards and keeps common queries on one shard. Mention that cross-shard queries and transactions are hard, and that consistent hashing eases resharding. That's the full, senior-level answer.

Key Takeaways

  • Sharding = splitting a big database into pieces (shards) across servers, each holding part of the data.
  • The shard key is critical: spread data evenly and keep common queries on a single shard.
  • A bad key causes hot shards; naive modulo sharding makes resharding very painful (use consistent hashing).
  • Cross-shard queries, joins, and transactions are hard — shard only when you must.
  • Replication = copies; Sharding = splits. Often used together.

#28Database Partitioning

One-line definition

Partitioning means dividing a large table into smaller, more manageable parts. Sharding is actually one kind of partitioning (across different servers); partitioning is the broader idea of splitting data logically.

As tables get huge, working with them as one block is slow and unwieldy. Partitioning breaks a big table into smaller parts that are easier and faster to manage — sometimes on the same server, sometimes across many.

Two Directions of Splitting

  • Horizontal partitioning — split by rows. Put some rows in one partition, other rows in another (e.g. 2023 orders in one part, 2024 orders in another). This is what sharding does across servers.
  • Vertical partitioning — split by columns. Put frequently used columns in one part and rarely used ones in another (e.g. keep a user's name/email separate from their long profile bio).

Common Strategies (How to Decide the Split)

StrategyHow it splitsExample
RangeBy a range of valuesOrders Jan–Jun vs Jul–Dec
HashBy a hash of the key (even spread)user_id hashed into 4 buckets
ListBy a fixed list of valuesRegion = India / USA / Europe

Why Partition?

  • Faster queries (partition pruning): if you ask for "March orders" and the table is partitioned by month, the database skips every other partition and reads only March. This skipping is called partition pruning and is the main performance win.
  • Easier management: you can back up or delete old partitions instantly. Dropping last year's data is one fast DROP PARTITION instead of a slow, giant DELETE across the whole table.
  • Foundation for scaling: horizontal partitioning across servers is sharding.
Real-world: time-series and logs

Systems that store events by time — logs, metrics, IoT sensor data, financial ticks — almost always range-partition by time (e.g. one partition per day or month). Queries are usually "the last 24 hours", so only recent partitions are touched, and ancient data is dropped by retiring whole partitions. This is exactly how time-series databases (Topic 79) stay fast.

Partitioning vs Sharding: the precise difference

People use these words loosely, so be precise: partitioning is the general idea of splitting a table into parts. When those parts live on the same server, it's just partitioning (it helps query speed and management). When those parts are spread across different servers, it becomes sharding (it adds scale by using many machines). So: all sharding is partitioning, but not all partitioning is sharding.

Common misconceptions, cleared

"Partitioning always speeds up every query." Only queries that filter on the partition key benefit (via pruning). A query that ignores the partition key may have to scan all partitions — sometimes slower.

"Vertical partitioning = normalization." Related but not identical — vertical partitioning splits columns of the same entity for performance/storage; normalization splits data by relationships to remove duplication.

Interview tip

For time-based or log-like data, proactively suggest "range partition by time so old data drops cheaply and recent queries prune to one partition". Then clarify the partitioning-vs-sharding distinction in one line — interviewers love when you untangle these two commonly-confused terms.

Key Takeaways

  • Partitioning = dividing a big table into smaller, manageable parts.
  • Horizontal = split by rows; Vertical = split by columns. Strategies: Range, Hash, List.
  • Partition pruning makes queries fast by reading only relevant partitions; old partitions drop instantly.
  • Ideal for time-series/log data (range by time).
  • All sharding is partitioning, but not all partitioning is sharding (sharding = partitions across servers).

#29CAP Theorem

One-line definition

The CAP Theorem says that a distributed system can guarantee only two out of three things at the same time: Consistency, Availability, and Partition tolerance. You must choose which two matter most.

This is one of the most famous ideas in system design. It explains why you cannot have everything, and it forces a clear choice. Let us define the three words simply first.

The Three Letters

  • C — Consistency: every user sees the same, latest data at the same time.
  • A — Availability: the system always responds to requests (it is never "down").
  • P — Partition tolerance: the system keeps working even if the network breaks between servers (messages get lost between them).

Why You Can Only Pick Two

In any real distributed system spread across servers, network failures will happen — so Partition tolerance (P) is mandatory. That leaves a real choice between C and A during a network break:

CP — choose Consistency

When the network breaks, the system refuses to answer rather than give possibly-wrong data. It stays correct but may become temporarily unavailable.

Example: banking systems — better to reject than show a wrong balance.

AP — choose Availability

When the network breaks, the system keeps answering with whatever data it has, even if slightly stale. It stays up but may be briefly inconsistent.

Example: social feeds — better to show a slightly old like count than go down.

Simple way to feel it

Two copies of your data lose contact with each other (a partition). Someone updates one copy. Now: do you block reads on the other copy until they sync (consistency, CP)? Or do you serve the old value to stay available (availability, AP)? You cannot do both perfectly at that moment.

In practice

Most SQL databases lean CP (correctness first). Many NoSQL databases (Cassandra, DynamoDB) lean AP (availability first). The right pick depends on whether your app can tolerate brief stale data.

A Concrete Partition: Two ATMs, One Cut Cable

Imagine your bank balance is replicated on two servers, and the network link between them is cut (a partition). You have โ‚น1,000. You withdraw โ‚น1,000 at ATM-1 while your sibling withdraws โ‚น1,000 at ATM-2 — and the two servers can't talk to confirm.

  • CP choice: at least one ATM refuses ("service unavailable, try later"). No double withdrawal, but you lost availability for a moment. Banks pick this.
  • AP choice: both ATMs say yes to stay available — now โ‚น2,000 left an account with โ‚น1,000. The conflict gets reconciled later. A shopping cart might accept this; a bank never would.

This single scenario is the entire CAP theorem made tangible: during a partition, you either sacrifice correctness or sacrifice availability — you can't keep both.

Common misconceptions, cleared

"You permanently pick 2 of 3 forever." No — the choice between C and A only matters during a partition. When the network is healthy, a well-designed system gives you both. (This is exactly what PACELC, Topic 30, refines.)

"You can choose to drop Partition tolerance (pick CA)." Not for a real distributed system — networks will fail, so P is mandatory. "CA" only describes a single-node system.

"CAP's Consistency = ACID's Consistency." No — CAP's C means "all nodes see the same latest value"; ACID's C means "constraints aren't violated". Same letter, different meaning.

Interview tip

Don't recite "pick 2 of 3". Instead say: "P is a given, so the real question is: during a network partition, does this feature need to stay correct (CP) or stay available (AP)?" Then answer per feature — payments CP, feed/likes AP. That framing is what separates a strong answer from a memorised one.

Key Takeaways

  • CAP: a distributed system can fully guarantee only two of Consistency, Availability, Partition tolerance.
  • Network breaks are inevitable, so Partition tolerance is required — the real choice is C vs A during a partition.
  • CP = stay correct, may go unavailable (banking). AP = stay up, may be stale (social feeds).
  • The C/A choice only applies during a partition; healthy systems give both.
  • SQL often leans CP; many NoSQL lean AP. Choose per feature, based on tolerance for stale data.

#30PACELC Theorem

One-line definition

PACELC extends the CAP theorem with a missing piece. It says: if there is a Partition (P), choose between Availability and Consistency (A/C) — Else (E), when things are normal, choose between Latency and Consistency (L/C).

CAP only talks about what happens during a network failure. But failures are rare — what about the other 99% of the time when everything is fine? PACELC fills that gap, which makes it more complete and realistic.

Reading the Name

PACELC is literally a sentence: Partition → A or C, Else → L or C.

  • If a Partition (network break) happens: trade off Availability vs Consistency (this is the original CAP choice).
  • Else (normal operation, no failure): trade off Latency vs Consistency — do you want faster responses (low latency) or perfectly up-to-date data (consistency)?

Why the "Else" Part Matters

The everyday trade-off CAP ignores

Even with no failures, keeping every copy of your data perfectly in sync takes time (you wait for all copies to confirm). If you want the lowest latency, you might answer immediately from the nearest copy without waiting — risking slightly stale data. So even on a normal day, you are choosing between speed and perfect consistency. That is the insight PACELC adds.

Examples

SystemDuring PartitionNormal (Else)
Cassandra / DynamoDBAvailability (AP)Latency (EL) — speed first
Traditional SQLConsistency (CP)Consistency (EC) — correctness first
Google SpannerConsistency (CP)Consistency (EC) — pays latency for it

Why This Reflects Reality Better

Partitions are rare — maybe a few minutes a year. CAP only describes those rare minutes. But the latency-vs-consistency tension is present in every single request of a replicated system, all day, every day. If your data lives in 3 regions and you insist every write be confirmed by all 3 before returning, every write pays cross-region latency. If you return after one region confirms, you're fast but might read slightly stale data elsewhere. PACELC names this everyday trade-off that CAP simply ignores.

Common misconceptions, cleared

"PACELC replaces CAP." It extends it. The "PAC" part is CAP; the "ELC" part is the new addition for normal operation.

"Low latency always means weak consistency." Not always — systems like Spanner achieve strong consistency globally, but they accept higher latency (the EC choice) to do it. You're choosing where on the dial to sit, not flipping a switch.

Interview tip

Mentioning PACELC unprompted signals senior-level depth. Use it like this: "This is an EL system — even with no partition, I'll favour low latency over strict consistency, since stale-by-a-few-ms is fine for this feed." It shows you think about the 99% of normal time, not just failures.

Key Takeaways

  • PACELC completes CAP by also covering normal (non-failure) operation.
  • If Partition: choose Availability or Consistency. Else: choose Latency or Consistency.
  • Key insight: even with no failures, keeping data perfectly in sync costs latency — and this applies to every request.
  • Classify systems as e.g. PA/EL (Cassandra) or PC/EC (SQL, Spanner).
  • A more realistic model than CAP for designing real distributed databases.

#31What is Caching?

One-line definition

Caching means storing a copy of frequently-used data in a fast, temporary place (a cache) so that future requests for it are served much faster — instead of fetching it from the slow original source every time.

Caching is one of the most powerful and common ways to make systems fast. The idea is simple: if you keep needing the same thing, keep a copy close by so you do not fetch it from far away each time.

Real-life Analogy: Keeping Snacks on Your Desk

Desk drawer vs the shop

If you want a snack, walking to the shop every time (slow) is wasteful. So you keep a few snacks in your desk drawer (fast, nearby). You only go to the shop when the drawer is empty. The drawer is your cache — small, fast, and holds the things you reach for most.

How It Works: Cache Hit vs Cache Miss

  • Cache hit: the data is found in the cache — returned instantly. ๐ŸŽ‰
  • Cache miss: the data is not in the cache — the system fetches it from the slow source (like the database), returns it, and usually stores a copy in the cache for next time.

The goal is a high cache hit ratio — as many requests as possible served from the fast cache.

Request
→
Cache?in here?
hit →
Return fast
miss →
Databaseslow, then store copy
On a hit, return instantly. On a miss, fetch from the database and cache it for next time.

Why Caching Is So Valuable

  • Speed: memory-based caches (like Redis) are far faster than disk-based databases.
  • Less load: the database does far less work, since the cache absorbs most reads.
  • Scale: caching lets a system serve many more users with the same database.
The hard part of caching

Caches hold copies, so the copy can become stale (out of date) when the real data changes. Deciding when to refresh or remove cached data (covered in the next two topics) is famously one of the trickiest problems in computing. As the joke goes: "There are only two hard things in computer science: cache invalidation and naming things."

Just How Much Faster? (feel the numbers)

Caching is not a small optimisation — the speed gap is enormous. Rough real-world numbers:

  • Reading from memory (RAM / Redis): well under 1 millisecond.
  • Reading from an SSD database: a few milliseconds.
  • Reading from disk / a complex query / another service: tens to hundreds of milliseconds.

So a cache hit can be 100–1000× faster than going to the source. That is why caching is often the single highest-impact performance change you can make.

Caches Live at Many Layers

Caching is not one box — it happens all along the path of a request (you'll see this again in Topic 36):

  • Browser cache on the user's device (images, JS, CSS).
  • CDN at the edge, near the user (Topic 18).
  • Application/server cache (in-memory or Redis/Memcached) for query results and computed data.
  • Database cache (the DB keeps hot pages in memory itself).

What Should You Cache?

Cache data that is read often but changes rarely, and is expensive to produce — product details, user profiles, configuration, computed feeds, the result of a heavy query. Avoid caching data that changes every second or is unique to every single request (low reuse = low hit ratio = little benefit).

Common misconceptions, cleared

"Cache everything to make everything fast." No — caching rarely-reused data wastes memory and adds staleness risk for no benefit. Cache the hot, reusable data.

"A cache is permanent storage." No — caches are temporary and can be evicted or lost at any time. The source of truth is still your database; the cache must be rebuildable from it.

"Caching is free." It adds complexity: stale data, invalidation logic, and a new failure mode (what happens when the cache goes down? — see thundering herd, Topic 33).

Interview tip

In almost any read-heavy design, propose caching early and be specific: "I'll cache hot product data in Redis with a TTL; expected hit ratio ~90%, which cuts DB load by 10×." Then acknowledge the hard part — invalidation and staleness — to show you know caching isn't free.

Key Takeaways

  • Caching = keeping copies of hot data in a fast, temporary store to serve requests quickly.
  • Cache hit = found in cache (fast); cache miss = fetch from source and store it. Aim for a high hit ratio.
  • A hit can be 100–1000× faster; caches exist at browser, CDN, app, and DB layers.
  • Cache data that is read-often, changes-rarely, expensive to produce — not everything.
  • The main challenge is stale data / invalidation — caching is powerful but not free.

#32Cache Strategies

One-line definition

A cache strategy is the rule for how data is read from and written to the cache and the database, so they stay in sync. Different strategies balance speed, freshness, and complexity differently.

Once you have a cache, you must decide: when do you read from it, and when do you update it? These patterns answer that. The main four are below.

Read Strategy: Cache-Aside (most common)

The app checks the cache first. On a miss, it reads from the database, stores the result in the cache, and returns it. The app manages the cache itself. Simple and widely used.

Write Strategies

  • Write-Through: every write goes to the cache and the database at the same time. The cache is always fresh, but writes are a bit slower (two writes). Great when you read data soon after writing it.
  • Write-Back (Write-Behind): writes go to the cache first and are saved to the database later (in the background). Very fast writes, but risky — if the cache fails before saving, data can be lost.
  • Write-Around: writes go straight to the database, skipping the cache. The cache is only filled on later reads. Good when written data is rarely read again soon (avoids filling the cache with cold data).
StrategyHow writes workBest for
Write-ThroughCache + DB togetherData read soon after writing; freshness
Write-BackCache now, DB laterVery fast writes (accepts some risk)
Write-AroundDB only, skip cacheRarely re-read data
Quick intuition

Write-Through = update your notebook and the official record together (safe, slightly slower). Write-Back = jot in your notebook now, copy to the official record later (fast, but risky if you lose the notebook). Write-Around = write straight to the official record and only note it later if you actually need it.

Cache-Aside in Detail (and how you keep it fresh)

Cache-aside (also called "lazy loading") is the pattern you'll use 90% of the time, so understand its full lifecycle:

  • Read: check cache → hit returns instantly; miss reads the DB, stores it in cache, returns it.
  • Write/update: write to the DB, then invalidate (delete) the cache key. The next read will miss and reload fresh data.

Notice the subtle rule: on a write, you usually delete the cached value rather than updating it. Deleting is safer because two concurrent updates writing into the cache can race and leave a stale value, whereas "delete and let the next read reload" avoids that trap.

Two Classic Cache Failures to Know

  • Cache stampede / thundering herd: a hot key expires, and thousands of requests all miss at once and hammer the DB together. Fixes: add a small random jitter to TTLs, or let only one request rebuild the value while others wait.
  • Cache penetration: requests for data that doesn't exist always miss the cache and hit the DB every time. Fix: cache the "not found" result (a null) briefly, or use a Bloom filter (Topic 99).
Real-world mapping

Cache-aside + TTL powers most web apps (product pages, profiles). Write-through suits data you read right after writing (a user's own just-updated settings). Write-back shows up in high-throughput counters and metrics buffers where occasional loss is acceptable. Write-around fits write-heavy logs you rarely re-read.

Common misconceptions, cleared

"The cache and DB are always perfectly in sync." Never fully — there is always a tiny window where they can differ. You design to make that window small and harmless, not zero.

"On update, always rewrite the cache." Often better to delete the key and let the next read repopulate it — safer against concurrent-write races.

Interview tip

Default to "cache-aside with a TTL, and invalidate on write" — it's the expected answer. Then earn bonus points by mentioning cache stampede and how you'd prevent it (jittered TTLs / single-flight rebuild). That shows you've run caches in production, not just read about them.

Key Takeaways

  • Cache strategies define how cache and database stay in sync.
  • Cache-Aside: app checks cache, fetches from DB on miss, caches it, and invalidates on write (the default).
  • Write-Through (fresh, safe), Write-Back (fast, risky), Write-Around (skip cache on write).
  • Watch for cache stampede (hot key expiry) and penetration (missing keys) — and their fixes.
  • Choice depends on your read/write patterns and tolerance for risk.

#33Cache Eviction Policies

One-line definition

A cache is small and fills up, so when it is full and you need to add something new, you must remove (evict) something old. An eviction policy is the rule for choosing what to remove.

Caches are deliberately small and fast, so they cannot hold everything. When full, something must go. Picking the right thing to remove keeps the most useful data in the cache.

The Main Policies

  • LRU (Least Recently Used): remove the item that has not been used for the longest time. The most popular policy — the logic is "if you have not needed it in a while, you probably will not soon".
  • LFU (Least Frequently Used): remove the item used the fewest times. Keeps the most popular items, even if not used very recently.
  • FIFO (First In, First Out): remove the oldest-added item, regardless of usage. Simple but not very smart.
  • Random: remove a random item. Rarely used, but simplest and cheap.
Analogy: cleaning out your wardrobe

Your wardrobe is full and you bought a new shirt. What do you remove? LRU: the shirt you have not worn in the longest time. LFU: the shirt you wear least often overall. FIFO: the oldest shirt you bought. Most people naturally use LRU — and so do most caches.

PolicyRemovesIdea
LRULeast recently usedDrop what you have not touched lately
LFULeast frequently usedKeep the most popular items
FIFOOldest addedSimple age-based
RandomAny random itemCheapest, no tracking

There is also TTL (Time To Live) — not exactly an eviction policy, but a rule that automatically removes an item after a set time, so cached data does not stay stale forever.

Eviction vs Expiry (don't mix them up)

  • Expiry (TTL): removes data because it's too old — mainly about freshness/staleness. Happens whether the cache is full or not.
  • Eviction (LRU/LFU/...): removes data because the cache is full — mainly about limited space. Happens only under memory pressure.

In practice you use both: a TTL caps staleness, and an eviction policy handles "we're out of room".

When LRU Fails: the scan problem

LRU is a great default, but it has a famous weakness. Imagine a one-time bulk job that reads millions of rows once (a "scan"). Pure LRU will treat each newly-read item as "most recently used" and evict your genuinely-hot data to make room for this one-time junk — trashing your hit ratio. This is why real systems (like Redis) often use smart variants: LFU (keeps truly popular items) or LRU-K / segmented LRU that require an item to be used a few times before it's considered hot.

Real-world: what Redis actually offers

Redis lets you choose the policy via maxmemory-policy: allkeys-lru, allkeys-lfu, volatile-ttl (evict keys closest to expiring), and more. LFU was added precisely because pure LRU struggled with the scan problem on real workloads.

Common misconceptions, cleared

"TTL and eviction are the same thing." No — TTL is about age/freshness; eviction is about running out of space. They solve different problems.

"LRU is always best." Usually good, but it suffers under scan-heavy workloads — LFU or hybrid policies can keep a higher hit ratio there.

Interview tip

Say "LRU as the default eviction policy, plus a TTL to bound staleness." If the workload includes big scans or analytics, mention switching to LFU to protect hot keys. Knowing why you'd deviate from LRU is the senior signal.

Key Takeaways

  • When a cache is full, an eviction policy decides what to remove.
  • LRU (least recently used) is the most common and usually a great default.
  • LFU keeps the most popular (and resists the scan problem); FIFO removes oldest; Random is simplest.
  • Eviction (out of space) is different from expiry/TTL (too old) — real systems use both.

#34Redis vs Memcached

One-line definition

Redis and Memcached are the two most popular tools used as caches (they store data in fast memory). Memcached is simple and pure-cache; Redis is more powerful with extra features.

When you decide to add a cache, these are the two names you will hear most. Both keep data in RAM (very fast). The difference is how much they can do.

Memcached — Simple and Fast

Memcached is a straightforward key-value cache. It stores simple string values in memory and does that one job extremely well. It is lightweight and easy.

  • Only simple key-value strings.
  • Very simple, slightly faster for the basic case, easy to scale across machines.
  • No saving to disk — if it restarts, the cache is empty.

Redis — Powerful and Feature-Rich

Redis is also a fast in-memory store, but it can do much more than plain caching.

  • Supports rich data structures: lists, sets, sorted sets, hashes, counters.
  • Can save data to disk (persistence), so it survives restarts.
  • Supports replication, pub/sub messaging, and more — often used beyond caching.
PointMemcachedRedis
Data typesSimple key-value onlyMany (lists, sets, counters…)
Persistence (disk)NoYes (optional)
Extra featuresMinimalPub/sub, replication, scripting
Best forSimple, pure cachingCaching + richer needs
Which to pick?

If you just need a simple, fast cache, Memcached is great. If you want extra power (data structures, persistence, leaderboards, queues), choose Redis. Today, Redis is the more popular default because of its flexibility.

Beyond Caching: What Redis's Data Structures Unlock

The reason Redis dominates is that its data structures let it act as more than a cache — a single tool for many jobs:

  • Sorted sets → real-time leaderboards and ranking (game scores, trending posts).
  • Lists → simple queues and recent-activity feeds.
  • Hashes → storing objects (a user's fields) compactly.
  • Counters (INCR) → atomic rate limiters and view counts.
  • Pub/Sub & Streams → lightweight messaging and event feeds.
  • TTL + atomic ops → distributed locks and session stores.

Memcached's One Real Advantage: Multithreading

Memcached is multithreaded, so on a big multi-core machine it can use all cores for simple get/set at very high throughput. Classic Redis is largely single-threaded for command execution (simpler, avoids locks, still extremely fast). So for a pure, massive, simple key-value cache on beefy hardware, Memcached can edge ahead — but for almost everything else, Redis's features win.

Common misconceptions, cleared

"Redis is just a cache." No — it's an in-memory data-structure store used as a cache, message broker, rate limiter, leaderboard, session store, and more.

"Redis persistence makes it a safe primary database." Be careful — persistence (RDB snapshots / AOF) helps survive restarts, but Redis is memory-first; treat it as a cache/secondary store unless you've specifically designed for durability.

"Memcached is outdated." No — it's still excellent and widely used for simple high-throughput caching; it's just less versatile.

Interview tip

Default to "Redis" and justify with a concrete use beyond caching ("I'll also use it for the rate limiter and the leaderboard via sorted sets"). Only pick Memcached when the requirement is explicitly a simple, huge, pure key-value cache — and mention its multithreading as the reason.

Key Takeaways

  • Both are fast in-memory stores used for caching.
  • Memcached = simple, multithreaded, pure key-value cache, no persistence.
  • Redis = rich data structures (sorted sets, lists, counters), optional persistence, pub/sub — a multi-tool.
  • Redis's structures power leaderboards, queues, rate limiters, locks, sessions, not just caching.
  • Redis is the popular default; Memcached shines for pure, high-throughput simple caching.

#35CDN Caching

One-line definition

CDN caching is caching applied at the edge of the network — storing copies of content on CDN servers around the world (Topic 18), so users get it from a nearby location instead of the distant origin server.

We met CDNs earlier. This topic connects them to caching: a CDN is essentially a giant, globally-distributed cache for your static content.

How CDN Caching Works

The first time someone in a region requests, say, an image, the nearby edge server does not have it yet (a miss), so it fetches it from your origin and caches it. Every later user in that region gets the cached copy instantly (hits) — without ever touching your origin.

1st usermiss
→
Edge cache
→
Originfetch once
Later usershits
→
Edge cacheserved instantly
The origin is hit once; after that, the edge cache serves everyone in that region.

Key Ideas: TTL and Cache Invalidation

  • TTL (Time To Live): each cached item has an expiry time. After it expires, the edge re-fetches a fresh copy from the origin. This balances freshness vs speed.
  • Cache invalidation / purge: if content changes before its TTL (e.g. you fix a typo in an image), you can manually tell the CDN to clear that item so users get the new version.
  • Cache busting: a common trick — change the file's name/URL (like logo.v2.png) so the CDN treats it as new content.

Cache Busting Is the Trick You'll Actually Use

Purging a CDN globally is slow and not instant. So the professional approach is cache busting via content hashing: build tools rename files to include a hash of their contents, e.g. app.4f9a2c.js. When the file changes, the hash changes, so the URL changes — and a new URL is automatically a cache miss that fetches the new version. Meanwhile you can set a very long TTL (a year) on these files, because a changed file always gets a brand-new name. This is why you can have both "cache forever" and "instant updates".

Static vs Dynamic, and Edge Compute

Classic CDN caching is for static assets (images, CSS, JS, videos). But modern CDNs (Cloudflare, Fastly, CloudFront) increasingly cache dynamic content too — even running code at the edge (edge functions) to personalise or assemble pages near the user. The line between "CDN" and "compute" is blurring.

Common misconceptions, cleared

"After I deploy, users instantly see the new version." Not if it's cached with a long TTL and the same filename — they'll see the old one until it expires or you purge. Use cache busting to avoid this surprise.

"CDNs only cache; they don't help reliability." They also absorb traffic spikes and DDoS, and keep serving cached content even if your origin is temporarily down.

Interview tip

When serving static assets, say: "long TTL at the CDN plus content-hashed filenames for cache busting" — this one sentence shows you understand how to cache aggressively and ship updates instantly, a very common real-world requirement.

Key Takeaways

  • A CDN is a worldwide cache for static content at the network edge.
  • First request is a miss (fetched from origin and cached); later ones are fast hits.
  • TTL controls freshness; content-hashed cache busting enables "cache forever + instant updates".
  • Modern CDNs also cache some dynamic content and run edge compute.
  • Bonus: CDNs add reliability — absorbing spikes and serving during origin outages.

#36Client, Server & Database Caching (Caching Layers)

One-line definition

Caching does not happen in just one place — it happens at many layers along the path from the user to the database. Each layer that caches reduces work for the layers behind it.

Data travels from the user's device, across the network, to your servers, and finally to the database. A cache can sit at every step of this journey. Understanding the layers helps you place caches wisely.

The Caching Layers (from user to database)

  • Client-side cache: stored on the user's own device. The browser caches images, CSS, and files so revisiting a page is instant. Fastest of all — no network needed.
  • CDN cache: at the network edge, near the user (Topic 35). Serves static content without reaching your servers.
  • Server-side / application cache: on your servers (often Redis/Memcached). Stores results of expensive computations or frequent queries.
  • Database cache: the database itself keeps recently-used query results and data pages in memory to answer repeat queries faster.
Browser cache
→
CDN cache
→
App cache (Redis)
→
DB cache
A request can be answered at any layer; the earlier it is answered, the faster and cheaper.

The Golden Principle

Cache as close to the user as possible

The earlier (closer to the user) a request is answered, the faster it is and the less load it puts on everything behind it. A request served by the browser cache never even touches the network; one served by the CDN never touches your servers. Each layer protects the next.

The Funnel: Each Layer Absorbs a Share

Think of the layers as a funnel that thins traffic at every step. Suppose 1,000,000 requests arrive:

  • Browser cache handles, say, 40% → 600,000 reach the network.
  • CDN handles 80% of those → 120,000 reach your servers.
  • App cache (Redis) handles 90% of those → 12,000 reach the database.
  • DB cache handles most of those in memory → only a trickle hits disk.

So a million requests became twelve thousand DB queries. This layered funnel is exactly how a modest database survives internet-scale traffic. Removing any one layer dramatically increases load on the next.

The Cost: Many Layers = Many Places for Staleness

Every layer holds a copy, so an update must propagate through all of them. A user might see old content because their browser cached it, even after your CDN and server are updated. This is why short, sensible TTLs and cache-busting matter at each layer — the more layers you cache at, the more carefully you manage freshness.

Common misconceptions, cleared

"One big cache is enough." Layers each solve different problems — the browser saves the network trip, the CDN saves the server trip, Redis saves the DB trip. They compound.

"If a cache layer goes down, only that layer's speed is lost." Worse — its entire load suddenly falls on the next layer, which can cascade into an overload (a real cause of outages).

Interview tip

When asked "how do you handle huge read traffic?", walk the layers from the user inward: browser → CDN → app cache (Redis) → DB. State the rough hit ratio at each and conclude "so the DB only sees a tiny fraction". This funnel narrative is a textbook strong answer.

Key Takeaways

  • Caching happens at multiple layers: client (browser), CDN, server/app, and database.
  • The layers form a funnel — each absorbs a share, so the DB sees only a tiny fraction of traffic.
  • Each layer that answers a request shields the layers behind it; losing one can overload the next.
  • Golden rule: cache as close to the user as possible for maximum speed and minimum load.
  • Trade-off: more layers = more places where data can be stale, so manage TTLs/busting carefully.

#37Synchronous vs Asynchronous Communication

One-line definition

Synchronous means you send a request and wait for the reply before doing anything else. Asynchronous means you send a request and carry on — the reply comes later, whenever it is ready.

This is about whether you wait or not. It is one of the most important ideas for building responsive, scalable systems, because waiting too much makes systems slow and fragile.

Real-life Analogy: Phone Call vs Text Message

Wait on the line, or text and move on

Synchronous = a phone call: you call, and you must stay on the line and wait for the other person to respond. You cannot do anything else until they answer. Asynchronous = a text message: you send it and immediately go back to your work; they reply when they can, and you read it later.

Synchronous Communication

The caller is blocked (stuck waiting) until the response arrives. Simple to understand and code, and you get the answer immediately — but if the other side is slow, you are stuck too.

  • Good for: when you truly need the answer right now (e.g. checking a password to log in).
  • Bad when: the task is slow — waiting wastes time and can cause a chain of delays.

Asynchronous Communication

The caller sends the request and does not wait — it continues working, and the result is handled later (via a callback, a message queue, or a notification).

  • Good for: slow tasks like sending emails, processing videos, generating reports — the user should not wait for these.
  • Benefit: the system stays responsive and can handle far more work.
  • Cost: more complex — you must handle results arriving later and possible failures.
Real example: uploading a video to YouTube

When you upload a video, YouTube does not make you wait while it processes the video into all qualities (360p, 720p, 1080p). It says "upload complete" quickly and processes the rest asynchronously in the background. Minutes later, the qualities appear. Imagine if you had to stare at a loading screen the whole time — that is what synchronous would force.

Why Synchronous Chains Are Dangerous at Scale

The real risk of synchronous calls appears when services call services that call other services. Suppose request A waits on B, B waits on C, and C waits on D. Now:

  • Latency adds up: the total time is the sum of every hop. One slow service makes the whole chain slow.
  • Failures cascade: if D is down or slow, C is stuck waiting, so B is stuck, so A is stuck. One failure can freeze the entire chain — and tie up threads/memory everywhere while they wait. This is how a single slow dependency causes a system-wide outage (see Circuit Breaker, Topic 56).

Asynchronous communication breaks these chains: hand the work to a queue and move on, so a slow downstream service can't freeze everyone upstream.

The Trade-off You're Really Making

Async isn't "better" — it shifts complexity. You gain responsiveness and resilience, but you must now handle: results arriving later, doing work out of order, retries and duplicate messages, and telling the user "we're working on it" instead of "here's your answer". Use sync for fast, must-have-now answers (login, reading a price); use async for slow or non-blocking work (emails, video processing, notifications, analytics).

Common misconceptions, cleared

"Asynchronous means multithreading/parallel." Not the same thing. Async is about not waiting for a result; parallelism is about doing things at the same time. You can be async on a single thread.

"Async is always faster." A single async task isn't faster — it often finishes later. What improves is the responsiveness and throughput of the overall system, because nothing sits blocked.

Interview tip

A reliable design instinct: "anything slow or non-essential to the immediate response should be async (via a queue)." In interviews, explicitly move email, notifications, image/video processing, and analytics off the request path — this single move makes your design faster and more resilient.

Key Takeaways

  • Synchronous = send and wait (phone call). Simple, immediate, but blocks you.
  • Asynchronous = send and continue (text message). Responsive and scalable, but more complex.
  • Long synchronous chains add up latency and let one slow service freeze everyone — async breaks the chain.
  • Async ≠ parallel, and a single async task isn't faster — the system's responsiveness improves.
  • Use sync when you need the answer now; use async for slow/non-essential work. Foundation for queues & events.

#38Message Queues

One-line definition

A Message Queue is a buffer that sits between two services. One service puts messages in (producer), and another takes them out and processes them (consumer) — at its own pace. It decouples the two so neither has to wait for the other.

Message queues are the classic tool for asynchronous communication. They let a fast service hand off work to a slower one without getting stuck waiting.

Real-life Analogy: A Restaurant Order Ticket Rail

Waiter pins the order; chef picks it up

A waiter does not stand and wait for the chef to finish each dish. They pin the order ticket on a rail (the queue) and go serve other tables. The chef picks tickets off the rail one by one, at their own pace. The rail decouples the waiter from the chef — both work without blocking each other.

The Key Parts

  • Producer: the service that creates and sends messages into the queue.
  • Queue: holds the messages safely in order until they are processed.
  • Consumer: the service that reads messages from the queue and does the work.
Produceradds tasks
→
Queue[ msg ][ msg ][ msg ]
→
Consumerprocesses one by one
The producer drops work into the queue; the consumer pulls it out at its own pace.

Why Message Queues Are So Useful

  • Decoupling: producer and consumer do not need to know about or wait for each other.
  • Buffering / smoothing spikes: if a sudden flood of work arrives, the queue holds it so the consumer is not overwhelmed.
  • Reliability: if the consumer crashes, messages wait safely in the queue until it recovers — nothing is lost.
  • Scaling: add more consumers to process the queue faster.

Popular tools: RabbitMQ, Amazon SQS, and Kafka (a special kind, covered later).

Real example: Swiggy order processing

When you place an order, the order service quickly drops messages into queues: one to notify the restaurant, one to find a delivery partner, one to send you a confirmation. Each of these is handled by a separate consumer at its own pace. Your app feels instant, while the heavy work happens in the background.

Delivery Guarantees: the detail that matters most

What happens if a consumer reads a message and then crashes before finishing? Queues offer different promises:

  • At-most-once: delivered zero or one time — may be lost, never duplicated. Fine for disposable data.
  • At-least-once (most common): never lost, but may be delivered more than once (e.g. the consumer processed it but crashed before acknowledging). You must design for duplicates.
  • Exactly-once: the holy grail — once and only once. Hard and expensive; often approximated.

Because at-least-once is the norm, consumers must be idempotent — processing the same message twice gives the same result (e.g. "charge order #123" should not charge twice). This is one of the most important practical rules in distributed systems.

Two Concepts You'll Be Asked About

  • Acknowledgement (ack): the consumer tells the queue "done, you can delete it". If it crashes before acking, the message is redelivered — that's why duplicates happen.
  • Dead Letter Queue (DLQ): a message that keeps failing (a "poison message") is moved aside to a DLQ after N tries, so it doesn't block the queue forever. Engineers inspect the DLQ later.
Common misconceptions, cleared

"A queue guarantees each message is processed exactly once." Usually no — most systems are at-least-once, so duplicates happen. Design idempotent consumers.

"Queues guarantee perfect global ordering." Often only within a partition/single queue, not across many consumers. If strict order matters, say so explicitly.

"A queue is the same as Pub/Sub." No — a queue typically delivers each message to one consumer (work sharing); Pub/Sub broadcasts to all subscribers (next topic).

Interview tip

When you add a queue, immediately mention idempotency ("consumers are idempotent because delivery is at-least-once") and a dead letter queue for poison messages. These two phrases instantly signal real production experience.

Key Takeaways

  • A message queue is a buffer between a producer and a consumer for async work.
  • It decouples services, smooths traffic spikes, and prevents lost work if a consumer crashes. Scale by adding consumers.
  • Most queues are at-least-once — expect duplicates, so make consumers idempotent.
  • Know acknowledgements and dead letter queues (DLQ) for failed messages.
  • Common tools: RabbitMQ, Amazon SQS, Kafka.

#39Pub/Sub Pattern

One-line definition

Pub/Sub (Publish/Subscribe) is a messaging pattern where a publisher sends a message to a topic, and every subscriber to that topic receives a copy. One message, many receivers — and the publisher does not know or care who is listening.

A normal message queue usually delivers each message to one consumer. Pub/Sub is different: it broadcasts each message to all interested subscribers. This is perfect when one event should trigger many different actions.

Real-life Analogy: A Newspaper Subscription

One paper, many readers

A newspaper (publisher) prints an edition. Everyone who subscribed (subscribers) gets their own copy delivered. The newspaper does not know each reader personally — it just publishes, and all subscribers receive it. New readers can subscribe anytime; some can unsubscribe. That is exactly Pub/Sub.

How It Works

  • Publisher: sends a message to a named topic (e.g. "order_placed").
  • Topic / channel: the category the message belongs to.
  • Subscribers: any service that subscribed to that topic gets its own copy of the message.
Publisher
→
Topic: "order_placed"
↓ copy to each
Email service
Inventory service
Analytics service
One published event reaches all subscribers at once, each acting independently.

Why It Is Powerful

  • One event, many reactions: "order placed" can trigger email, inventory update, and analytics — all at once.
  • Loose coupling: the publisher does not know who the subscribers are. You can add new subscribers without changing the publisher.
  • Scalable & flexible: services evolve independently.

Tools: Kafka, Redis Pub/Sub, Google Pub/Sub, AWS SNS.

Queue vs Pub/Sub (key difference)

Queue: each message goes to one consumer (work is shared out). Pub/Sub: each message goes to all subscribers (work is broadcast). Use a queue to split up work; use pub/sub to notify many parties of an event.

The Superpower: Adding Features Without Touching Old Code

The biggest practical win of Pub/Sub is extensibility. Because the publisher doesn't know its subscribers, you can add a brand-new reaction to an event without modifying (or risking breaking) the publisher. Want to add a "fraud check" when an order is placed? Just subscribe a new fraud service to the order_placed topic. The order service never changes. This is how large systems grow safely — new teams plug into existing event streams.

Consumer Groups: Combining Both Models

Real platforms (like Kafka) blend queue and pub/sub using consumer groups. Each group is a separate subscriber that gets all the messages (pub/sub across groups), but within a group the messages are shared among its members (queue-like load sharing). So the "Email team" and "Analytics team" each get every event (broadcast), while each team can run 10 workers that split the load (sharing). Best of both worlds.

Common misconceptions, cleared

"Pub/Sub guarantees every subscriber processes the message." Delivery still depends on the system's guarantees and whether subscribers are online/durable. With ephemeral pub/sub (like basic Redis Pub/Sub), a subscriber that's offline simply misses the message; durable systems (Kafka) let it catch up later.

"More subscribers slow the publisher down." In a good broker the publisher just writes once; fan-out to subscribers is the broker's job, so the publisher stays fast and unaware.

Interview tip

Use Pub/Sub whenever "one thing happens and several unrelated systems must react". Say: "I'll publish an order_placed event; email, inventory, and analytics each subscribe independently, so I can add new reactions later without changing the order service." Emphasising the extensibility benefit is what stands out.

Key Takeaways

  • Pub/Sub = a publisher sends to a topic; all subscribers get a copy.
  • Great when one event must trigger many independent actions; you can add subscribers without touching the publisher.
  • Consumer groups combine both models: broadcast across groups, load-share within a group.
  • Delivery depends on durability — ephemeral pub/sub drops messages for offline subscribers; durable (Kafka) lets them catch up.
  • Queue = one receiver; Pub/Sub = many receivers.

#40Event-Driven Architecture

One-line definition

Event-Driven Architecture (EDA) is a design where services communicate by producing and reacting to events ("something happened") instead of directly calling each other. Services announce events; other services react whenever they care.

In a traditional design, services call each other directly ("hey Payment service, charge this card"). In event-driven design, a service just announces "an order was placed", and any interested service reacts on its own. It builds on queues and pub/sub.

Real-life Analogy: A Doorbell

Announce, do not command

When someone rings your doorbell, the bell does not tell you exactly what to do. It just announces an event: "someone is at the door". You decide how to react — answer it, ignore it, check the camera. Other people in the house may react differently. The event is broadcast; reactions are independent.

Command vs Event (the mindset shift)

  • Command (traditional): "Payment service, charge this card." — the caller tells a specific service what to do and waits.
  • Event (event-driven): "Order was placed." — the producer just states a fact; whoever cares (payment, email, analytics) reacts on its own.

Strengths and Trade-offs

Strengths

  • Loose coupling — services do not depend on each other directly.
  • Easy to extend — add new reacting services without changing existing ones.
  • Scalable & responsive — everything is asynchronous.

Trade-offs

  • Harder to trace — the flow is spread across many reactions.
  • Eventual consistency — reactions happen slightly later.
  • Debugging is tougher — no single straight-line call path.
Real example: Uber ride

When a ride ends, Uber emits an event "ride_completed". Independently: the payment service charges the rider, the ratings service asks for a rating, the analytics service logs the trip, and the receipt service emails you. None of them were directly "called" — they all reacted to one event.

Event Notification vs Event-Carried State vs Event Sourcing

"Event-driven" comes in flavours worth knowing:

  • Event notification: the event is a thin "this happened" with an id; reactors call back to fetch details if needed. Lightweight, but creates extra lookups.
  • Event-carried state transfer: the event carries all the data reactors need, so they don't need to call back. More data per event, but fully decoupled.
  • Event sourcing: instead of storing only the current state, you store the full sequence of events as the source of truth, and rebuild state by replaying them. Gives a perfect audit log and time-travel, at the cost of complexity. (Kafka, Topic 41, is a natural fit.)

The Hard Parts You Must Plan For

EDA's loose coupling has real costs that interviewers probe:

  • Tracing a flow: "what happened to order #123?" is now spread across many services reacting independently — you need distributed tracing (Topic 83) and correlation IDs.
  • Eventual consistency: reactions happen slightly later, so the system is briefly inconsistent (the order exists but the email hasn't sent yet).
  • Ordering & duplicates: events can arrive out of order or twice — reactors must be idempotent and order-tolerant.
  • Failure handling: what if a reactor fails? You need retries, DLQs, and sometimes compensating actions (see Saga, Topic 63).
Common misconceptions, cleared

"Event-driven means no coupling at all." Reactors are still coupled to the event's shape/schema. Changing an event's structure can break every subscriber — schema versioning matters.

"EDA is always better than direct calls." No — for a simple, immediate request/response (login, fetch price), a direct synchronous call is clearer and easier to debug. EDA shines for one-to-many reactions and decoupling.

Interview tip

Pitch EDA for workflows with many independent reactions ("order placed → payment, email, inventory, analytics"). Then proactively address the cost: "I'll use correlation IDs and distributed tracing to follow a flow, and idempotent consumers since events can repeat." Naming the downsides upfront is what makes the answer credible.

Key Takeaways

  • Event-Driven Architecture = services react to events ("something happened") instead of calling each other directly.
  • Shifts from commands ("do this") to events ("this happened"); flavours: notification, state-transfer, event sourcing.
  • Pros: loose coupling, easy to extend, scalable. Cons: harder to trace, eventual consistency, ordering/duplicate handling.
  • Plan for tracing (correlation IDs), idempotency, retries/DLQs, and schema versioning.
  • Built on top of message queues and pub/sub.

#41Apache Kafka Basics

One-line definition

Apache Kafka is a high-performance distributed event streaming platform. Think of it as a super-powered, durable log of events that many producers write to and many consumers read from — able to handle millions of messages per second.

Kafka is the backbone of data pipelines at companies like LinkedIn, Netflix, and Uber. It is more than a queue — it stores a permanent, replayable log of events.

The Big Idea: A Durable Log

Like a never-ending diary

Most queues delete a message once it is read. Kafka instead writes every event into an append-only log (like a diary you only add to). Events stay for a set time (e.g. 7 days), so consumers can re-read them, and new consumers can read history from the start. This replayability is Kafka's superpower.

Key Concepts (in plain words)

  • Topic: a named stream of events (e.g. "clicks", "orders").
  • Partition: each topic is split into partitions so it can scale across servers and be read in parallel.
  • Producer: writes events into a topic.
  • Consumer: reads events from a topic. Each consumer tracks its own position (offset) in the log.
  • Broker: a Kafka server. A cluster of brokers shares the data for scale and reliability.
  • Offset: a consumer's bookmark — which message it has read up to.
Producers
→
Topic (partitions)append-only log
→
Consumer groups
Producers append events to partitioned topics; consumers read at their own offset and can replay.

Why Kafka Is Special

  • Massive throughput: handles millions of events per second.
  • Durable & replayable: events are stored, so you can re-process them.
  • Scalable: partitions spread load across many servers.
  • Decoupling: many producers and many consumer groups, all independent.

The Two Ideas That Make Kafka Click

1. The log is the truth, and consumers just track a position. A normal queue deletes a message after delivery, so it must remember "who got what". Kafka flips this: it keeps every event and each consumer simply stores an offset (a bookmark). To re-process the last hour, a consumer just rewinds its offset. To add a new analytics system that needs all history, point it at offset 0. This is why Kafka is the backbone of data pipelines — the same events feed many independent systems, each reading at its own speed.

2. Partitions give you both ordering and scale. A topic is split into partitions, and events are spread across them (often by a key, e.g. user_id). Kafka guarantees order within a partition, not across the whole topic. So all events for one user land in the same partition (ordered), while different users spread across partitions (parallel). Within a consumer group, each partition is read by exactly one consumer — so your max parallelism equals your number of partitions.

Where Kafka Fits (and where it doesn't)

Great for: event streaming, activity tracking (clicks, views), log aggregation, metrics pipelines, connecting microservices, and feeding data lakes/analytics. Less ideal for: simple task queues with per-message ack/retry semantics (RabbitMQ/SQS are simpler there), or low-volume request/response.

Common misconceptions, cleared

"Kafka is just a faster message queue." It's a durable, replayable log. The retention + offset model (re-read history) is a fundamentally different capability from a delete-on-read queue.

"Kafka guarantees total ordering of a topic." Only within a partition. Use the same key for events that must stay ordered together.

"More partitions are always better." Partitions add parallelism but also overhead and rebalancing cost; pick based on target throughput, not "as many as possible".

Interview tip

Reach for Kafka when the prompt involves high-volume streams, multiple independent consumers, or replay/audit needs (analytics, activity feeds, log pipelines). Mention partitioning by a key for ordering and that parallelism is bounded by partition count. For a simple background job queue, say RabbitMQ/SQS instead — choosing the simpler tool when appropriate is a plus.

Key Takeaways

  • Kafka is a distributed event streaming platform — a durable, replayable log of events.
  • Key terms: topic, partition, producer, consumer, broker, offset; consumers track their own offset and can replay.
  • Ordering is guaranteed per partition; parallelism within a group is bounded by partition count.
  • Ideal for streams, activity tracking, log/metrics pipelines, and feeding many independent consumers.
  • For simple task queues with per-message retries, a classic queue (RabbitMQ/SQS) may be simpler.

#42WebSockets

One-line definition

A WebSocket is a persistent, two-way connection between the client and server that stays open, letting both sides send data anytime — instantly, without opening a new connection each time.

Normal HTTP is one-way and one-shot: the client asks, the server answers, done. That is poor for live features like chat. WebSockets fix this by keeping a single connection open for continuous, instant, two-way talk.

Real-life Analogy: HTTP is Letters, WebSocket is a Phone Call

Open line vs sealed envelopes

Plain HTTP is like sending letters: each message needs a new envelope and trip, and only the sender can start. A WebSocket is like an open phone call: the line stays connected, and either person can speak at any moment, instantly. Perfect for real-time conversation.

How It Works

It starts as a normal HTTP request, then "upgrades" the connection to a WebSocket. After that, the connection stays open, and both client and server can push messages to each other at will — with very low overhead (no repeated handshakes).

Client
↔
Open WebSocketboth can send anytime
↔
Server
One always-open, two-way channel — either side can push instantly.

When to Use WebSockets

  • Chat apps (WhatsApp Web, Slack) — messages arrive instantly.
  • Live sports scores, stock tickers — constant updates pushed by the server.
  • Multiplayer games & collaborative editing (like Google Docs or Cursor) — everyone sees changes live.
  • Live notifications.
The cost

Open connections consume server memory. Holding millions of WebSocket connections needs careful scaling. Use them when you genuinely need real-time two-way communication — not for simple request/response, where plain HTTP is better.

Why Scaling WebSockets Is Hard (stateful connections)

Normal HTTP is stateless — any server can handle any request, so load balancing is easy. WebSockets are stateful: the connection lives on one specific server for its whole lifetime. This creates real challenges:

  • Sticky routing: the load balancer must keep a connection pinned to its server.
  • Cross-server messaging: if user A (on server 1) messages user B (on server 3), the servers must talk. The standard fix is a Pub/Sub backplane (often Redis): server 1 publishes the message, server 3 (where B is connected) is subscribed and pushes it to B.
  • Reconnection & missed messages: unlike SSE, WebSockets don't auto-reconnect — you must add reconnect logic and a way to resend messages missed while disconnected.

The Connection Lifecycle

A WebSocket begins with a normal HTTP request carrying an Upgrade: websocket header. The server agrees, and the same TCP connection is "upgraded" — from then on it's a persistent full-duplex channel with tiny per-message overhead (no repeated headers/handshakes). Periodic ping/pong frames keep it alive and detect dead connections.

Common misconceptions, cleared

"WebSockets replace HTTP." No — they complement it. You still use HTTP/REST for normal requests and WebSockets only for the live, two-way bits.

"WebSockets guarantee delivery." No — if the connection drops, in-flight messages can be lost. You need acknowledgements and resend logic for reliability.

"One server can hold unlimited connections." Each open connection uses memory/file descriptors; a single server handles tens of thousands, not millions — you scale out and coordinate via a backplane.

Interview tip

For chat/live features, don't stop at "use WebSockets". Add the scaling story: "connections are stateful, so I'll use sticky routing and a Redis Pub/Sub backplane to route messages between users on different servers, plus reconnect + missed-message handling." That depth is exactly what a chat-system question (Topic 93) is testing.

Key Takeaways

  • A WebSocket is a persistent, two-way connection — both sides can send anytime, instantly.
  • It starts as HTTP, then upgrades and stays open (kept alive with ping/pong).
  • Ideal for real-time features: chat, live scores, games, collaborative editing.
  • Connections are stateful → need sticky routing and a Pub/Sub backplane to scale across servers.
  • Cost: open connections use server resources, no auto-reconnect, no built-in delivery guarantee.

#43Server-Sent Events (SSE)

One-line definition

Server-Sent Events (SSE) let a server push a continuous one-way stream of updates to the client over a single open HTTP connection. The server keeps sending; the client just listens.

Sometimes you only need updates to flow one way — server to client — like a live news feed. WebSockets (two-way) are overkill for that. SSE is the simpler, one-way tool built right into HTTP.

Real-life Analogy: A Live Radio Broadcast

You listen, you do not talk back

A radio station broadcasts continuously, and you just tune in and listen. You cannot talk back through the radio. SSE is like this: the server continuously broadcasts updates, and the client listens — it does not send data back on the same channel.

How It Works

The client opens one HTTP connection and the server keeps it open, sending new events down it whenever there is something new. It is simple, uses standard HTTP, and the browser automatically reconnects if the connection drops.

SSE vs WebSocket

PointSSEWebSocket
DirectionOne-way (server → client)Two-way
Built onPlain HTTPUpgraded protocol
ComplexitySimpleMore complex
Auto-reconnectYes, built inManual
Best forLive feeds, notificationsChat, games (two-way)
Where SSE fits well

Live stock price tickers, news feeds, notification streams, and progress updates (e.g. an AI assistant streaming its answer token-by-token) — all great fits for SSE, because data only flows server → client.

Why SSE Powers AI Chat Streaming

You've seen ChatGPT, Claude, and Cursor "type" their answers word by word — that's typically SSE. The browser opens one connection, and the server streams tokens as the model generates them. SSE is perfect here because the data flows purely one way (server → client), it runs over ordinary HTTP (works through proxies/firewalls easily), and the browser's built-in EventSource auto-reconnects if the line drops. A full WebSocket would be overkill since the client isn't sending anything back on that channel.

The Practical Catches

  • Connection limits: over old HTTP/1.1, browsers allow only ~6 connections per domain, and each SSE stream eats one. Over HTTP/2 this limit largely disappears (many streams share one connection).
  • One-way only: to send data to the server, the client makes a separate normal HTTP request — that's fine for most "listen to updates" use cases.
  • Text-based: SSE sends UTF-8 text events; for binary, WebSockets are better.
Three ways to get "live" data — pick the lightest that works

Polling (ask repeatedly) → simplest, wasteful. SSE (server streams one-way) → great for live feeds/notifications/AI streaming. WebSocket (two-way) → only when the client must also send continuously (chat, games, collaboration). Always choose the simplest tool that meets the need.

Common misconceptions, cleared

"SSE is outdated; WebSockets replaced it." No — for one-way streaming, SSE is simpler, auto-reconnects, and rides on plain HTTP. It's widely used (including AI token streaming).

"SSE can do two-way like WebSockets." No — SSE is strictly server → client. Client-to-server needs a separate request.

Interview tip

For one-directional live updates (notifications, feeds, progress bars, streaming LLM responses), suggest SSE and note it auto-reconnects and needs no special protocol. Reserve WebSockets for genuine two-way needs. Knowing when not to use WebSockets shows good judgement.

Key Takeaways

  • SSE = a server pushes a one-way stream of updates to the client over one HTTP connection.
  • Simpler than WebSockets, uses plain HTTP, and auto-reconnects (built-in EventSource).
  • Powers AI chat token streaming, live feeds, tickers, and notifications.
  • One-way and text-only; client→server needs a separate request. HTTP/2 removes old connection limits.
  • Use WebSockets only when you need two-way real-time communication.

#44Polling vs WebSockets

One-line definition

Before WebSockets, getting "live" updates meant polling — the client repeatedly asking the server "anything new?". This topic compares polling (short and long) with WebSockets so you know which to use.

How do you get fresh data from the server? You either keep asking (polling) or you keep a connection open (WebSockets/SSE). Let us see the options from simplest to best.

1. Short Polling

The client asks the server "any updates?" on a fixed timer (e.g. every 5 seconds). Simple, but wasteful — most requests return "nothing new", and updates are delayed by up to the interval.

Analogy

Like a kid asking "are we there yet?" every minute on a road trip. Most answers are "no", wasting everyone's energy.

2. Long Polling

The client asks, but the server holds the request open until it actually has an update (or a timeout), then responds. The client immediately asks again. Fewer wasted requests and faster updates than short polling, but still opens many connections.

3. WebSockets

One open connection; the server pushes updates the instant they happen. No repeated asking, truly real-time, most efficient for frequent updates.

MethodHowFreshnessEfficiency
Short PollingAsk on a timerDelayedWasteful
Long PollingAsk, server holds until updateFairly freshBetter
WebSocketsOpen connection, server pushesInstantBest for frequent updates
Which to choose

Rare updates and simplicity? Short/long polling is fine. Frequent, real-time updates (chat, live data)? WebSockets. One-way live stream only? SSE (previous topic). Match the tool to how often and which direction data flows.

Why Polling Still Matters (don't dismiss it)

It's tempting to think "WebSockets are best, always use them" — but polling remains the right choice surprisingly often:

  • Simplicity & compatibility: polling is just normal HTTP requests — it caches, load-balances, authenticates, and passes through proxies/firewalls trivially. No special infrastructure.
  • Stateless & easy to scale: each poll is independent, so any server can answer. No long-lived connections to manage.
  • Low update frequency: if data changes every few minutes (e.g. a dashboard, order status), short polling every 30s is perfectly fine and far simpler than a socket.

Long polling was the clever bridge before WebSockets — it gives near-real-time updates over plain HTTP, and is still used as a fallback when WebSockets are blocked.

Real-world choices

An order-status page ("preparing → out for delivery") → short polling is plenty. A live cricket score pushed to millions → SSE. WhatsApp/Slack chat → WebSockets for instant two-way. The right answer depends on frequency and direction, not on what's "newest".

Common misconceptions, cleared

"WebSockets are always better than polling." No — for infrequent updates, polling is simpler, cheaper to operate, and easier to scale. WebSockets shine only with frequent/real-time two-way data.

"Long polling is real-time." Close, but each cycle still re-establishes a request; under very high frequency it creates lots of connection churn that WebSockets avoid.

Interview tip

Resist jumping to WebSockets. Ask "how often does the data change, and is it one-way or two-way?" Then justify: rare → polling; one-way frequent → SSE; two-way frequent → WebSockets. Showing you'd pick the simplest sufficient option is a maturity signal.

Key Takeaways

  • Short polling: ask on a timer — simple but wasteful and delayed.
  • Long polling: server holds the request until there is news — fresher, less wasteful; classic WebSocket fallback.
  • WebSockets: open connection, instant server push — best for frequent real-time updates.
  • Polling is stateless, simple, and easy to scale — still the right call for infrequent updates.
  • Pick based on update frequency and whether you need one-way or two-way.

#45gRPC vs REST vs GraphQL

One-line definition

These are three popular ways for systems to talk via APIs. REST = simple, resource-based over HTTP. GraphQL = ask for exactly the data you want in one query. gRPC = super-fast binary communication, great between microservices.

We covered REST in Topic 8. Now we compare it with two modern alternatives, each solving a specific pain point of REST.

REST — The Familiar Standard

Resources at URLs, actions via HTTP methods (GET/POST/PUT/DELETE), usually returning JSON. Simple, universal, easy to cache. Its weaknesses: over-fetching (you get more data than you need) and under-fetching (you need several calls to gather related data).

GraphQL — Ask for Exactly What You Need

Created by Facebook. You send one query describing precisely the fields you want, and the server returns exactly that — no more, no less, in a single request.

  • Solves over/under-fetching: get exactly the right data in one call.
  • Great for: apps with complex, nested data and many screens (mobile apps especially).
  • Cost: more complex on the server; caching is harder than REST.

gRPC — Fast Machine-to-Machine

Created by Google. It uses a compact binary format (Protocol Buffers) instead of text JSON, making it very fast and small. Built for high-performance communication between services.

  • Solves speed: binary is much faster and smaller than JSON.
  • Great for: internal microservice-to-microservice calls, low-latency systems.
  • Cost: not human-readable, harder to use directly from browsers.
PointRESTGraphQLgRPC
Data formatJSON (text)JSON (text)Binary (Protobuf)
FetchingFixed responsesExactly what you askDefined contracts
SpeedGoodGoodFastest
Best forPublic APIs, simplicityComplex/nested data, mobileInternal microservices
ReadabilityEasyEasyNot human-readable
Rule of thumb

REST for public, simple APIs. GraphQL when clients need flexible, precise data (e.g. a rich mobile app). gRPC for fast internal communication between microservices. Many big systems use all three in different places.

Understanding the Specific Pain Each One Solves

These aren't random alternatives — each was born to fix a concrete REST limitation:

  • Over-fetching (REST → GraphQL): a mobile screen needs just a user's name and avatar, but GET /user/1 returns the entire profile. Wasted bandwidth. GraphQL asks for exactly those two fields.
  • Under-fetching / N+1 calls (REST → GraphQL): to show a post with its author and comments, REST needs 3+ round trips. GraphQL fetches the whole nested shape in one query.
  • Text overhead & loose contracts (REST → gRPC): JSON is bulky to parse and has no enforced schema. gRPC uses compact binary Protobuf with a strict, generated contract — faster on the wire and type-safe across languages.

The Streaming and Browser Angle

A practical differentiator: gRPC supports bidirectional streaming natively (great for service-to-service real-time), but it doesn't run directly in browsers without a proxy (gRPC-Web). REST and GraphQL run anywhere a browser can make HTTP calls. This is a big reason gRPC dominates internal microservice traffic while REST/GraphQL face the public/browser world.

The Hidden Cost of GraphQL

GraphQL's flexibility shifts complexity to the server: caching is harder (every query can be different, vs REST's cacheable URLs), and a single malicious deep/nested query can be expensive — so you need query-cost limits and depth limits. Power, but with new responsibilities.

Common misconceptions, cleared

"GraphQL replaces REST." No — REST is still the default for most public APIs; GraphQL wins when clients need flexible, nested data. Many teams use both.

"gRPC is just faster REST." It's a different model: contract-first (Protobuf), binary, supports streaming, but not human-readable and browser-unfriendly. Different tool, different niche.

"GraphQL means fewer total requests, so always faster." Often fewer round trips, yes — but a heavy nested query can hammer your backend; you must guard query cost.

Interview tip

Map the choice to the boundary: "gRPC for internal service-to-service (fast, typed, streaming), REST for public/simple APIs, GraphQL for flexible client-driven data like mobile." Mentioning GraphQL's caching/query-cost challenge and gRPC's browser limitation shows you understand the trade-offs, not just the buzzwords.

Key Takeaways

  • REST: simple, resource-based, JSON, cacheable — but can over/under-fetch.
  • GraphQL: ask for exactly the data you need in one query — great for complex/mobile; harder caching, watch query cost.
  • gRPC: fast binary (Protobuf), strict contracts, native streaming — ideal between microservices, browser-unfriendly.
  • Each solves a specific REST pain: over-fetch, under-fetch, and text/contract overhead.
  • Not rivals to pick once — large systems use the right one per boundary.

#46Scaling Strategies

One-line definition

Scaling strategies are the overall plan for how a system grows to handle more users and data. It pulls together everything you have learned — vertical/horizontal scaling, load balancers, caching, replication, sharding — into a step-by-step growth path.

You now know many individual tools. This topic shows the order in which real systems typically apply them as they grow from 100 users to 100 million.

The Typical Scaling Journey

  • Step 1 — Single server: app + database on one machine. Perfect for the start.
  • Step 2 — Split the database out: move the database to its own server so each can be tuned separately.
  • Step 3 — Vertical scaling: upgrade to a bigger machine. Quick fix, but hits a ceiling.
  • Step 4 — Add caching: use Redis/CDN to cut repeated work and database load.
  • Step 5 — Horizontal scaling + load balancer: add many app servers behind a load balancer.
  • Step 6 — Database read replicas: spread reads across replicas (replication).
  • Step 7 — Sharding / partitioning: split the database when it is too big for one server.
  • Step 8 — Microservices, queues, async: break into services and offload slow work to queues.
1 server
→
+ cache
→
+ load balancer & many servers
→
+ replicas & sharding
Scale step-by-step, adding complexity only when the current setup hurts.

The Guiding Principles

  • Scale only when needed: do not add complexity before you feel the pain. Premature scaling wastes time and money.
  • Measure first: find the actual bottleneck (CPU? database? network?) before fixing.
  • Cache aggressively: caching is often the cheapest, biggest win.
  • Make services stateless (next topic) so horizontal scaling is easy.

The Database Is Almost Always the First Wall

Here's a pattern that holds across nearly every real system: app servers are easy to scale, but the database is hard. You can add ten app servers in minutes (they're stateless), but they all still talk to one database. So the database becomes the bottleneck first. That's exactly why the scaling journey front-loads caching (Step 4) and read replicas (Step 6) before the painful step of sharding (Step 7) — each one buys time before you must split the data. Most apps never need sharding; caching + replicas + a beefy database carry them very far.

Scale Up Before You Scale Out

A counterintuitive truth: modern single machines are enormous (128+ cores, terabytes of RAM). Vertical scaling (a bigger box) is often the fastest, simplest fix and should not be skipped out of pride. You only go horizontal when you hit the ceiling of one machine or need fault tolerance (one big box is a single point of failure). The art is sequencing: cheap+simple wins first, complex distributed steps last.

Common misconceptions, cleared

"Design for a billion users from day one." Almost always wrong — you'd over-engineer, ship slowly, and burn money solving problems you don't have. Build for ~10× your current scale, and evolve.

"Microservices make you scale better." Not automatically — a well-built monolith on good hardware with caching outscales a poorly-built microservice mess. Microservices solve team/organisational and independent-deployment problems more than raw scale.

Interview tip

When asked "how would you scale this?", walk the journey in order and tie each step to a bottleneck: "first measure, then cache to cut DB reads, then add app servers behind a load balancer, then read replicas, and only shard if the DB itself can't cope." Showing you scale incrementally driven by bottlenecks beats listing every technology at once.

Key Takeaways

  • Scaling is a journey: single server → cache → load balancer + many servers → replicas → sharding → microservices.
  • The database is usually the first bottleneck — caching and replicas delay the pain of sharding.
  • Scale up (bigger box) before scaling out; add complexity only when a measured bottleneck demands it.
  • Caching is usually the biggest, cheapest win; stateless services make horizontal scaling easy.
  • Don't over-engineer for imaginary scale — design for ~10× current load and evolve.

#47Stateless vs Stateful Services

One-line definition

A stateless service keeps no memory of past requests — each request carries everything needed. A stateful service remembers information about the client between requests (stored on the server).

This single property decides how easily you can scale horizontally. Stateless services are the secret to scaling to millions of users smoothly.

Real-life Analogy: Two Kinds of Counters

A new clerk vs your regular clerk

Stateless = any clerk can serve you, because you bring all your documents each time. It does not matter who helps you. Stateful = only "your" clerk can help, because only they remember your case from last time. If that clerk is on leave, you are stuck.

Why Stateless Is Better for Scaling

With horizontal scaling, requests from one user may hit different servers each time (the load balancer spreads them out). If servers are stateless, this is fine — any server can handle any request. If servers are stateful, the user must keep hitting the same server (called "sticky sessions"), which is fragile: if that server dies, the user's data is lost.

User requestcarries all info
→
Any server works
Stateless: any server can handle any request — perfect for load balancing.

Where Does the State Go Then?

Real apps still need to remember things (like who is logged in). The trick: keep the servers stateless, and store the state in a shared external place — a database or a cache like Redis. Then every server can read it, and no single server "owns" the user.

PointStatelessStateful
Remembers past?NoYes
ScalingEasy (any server)Hard (sticky sessions)
If a server diesNo problemUser data may be lost
Where state livesExternal (DB/Redis)On the server itself

The Restart Test (a simple way to check)

Here's a quick mental test for whether a service is truly stateless: "If I kill this server right now and a new one takes over, does any user lose anything?" If the answer is no, you're stateless — you can restart, redeploy, and auto-scale freely. This is also why stateless services pair perfectly with auto-scaling and rolling deployments (Topic 88): instances can be added and removed at will.

Some State Is Unavoidable — Push It to the Edges

You can never truly eliminate state — logins, shopping carts, and uploaded files all exist. The principle isn't "no state anywhere", it's "keep state out of the application servers". Move it to purpose-built stateful systems: databases (durable data), Redis (sessions/cache), object storage like S3 (files), Kafka (event logs). Those systems are designed to handle state safely; your app servers should stay disposable.

Common misconceptions, cleared

"Stateless means the app stores nothing anywhere." No — the data still lives in a shared store; it's just not held on the app server's local memory/disk. The server is stateless, the system isn't.

"Sticky sessions are a good way to scale." They're a crutch — they unbalance load and lose sessions when a server dies. Prefer a shared session store or tokens (next topic).

Interview tip

Almost every scalable design starts with "I'll keep the app servers stateless and store session/state in Redis (or a JWT), so any server can handle any request and I can auto-scale freely." Saying this early frames the rest of your design and is something interviewers actively look for.

Key Takeaways

  • Stateless = no server memory of past requests; each request is self-contained.
  • Stateful = server remembers the client between requests.
  • Stateless services scale horizontally easily, survive failures, and suit auto-scaling/rolling deploys (the "restart test").
  • State is unavoidable — push it to purpose-built stores (DB, Redis, S3, Kafka), not the app server.
  • Avoid relying on sticky sessions at scale.

#48Session Management at Scale

One-line definition

A session is how a server remembers a logged-in user across requests (since HTTP is stateless). Session management at scale is about storing that "who is logged in" info so it works across many servers.

From Topic 6 we know HTTP forgets you after each request. So apps create a session to remember you. With one server this is easy — but with many servers (horizontal scaling), where do you keep the session? That is the challenge.

The Problem

Logged in on Server A, next request hits Server B

You log in and Server A remembers you. But your next click is routed to Server B, which has never seen you — suddenly you appear logged out! With many servers, storing the session inside one server breaks. We need a smarter approach.

Approaches to Solve It

  • Sticky sessions: the load balancer always sends a user to the same server. Simple, but fragile — if that server dies, the session is lost, and load can become uneven.
  • Centralized session store (recommended): store sessions in a shared, fast store like Redis. Every server reads the session from there, so any server can serve any user. This is the most common, scalable solution.
  • Token-based (stateless) sessions: store the session data in a signed token (like a JWT, Topic 71) held by the client. The server does not store anything — it just verifies the token. Fully stateless and very scalable.
Any server
→
Shared Redissession store
All servers read sessions from one shared store, so any server can serve any user.
ApproachWhere session livesTrade-off
Sticky sessionsOn one serverSimple but fragile
Central store (Redis)Shared storeScalable, needs a fast store
Token (JWT)On the clientStateless, but hard to revoke

The Real Trade-off: Server-Side Sessions vs Tokens

This choice comes up constantly, so understand it deeply — it's a classic revocation vs scalability tug-of-war:

  • Server-side sessions (Redis): the server holds the truth. To log someone out or ban them, you just delete their session — instant. The cost: every request does a lookup in the shared store, and that store must be highly available.
  • Stateless tokens (JWT): the token itself proves who you are, so no lookup is needed — extremely scalable. The cost: you can't easily revoke a token before it expires (it's valid until then). Mitigations: short token lifetimes + refresh tokens, or a small "blocklist" of revoked tokens (which reintroduces a little state).

The common production pattern: short-lived JWT access tokens (minutes) plus a long-lived refresh token stored server-side, giving you both scalability and a revocation point.

Common misconceptions, cleared

"JWTs are always more secure/better than sessions." No — they trade revocability for statelessness. For apps needing instant logout/ban (banking), server-side sessions can be the better choice.

"Put everything in the JWT to avoid DB calls." Risky — tokens are sent on every request (bloat) and can't be updated until they expire, so stale data lingers. Keep tokens small; fetch changeable data fresh.

Interview tip

When asked about sessions across many servers, name both options and the deciding factor: "Redis-backed sessions if I need instant revocation; short-lived JWTs + refresh tokens if I want maximum statelessness/scale." Explicitly mentioning the revocation problem of JWTs is the detail interviewers probe for.

Key Takeaways

  • A session remembers a logged-in user across stateless HTTP requests.
  • With many servers, storing sessions inside one server breaks — the user looks logged out elsewhere.
  • Best solutions: a centralized store like Redis (easy revocation), or stateless JWTs (max scale, hard to revoke).
  • Common pattern: short-lived JWT + server-side refresh token for scale and a revocation point.
  • Sticky sessions work but are fragile — avoid relying on them at scale.

#49Rate Limiting

One-line definition

Rate limiting controls how many requests a user (or client) can make in a given time window. It protects your system from being overwhelmed by abuse, bugs, or attacks.

If anyone could hit your API a million times a second, they could crash it or run up huge bills. Rate limiting sets fair limits, like "100 requests per minute per user".

Real-life Analogy: A Theme Park Ride

Only so many per hour

A popular ride can only take so many people per hour. Staff control the queue so the ride is not overwhelmed and everyone gets a fair turn. Rate limiting does this for your API — it stops any one user from hogging or overwhelming the system.

Why Rate Limit?

  • Prevent abuse & attacks (e.g. brute-force login attempts, scrapers).
  • Protect from overload — keep the system stable for everyone.
  • Ensure fairness — no single user starves others of capacity.
  • Control cost — especially for expensive operations.

Common Algorithms (Simply)

  • Token Bucket: you have a bucket of tokens; each request uses one. Tokens refill at a steady rate. Allows short bursts — the most popular method.
  • Leaky Bucket: requests flow out at a fixed steady rate, like water leaking from a bucket; extras wait or overflow. Smooths traffic.
  • Fixed Window: count requests in fixed time blocks (e.g. per minute). Simple, but can allow spikes at the edges.
  • Sliding Window: a smoother, rolling count that avoids the edge-spike problem of fixed windows.

When the limit is exceeded, the server responds with HTTP status 429 (Too Many Requests).

Real example

Login pages limit attempts (e.g. 5 per minute) to block password guessing. Public APIs (like Twitter or GitHub) limit calls per hour per key. AI assistants limit messages per minute to keep service fair and stable.

Token Bucket in Plain Terms (the one to remember)

Picture a bucket that holds up to 100 tokens and refills 10 tokens per second. Each request removes one token. If tokens are available, the request passes; if the bucket is empty, it's rejected (429). The beauty: after a quiet period the bucket is full, so a user can make a short burst of up to 100 quickly — but sustained traffic is capped at the 10/sec refill rate. This "allow bursts but bound the average" behaviour is why token bucket is the most widely used algorithm. Fixed window is simpler but has an edge flaw: a user can send 100 at 11:59:59 and 100 more at 12:00:00 — 200 in two seconds. Sliding window fixes that by counting over a rolling period.

The Distributed Catch: One Counter Across Many Servers

The hard part in real systems: if a user's requests hit 10 different servers, each server only sees part of their traffic, so a local counter under-counts. The standard solution is a shared, fast, atomic counter in Redis (using atomic INCR with expiry). Every server checks the same Redis counter, so the limit is enforced globally. This is why "rate limiter" interview questions (Topic 98) almost always involve Redis.

Where to Enforce It

Rate limiting usually lives at the edge — the API gateway or load balancer (Topic 50) — so bad traffic is rejected before it consumes your expensive backend. You also return helpful headers (Retry-After, X-RateLimit-Remaining) so well-behaved clients can back off gracefully.

Common misconceptions, cleared

"Rate limiting and throttling are the same as load balancing." No — load balancing spreads work; rate limiting rejects excess work. Different jobs.

"A per-server limit is enough." Not when you have many servers — you need a shared (Redis) counter, or the real global limit becomes limit × number of servers.

"429 means the user did something wrong." Not necessarily — it just means "slow down". Clients should honour Retry-After and retry with backoff (Topic 57).

Interview tip

Say "token bucket per user, enforced at the API gateway, backed by a Redis counter so the limit is global across servers". Mentioning the distributed counter problem and the 429 + Retry-After response is exactly what a rate-limiter design question wants.

Key Takeaways

  • Rate limiting caps how many requests a client can make in a time window.
  • It protects against abuse, overload, unfairness, and runaway cost.
  • Token Bucket (allows bursts, caps average) is the go-to; fixed window has edge spikes, sliding window fixes them.
  • Across many servers, enforce the limit with a shared atomic counter in Redis; apply it at the edge/gateway.
  • Over-limit requests get HTTP 429 Too Many Requests (with Retry-After).

#50API Gateway

One-line definition

An API Gateway is a single entry point that sits in front of all your microservices. Every client request goes through it, and it handles common jobs (routing, auth, rate limiting) before passing requests to the right service.

In a microservices system, there could be dozens of services. You do not want clients talking to each one directly — that is messy and insecure. The API Gateway is the single, organised "front door".

Real-life Analogy: A Building Reception Desk

One front desk for the whole building

Visitors do not wander a big office building looking for the right room. They check in at reception, which verifies them, then directs them to the correct department. The API Gateway is your system's reception desk — one door for all visitors, then routing inside.

What an API Gateway Does

  • Routing: sends each request to the correct microservice.
  • Authentication: checks who you are once, at the door.
  • Rate limiting: enforces request limits (Topic 49).
  • Aggregation: can combine results from several services into one response.
  • Other cross-cutting jobs: SSL/TLS handling, logging, caching, request transformation.
Clients
→
API Gatewayauth, routing, rate limit
→
Many microservices
One entry point handles common concerns, then routes to the right service.

Why It Helps

  • Simplifies clients: they talk to one endpoint, not dozens.
  • Centralizes shared logic: auth, rate limiting, and logging live in one place, not duplicated in every service.
  • Security: internal services are hidden behind the gateway.
Watch out

Because everything passes through it, the gateway can become a bottleneck or a single point of failure. So it must be made highly available and scalable too. (An API Gateway is a specialized reverse proxy.)

The Killer Feature: Cross-Cutting Concerns in One Place

The deepest reason API gateways exist: certain jobs are needed by every service — authentication, rate limiting, TLS termination, logging, request validation. Without a gateway, you'd copy that code into all 30 microservices (and fix bugs in 30 places). The gateway pulls these cross-cutting concerns out into one layer, so individual services stay small and focused purely on business logic. Change the auth rule once, at the gateway, and it applies everywhere.

API Gateway vs Load Balancer (don't confuse them)

Both sit in front of servers, but they operate at different levels. A load balancer mainly spreads traffic across identical instances (Topic 15) — it cares about distribution and health. An API gateway is application-aware — it routes by what the request is (/orders → order service, /users → user service), enforces auth, transforms requests, and aggregates responses. In practice a load balancer often sits in front of the gateway, and the gateway routes to services (which each have their own load balancing).

BFF: a Gateway per Client Type

A popular pattern is Backend-for-Frontend (BFF) — a separate gateway tailored to each client (one for the mobile app, one for the web app). Each BFF aggregates and shapes data exactly how that client needs, avoiding a bloated one-size-fits-all gateway. (GraphQL, Topic 45, is sometimes used to play this role.)

Common misconceptions, cleared

"An API gateway is just a load balancer." No — it's application-aware: routing by path, auth, rate limiting, aggregation, transformation. A load balancer just distributes traffic.

"Put all business logic in the gateway." Anti-pattern — that recreates a monolith at the door and makes the gateway a fragile bottleneck. Keep it to cross-cutting concerns and routing; logic belongs in services.

Interview tip

In any microservices design, add an API gateway and list what it offloads: "auth, rate limiting, TLS, routing, and response aggregation — so services stay focused." Then note it must be highly available (run multiple instances behind a load balancer) so it isn't a single point of failure. Distinguishing it from a load balancer earns credibility.

Key Takeaways

  • An API Gateway is the single front door to all your microservices.
  • It centralizes cross-cutting concerns (auth, rate limiting, TLS, logging, aggregation) so services stay focused.
  • It's application-aware (routes by path/content) — unlike a load balancer, which just distributes traffic.
  • The BFF pattern gives each client type its own tailored gateway.
  • Must be highly available (multiple instances), or it becomes a single point of failure.

#51Service Discovery

One-line definition

Service Discovery is how services find each other's network addresses automatically in a system where servers constantly come and go. Instead of hard-coding addresses, services look them up from a registry.

In a cloud system, servers are created and destroyed all the time (auto-scaling, crashes, restarts), so their addresses keep changing. Hard-coding "the payment service is at 10.0.0.5" breaks instantly. Service discovery solves this.

Real-life Analogy: A Phone Directory That Updates Itself

Always-current contact list

Imagine a phone directory that automatically updates whenever someone changes their number. You never need to memorize numbers — you just look up the name and always get the current number. A service registry is exactly that for services.

How It Works

  • Service Registry: a central directory that keeps an up-to-date list of all running services and their addresses.
  • Registration: when a new service instance starts, it registers itself in the registry. When it stops, it is removed.
  • Discovery: when a service wants to call another, it looks it up in the registry to get a current address.
  • Health checks: the registry removes instances that fail health checks, so only healthy ones are returned.
Service A
→ "where is B?"
Registrycurrent addresses
→
Service B (healthy)
Services ask the registry for current, healthy addresses instead of hard-coding them.

Two Styles

  • Client-side discovery: the calling service asks the registry and picks an instance itself.
  • Server-side discovery: the calling service hits a load balancer, which asks the registry and routes for it.

Common tools: Consul, etcd, Eureka, and Kubernetes' built-in DNS-based discovery.

Why You Can't Just Hard-Code Addresses Anymore

In the old days, you'd put "payment service = 10.0.0.5" in a config file and move on. In the cloud, that breaks within minutes because:

  • Auto-scaling spins up and tears down instances constantly — addresses are born and die all day.
  • Crashes & restarts give services new addresses (especially containers, which get fresh IPs).
  • Deployments replace old instances with new ones at different addresses.

So addresses are dynamic, and you need a live "who's alive right now and where" lookup. That's the registry. The deeper insight: service discovery is what makes elastic, self-healing infrastructure possible — without it, scaling and recovery would require manual reconfiguration.

How Kubernetes Hides All of This

If you use Kubernetes, you mostly get discovery for free. You call a stable name like http://payment-service, and Kubernetes' built-in DNS + Service abstraction resolves it to a healthy pod behind the scenes — registration, health checks, and load balancing included. This is why many teams don't run a separate Consul/Eureka: the platform does it. (More in Topic 87.)

Common misconceptions, cleared

"Service discovery is just DNS." DNS is one mechanism, but discovery adds health awareness (only return alive instances) and fast updates — plain DNS caching can serve dead addresses for too long.

"The registry is just nice to have." It's critical infrastructure — if the registry is wrong or down, services can't find each other. That's why registries (etcd, Consul) are themselves run as highly-available clusters (using consensus, Topic 64).

Interview tip

When your design has dynamic/auto-scaling services, mention "they find each other via service discovery (a registry with health checks), not hard-coded IPs" — and note that on Kubernetes this is built in via DNS + Services. It shows you understand how cloud systems actually wire themselves together.

Key Takeaways

  • Service discovery lets services find each other's addresses automatically as servers come and go.
  • A service registry keeps a live list; services register on start and are looked up on demand.
  • It exists because cloud addresses are dynamic (auto-scaling, crashes, deploys) — it enables self-healing, elastic infra.
  • Health checks ensure only healthy instances are returned; the registry itself must be highly available.
  • Tools: Consul, etcd, Eureka; Kubernetes provides it built-in via DNS + Services.

#52Service Mesh

One-line definition

A Service Mesh is an extra infrastructure layer that handles all the service-to-service communication in a microservices system — things like routing, retries, security, and monitoring — without the services themselves having to code it.

As microservices multiply, every service needs the same networking features: retries, encryption, timeouts, metrics. Writing this into every service is repetitive and error-prone. A service mesh moves all of it into a shared layer.

Real-life Analogy: A Postal System for Your Services

You write the letter; the post office handles delivery

You do not personally figure out routes, retries, and security to deliver a letter — the postal system does. A service mesh is like a smart postal system between your services: each service just "sends a message", and the mesh handles secure, reliable, observable delivery.

How It Works: Sidecars

The mesh runs a small helper called a sidecar proxy next to each service. All traffic in and out of the service goes through its sidecar. The sidecars together form the "mesh" and handle the networking, so the service code stays clean.

Service A + sidecar
↔ mesh
Service B + sidecar
Each service gets a sidecar proxy; the sidecars handle all communication concerns.

What a Service Mesh Provides

  • Traffic management: smart routing, load balancing, retries, timeouts.
  • Security: automatic encryption (mutual TLS) between services.
  • Observability: built-in metrics, logs, and tracing for all calls.
  • Resilience: circuit breaking and fault handling (Part 7) without app code.

Popular tools: Istio and Linkerd.

Is it always needed?

A service mesh adds real complexity and overhead. It pays off when you have many microservices and need consistent security and observability across all of them. For small systems, it is overkill — another "add complexity only when needed" call.

The Core Idea: Move Networking Out of the App

The whole point of a service mesh is to solve a duplication problem. In microservices, every service needs retries, timeouts, encryption, and metrics for its calls. Without a mesh, each team re-implements these (differently, with bugs) in every language they use. The mesh extracts all of it into the sidecar, so the application just makes a plain network call and the sidecar transparently adds retries, mTLS, tracing, and circuit breaking. Your business code shrinks to business logic — the platform owns reliability and security uniformly.

Control Plane vs Data Plane

A mesh has two parts worth naming: the data plane is the fleet of sidecar proxies that actually carry traffic; the control plane (e.g. Istio's controller) is the brain you configure — it pushes rules ("retry twice", "encrypt everything", "send 5% of traffic to v2") down to all sidecars. You set policy once centrally, and it applies everywhere instantly. This also enables advanced rollouts like canary (Topic 89) at the network level.

API Gateway vs Service Mesh (a common confusion)

They handle different directions of traffic. The API gateway manages north-south traffic — requests coming in from outside clients. The service mesh manages east-west traffic — calls between your internal services. Large systems use both: gateway at the edge, mesh inside.

Common misconceptions, cleared

"A service mesh replaces the API gateway." No — gateway = external/inbound (north-south), mesh = internal/service-to-service (east-west). Complementary.

"A mesh is free reliability." Sidecars add latency (an extra network hop per call) and resource overhead, and the mesh itself must be operated. For a handful of services, plain libraries are simpler.

Interview tip

Mention a service mesh only for large microservice estates: "with this many services, I'd use a mesh (Istio/Linkerd) for uniform mTLS, retries, and tracing via sidecars, instead of duplicating that logic everywhere." Then clarify gateway (north-south) vs mesh (east-west), and flag the overhead. Knowing when not to use it is as valuable as knowing what it does.

Key Takeaways

  • A service mesh moves service-to-service networking (retries, security, monitoring) into a shared layer.
  • It uses sidecar proxies (data plane) controlled by a central control plane, so app code stays clean.
  • Provides traffic management, security (mTLS), observability, and resilience uniformly.
  • Gateway handles north-south (external) traffic; mesh handles east-west (internal) traffic.
  • Tools: Istio, Linkerd. Use only with many services — it adds latency and operational complexity.

#53High Availability (HA)

One-line definition

High Availability (HA) means designing a system so it stays up and running almost all the time, even when individual parts fail. It is usually measured in "nines" of uptime (like 99.99%).

Users expect apps to "just work" 24/7. High availability is the goal of minimizing downtime by removing single points of failure and adding redundancy.

Measuring Availability: The "Nines"

Availability is the percentage of time the system is working. More nines = less downtime allowed per year:

AvailabilityNicknameDowntime per year
99%"two nines"~3.65 days
99.9%"three nines"~8.8 hours
99.99%"four nines"~52 minutes
99.999%"five nines"~5 minutes

How to Achieve High Availability

  • Redundancy: have backups for everything — multiple servers, databases, load balancers. No single component failure should take down the system.
  • Eliminate single points of failure (SPOF): find any one part that, if it dies, kills the whole system — and duplicate it.
  • Failover: if one component fails, automatically switch to a healthy backup.
  • Load balancing & health checks: route around dead servers (Topic 15).
  • Spread across zones/regions: run in multiple data centers so one outage does not take you down.
Real example

Banks, hospitals, and payment systems aim for very high availability because downtime is extremely costly. Netflix famously runs across multiple regions and even deliberately kills servers (Chaos Monkey) to prove it stays available when parts fail.

Availability vs Reliability

They are related but different. Availability = is it up right now? Reliability = does it work correctly without failing over time? A system can be available but unreliable (up, but giving wrong answers). You want both.

The Hidden Trap: Availability Multiplies

Here's a fact that surprises people. If your request must pass through 5 services in series, and each is 99.9% available, the overall availability isn't 99.9% — it's 0.9995 ≈ 99.5%. Dependencies multiply, so a chain is always less available than its weakest link. The lesson: every synchronous dependency you add lowers your ceiling. This is a powerful argument for fewer hard dependencies, async decoupling (queues), and graceful degradation (Topic 60) so a failed dependency doesn't drag your number down.

Redundancy: Active-Active vs Active-Passive

  • Active-passive: a standby sits idle and takes over only when the primary fails (failover). Simpler, but the standby is "wasted" capacity and failover takes a moment.
  • Active-active: all replicas serve traffic simultaneously; if one dies, the others already absorb the load. Better utilisation and instant tolerance, but harder (data must be consistent across all active nodes).

Why Each Extra Nine Costs Exponentially More

Going from 99% to 99.9% might just need redundant servers. But 99.99% → 99.999% can require multi-region deployment, automated failover, chaos testing, and 24/7 on-call — huge cost for ~50 fewer minutes of downtime a year. So you don't chase nines blindly; you pick a target that matches the business cost of downtime. A bank's core ledger needs five nines; an internal analytics dashboard is fine at two.

Common misconceptions, cleared

"More servers automatically means high availability." Only if there's no shared single point of failure. Ten app servers behind one load balancer, or all using one database, still have a SPOF.

"100% availability is the goal." It's practically impossible and absurdly expensive. Real targets are 99.9%–99.999%, chosen by business need.

Interview tip

State a concrete target ("I'll aim for ~99.9% / three nines") and justify it by the use case. Then list how: "redundancy with no SPOF, multi-AZ deployment, automated failover, and health checks." Mentioning that availability multiplies across dependencies is an advanced point that stands out.

Key Takeaways

  • High Availability = the system stays up almost all the time, even when parts fail.
  • Measured in "nines" — more nines means less allowed downtime, at exponentially higher cost.
  • Availability multiplies across serial dependencies — fewer hard dependencies = higher ceiling.
  • Achieved with redundancy (active-active/passive), no SPOF, failover, and multi-zone deployment.
  • Availability (is it up?) is different from reliability (does it work correctly?). Pick a target by business need.

#54Fault Tolerance

One-line definition

Fault tolerance is a system's ability to keep working correctly even when some parts fail. Instead of crashing, it absorbs the failure and continues (perhaps in a reduced way).

Failures are not "if" but "when" — servers crash, disks die, networks drop. Fault tolerance is about expecting failure and designing so it does not take the whole system down.

Real-life Analogy: An Airplane with Multiple Engines

Lose one engine, keep flying

A plane has multiple engines on purpose. If one fails, the others keep it flying safely to land. It does not fall out of the sky. Fault-tolerant systems are built the same way — lose a part, keep running.

How Systems Become Fault Tolerant

  • Redundancy: spare components ready to take over (like the spare engine).
  • Replication: multiple copies of data so losing one server does not lose data (Topic 26).
  • Failover: automatically switch to a backup when something fails.
  • Isolation: contain failures so one broken part does not spread (bulkhead, circuit breaker — later in this part).
  • Graceful degradation: drop non-essential features to keep core ones working (Topic 60).

Fault Tolerance vs High Availability

How they relate

Fault tolerance is about surviving failures correctly; high availability is about minimizing downtime. Fault tolerance is one of the main ways you achieve high availability. A fault-tolerant design keeps serving users through failures, which keeps availability high.

Real example

If one of YouTube's many servers crashes, you do not even notice — the load balancer routes you to another, and replicated data is still available. The failure is absorbed silently. That is fault tolerance in action.

The Mindset: "Design for Failure"

The deepest shift fault tolerance asks of you is to assume every component will fail and design accordingly. At scale this isn't pessimism — it's math: if you run 10,000 servers and each fails once every 3 years, you'll see a failure roughly every few hours. So failures aren't rare events to panic over; they're constant background noise the system must absorb automatically, without paging a human. Netflix made this famous with Chaos Monkey — a tool that randomly kills production servers on purpose, forcing engineers to build systems that shrug off failure. If you can't survive a random kill, you find out in a controlled test, not at 3am.

Redundancy Without Correlated Failure

A subtle but critical point: redundancy only helps if the copies don't fail together. Two servers in the same rack share one power supply and one network switch — lose those and both die at once (a correlated failure). That's why true fault tolerance spreads redundancy across availability zones and regions, uses different network paths, and avoids shared dependencies. Three replicas that all crash for the same reason give you the resilience of one.

Common misconceptions, cleared

"Fault tolerance means nothing ever fails." No — things fail constantly; fault tolerance means failures don't become user-visible outages. The parts fail; the system survives.

"Redundancy = fault tolerance." Only if failures are independent. Redundant copies sharing a SPOF (same rack, same DB, same region) fail together.

Interview tip

Sprinkle the "design for failure" mindset throughout: for each component, ask aloud "what happens if this dies?" and answer with redundancy/failover/degradation. Mentioning correlated failures (spread across AZs) and chaos testing shows senior-level reliability thinking.

Key Takeaways

  • Fault tolerance = keep working correctly even when parts fail.
  • Achieved via redundancy, replication, failover, isolation, and graceful degradation.
  • Design for failure — at scale failures are constant; chaos testing (Chaos Monkey) proves resilience.
  • Redundancy only helps against independent failures — spread across AZs/regions to avoid correlated ones.
  • Fault tolerance is a key way to achieve high availability.

#55Disaster Recovery (RTO / RPO)

One-line definition

Disaster Recovery (DR) is the plan for getting your system back after a major disaster (a whole data center burns down, a region goes offline, data gets corrupted). RTO and RPO are the two key targets that define how good your DR plan must be.

Fault tolerance handles small failures gracefully. Disaster recovery handles the big, catastrophic ones. It answers: "if everything goes wrong, how fast do we recover, and how much data can we afford to lose?"

The Two Key Numbers

  • RTO (Recovery Time Objective): how quickly must we be back up? E.g. "within 1 hour". It is about downtime you can tolerate.
  • RPO (Recovery Point Objective): how much data can we afford to lose? E.g. "at most 5 minutes of data". It is about how fresh your backups must be.
Easy way to remember

RTO = Time to recover (how long down). RPO = data Point you can roll back to (how much data lost). If your RPO is 5 minutes, you must back up at least every 5 minutes. If your RTO is 1 hour, your recovery process must complete within an hour.

Last backupRPO = data lost since here
→
๐Ÿ’ฅ Disaster
→
Back onlineRTO = time to get here
RPO looks backward (data lost); RTO looks forward (time to recover).

DR Strategies (cheaper/slower → pricier/faster)

  • Backup & Restore: keep backups, restore after disaster. Cheapest, but slow (large RTO).
  • Pilot Light: a minimal copy always running, scaled up when needed.
  • Warm Standby: a smaller running copy ready to take over quickly.
  • Hot Standby / Multi-site: a full duplicate running live; near-instant failover. Most expensive, smallest RTO/RPO.

RPO Drives Your Backup Frequency; RTO Drives Your Architecture

These two numbers aren't abstract — they directly dictate spending. RPO sets how often you must back up or replicate: a 5-minute RPO means continuous replication, while a 24-hour RPO means a nightly backup is fine. RTO sets how much standby infrastructure you keep running: a 5-minute RTO demands a hot standby ready to go, while a 2-day RTO lets you rebuild from backups. Tighter targets cost exponentially more, so you set them per system based on the cost of downtime/data loss — the payment ledger gets near-zero RPO, the analytics warehouse can lose a day.

DR Is Not Backups, and Untested DR Is a Myth

Two hard-won lessons from real outages: (1) High availability handles a server or zone dying; disaster recovery handles an entire region or your whole environment being lost (region outage, ransomware, fat-finger deletion). They're different scopes. (2) A DR plan you've never rehearsed will fail when you need it — backups that were silently corrupt, runbooks that are out of date, permissions nobody has. Mature teams run regular DR drills (even full region failovers) so recovery is proven, not hoped for.

Common misconceptions, cleared

"We have backups, so we have disaster recovery." Backups are necessary but not sufficient — DR also needs a tested restore process, somewhere to restore to, and an RTO/RPO you actually meet.

"Replication is a backup." No (repeating Topic 26) — replication faithfully copies a bad DELETE or corruption everywhere. You need point-in-time backups to roll back.

Interview tip

Quote concrete targets and tie them to strategy: "RPO of 5 min via continuous replication, RTO of 30 min via a warm standby in another region — and we run quarterly failover drills." Distinguishing DR (region-level) from HA (component-level), and mentioning testing, signals real operational maturity.

Key Takeaways

  • Disaster Recovery = the plan to recover from major catastrophes (region loss, corruption, ransomware).
  • RTO = how fast you recover (drives standby architecture). RPO = how much data you can lose (drives backup frequency).
  • Smaller RTO/RPO = safer but exponentially more expensive — set per system by cost of downtime.
  • Strategies range from cheap backup & restore to expensive hot standby; DR is broader than HA.
  • An untested DR plan doesn't really exist — rehearse with regular drills.

#56Circuit Breaker Pattern

One-line definition

A Circuit Breaker stops your system from repeatedly calling a service that is already failing. Like an electrical fuse, it "trips" to cut off the failing call, preventing the failure from cascading and giving the broken service time to recover.

When one service is down, naively retrying it again and again makes everything worse — requests pile up, threads get stuck, and the failure spreads. The circuit breaker prevents this "cascading failure".

Real-life Analogy: An Electrical Fuse

It trips to protect the house

When there is an electrical fault, the fuse trips and cuts the power, protecting the whole house from a fire. A software circuit breaker does the same: when a service keeps failing, it "trips" and stops sending calls to it, protecting the rest of the system.

The Three States

  • Closed (normal): calls flow through as usual. The breaker counts failures.
  • Open (tripped): too many failures, so the breaker blocks calls and fails fast (returns an error or fallback immediately) instead of waiting on the dead service.
  • Half-Open (testing): after a cooldown, it lets a few test calls through. If they succeed, it closes (back to normal); if they fail, it opens again.
Closedcalls flow
too many fails →
Openblock, fail fast
after wait →
Half-Opentest a few
The breaker trips open on repeated failures, then cautiously tests recovery.

Why It Helps

  • Prevents cascading failures — one dead service does not drag down the rest.
  • Fails fast — users get a quick error/fallback instead of long hangs.
  • Gives recovery time — the struggling service is not bombarded while it heals.

Often paired with a fallback (e.g. show cached data or a friendly message when the breaker is open).

The Failure It Prevents: a Cascade Walkthrough

To feel why this matters, trace a failure without a breaker. The Recommendations service slows to a crawl. Your Product page calls it synchronously, so each request now hangs for 30 seconds waiting. Those requests hold onto threads. Within seconds, every thread on every Product server is stuck waiting on Recommendations — so the Product page goes down too, even though it is perfectly healthy. Then services that call Product hang, and so on. One slow, non-critical service has taken down the whole site. This is a cascading failure, and it's one of the most common causes of large outages. The circuit breaker stops it at step one: after a few failures it trips open, Product instantly returns "recommendations unavailable" (a fallback), threads stay free, and the rest of the site stays up.

The Magic of "Fail Fast"

The counterintuitive insight: when something is broken, failing instantly is far better than trying hard. A slow hang ties up resources and spreads pain; an immediate error frees resources, gives the user a quick (if degraded) response, and protects the struggling dependency from being hammered while it tries to recover. "Fail fast" is a feature, not a defeat.

Common misconceptions, cleared

"Circuit breakers and retries do the same thing." Opposite, actually — retries (Topic 57) keep trying a call; a breaker stops trying once it's clearly hopeless. You use them together: retry transient blips, trip the breaker on sustained failure.

"A breaker fixes the broken service." No — it protects the callers and buys the broken service breathing room; you still must fix the root cause.

Interview tip

Whenever a design has service-to-service calls, say: "I'll wrap external/downstream calls in a circuit breaker with a fallback, so a slow dependency can't cascade into a full outage." Naming the three states and pairing it with a fallback + timeouts is exactly what resilience questions want.

Key Takeaways

  • A circuit breaker stops calls to a failing service, like an electrical fuse.
  • States: Closed (normal), Open (blocking, fail fast), Half-Open (testing recovery).
  • It prevents cascading failures — the #1 cause of one slow service taking down a whole system.
  • Fail fast beats slow hangs: it frees resources and shields the struggling service.
  • Best paired with a fallback, timeouts, and retries (which it complements, not duplicates).

#57Retry with Exponential Backoff

One-line definition

Retry with exponential backoff means: if a request fails, try again — but wait longer and longer between each retry (1s, 2s, 4s, 8s…) instead of hammering instantly. This recovers from temporary glitches without overwhelming the system.

Many failures are temporary (a brief network blip, a momentarily busy server). Retrying often fixes them. But retrying too aggressively can make a struggling system collapse. Exponential backoff retries politely.

Why Not Just Retry Immediately?

The danger of instant retries

If a server is overloaded and thousands of clients all retry immediately and repeatedly, they pile even more load on it — making it worse and preventing recovery (a "retry storm"). Backoff spreads the retries out so the server can breathe.

How Exponential Backoff Works

Each retry waits roughly double the previous wait:

Fail → wait 1s
→
Fail → wait 2s
→
Fail → wait 4s
→
Success or give up
Waits grow exponentially, giving the system time to recover.

Add "Jitter" (Randomness)

If everyone backs off by exactly the same amounts, they all retry at the same moments — causing synchronized spikes. Jitter adds a little randomness to each wait time so retries are spread out evenly. This is a well-known best practice.

Important Rules

  • Set a max retries / max wait — do not retry forever.
  • Only retry safe (idempotent) operations — retrying a payment could double-charge! (See idempotency in Topic 72/case studies.)
  • Combine with a circuit breaker — stop retrying entirely if the service is clearly down.

Seeing the Jitter Problem (the "thundering herd")

Imagine a popular service hiccups for 2 seconds, and 10,000 clients all fail at the same instant. With plain exponential backoff, all 10,000 wait exactly 1s, then all retry together — a synchronized wall of 10,000 requests. They fail again, all wait 2s, and slam it together again. The backoff didn't help because everyone is in lockstep. Jitter breaks the lockstep: each client waits a random amount within the window (e.g. anywhere from 0–1s), so the 10,000 retries spread smoothly over time instead of arriving as a spike. AWS's well-known guidance is that "full jitter" (fully randomised waits) dramatically improves recovery. Jitter is not optional polish — it's what makes backoff actually work at scale.

Retry Only Idempotent Operations — Why It's Critical

The most dangerous retry bug: the request actually succeeded, but the response got lost on the way back, so the client thinks it failed and retries. If the operation isn't idempotent, you've now done it twice — charged a card twice, sent two emails, created two orders. Safe retries require idempotency: design the operation so doing it twice equals doing it once (e.g. attach a unique idempotency key the server de-duplicates on). "Reads are safe to retry; writes need idempotency" is the rule of thumb.

Common misconceptions, cleared

"Just retry everything on failure." Dangerous — retrying non-idempotent writes causes duplicates, and retrying a hard error (like 400 Bad Request) is pointless. Retry only transient errors (timeouts, 503, 429-with-Retry-After).

"Backoff alone prevents overload." Not without jitter — synchronized backoff still produces spikes. And without a circuit breaker, you'll keep retrying a service that's fully down.

Interview tip

Say the full phrase: "retry transient failures with exponential backoff and jitter, capped, only for idempotent operations, behind a circuit breaker." Each qualifier (jitter, cap, idempotent, breaker) signals you've seen retries go wrong in production.

Key Takeaways

  • Retry with exponential backoff = retry failed requests with increasing wait times (1s, 2s, 4s…).
  • It recovers from temporary failures without causing a retry storm.
  • Jitter (randomness) is essential — without it, synchronized retries form a thundering-herd spike.
  • Only retry transient errors and idempotent operations (else lost-response retries cause duplicates).
  • Cap retries and pair with a circuit breaker to stop hammering a dead service.

#58Bulkhead Pattern

One-line definition

The Bulkhead Pattern isolates parts of a system into separate "compartments" so that if one part fails or gets overloaded, it cannot sink the whole system — the other compartments keep working.

The name comes from ships. This pattern is all about isolation — making sure one problem stays contained instead of spreading everywhere.

Real-life Analogy: A Ship's Watertight Compartments

Why the Titanic idea matters

Ships are divided into sealed bulkhead compartments. If one compartment floods, the watertight walls stop the water from spreading, and the ship stays afloat. Software bulkheads do the same: wall off resources so one flooded part does not sink everything.

How It Works in Software

You give each feature or dependency its own dedicated pool of resources (threads, connections, memory) instead of sharing one big pool. If one feature goes haywire and exhausts its pool, it only affects itself — other features have their own untouched pools.

Service A poolisolated
Service B pooloverloaded โœ—
Service C poolstill fine โœ“
Each part has its own resource pool, so one overload does not starve the others.

Example

Without vs with bulkheads

Suppose your app calls a slow third-party service. Without bulkheads: all your threads get stuck waiting on it, and your entire app freezes — even features that have nothing to do with that service. With bulkheads: only the small pool assigned to that service gets stuck; the rest of the app keeps serving users normally.

Works great with circuit breakers

Bulkheads (isolation) + circuit breakers (stop calling the dead service) + retries with backoff together form a powerful resilience toolkit. They are commonly used side by side.

Bulkheads vs Circuit Breakers (they solve different halves)

These two are easily confused but tackle different problems. A circuit breaker reacts to a dependency that is failing — it stops calling it. A bulkhead limits how many resources any single dependency can ever consume, whether it's failing or just slow. The classic case a breaker doesn't fully cover: a dependency that is slow but not erroring. It returns success eventually, so the breaker never trips — but meanwhile it's hogging all your threads. The bulkhead caps that dependency to, say, 20 threads, so even a slow-but-"healthy" service can never starve the other 180 threads serving everyone else. Use both: bulkheads contain the blast radius, breakers cut off the dead.

Levels of Bulkheading

  • Thread/connection pools per dependency (most common): each downstream call gets its own bounded pool.
  • Separate service instances per tenant/feature: e.g. a separate pool of servers for free users vs paying customers, so a free-tier traffic surge can't degrade paying customers.
  • Full physical isolation (cells/shards): partition all users into independent "cells", so an incident affects only one cell, not everyone — used by AWS, Slack, and others ("cell-based architecture").
Common misconceptions, cleared

"Bulkheads reduce total capacity." Slightly, yes — you can't pool everything together. That's the deliberate trade-off: a bit less peak efficiency for a lot more failure isolation.

"A circuit breaker makes bulkheads unnecessary." No — breakers handle failing dependencies; bulkheads also handle slow-but-succeeding ones and noisy-neighbour resource hogging.

Interview tip

Bring up bulkheads for the "noisy neighbour" / blast-radius question: "I'll isolate each dependency (and tenant tier) into its own resource pool, so one slow or busy part can't starve the rest." Mentioning cell-based architecture for limiting blast radius is an advanced, impressive touch.

Key Takeaways

  • The bulkhead pattern isolates parts into compartments so one failure cannot sink the whole system.
  • Each part gets its own resource pool, so one overload (even a slow-but-healthy one) does not starve the rest.
  • It complements circuit breakers: breakers cut off failing deps, bulkheads contain slow/greedy ones.
  • Scales up to cell-based architecture to limit blast radius across all users.
  • Named after a ship's watertight compartments; pairs with breakers and retries.

#59Health Checks & Heartbeats

One-line definition

Health checks and heartbeats are ways for a system to constantly know which components are alive and working, so it can route traffic away from dead ones and replace them automatically.

To stay highly available, a system must know the state of its parts at all times. Health checks and heartbeats are how it keeps a live picture of what is healthy.

Health Checks

A health check is a small endpoint (like /health) that a monitor calls periodically. The service replies "I'm OK" (HTTP 200) or signals trouble. Two levels:

  • Liveness check: "Are you alive at all?" If not, restart the service.
  • Readiness check: "Are you ready to handle traffic?" (e.g. finished starting up, database connected). If not ready, do not send it requests yet.

Heartbeats

A heartbeat is the reverse direction: each component periodically sends a small "I'm alive" signal to a monitor or to its peers. If the heartbeats stop coming, the system assumes that component is dead and reacts (removes it, fails over, replaces it).

Server
๐Ÿ’“ every few sec →
Monitorno beat = dead
Missing heartbeats tell the monitor a component has likely failed.

Why They Matter

  • Load balancers use health checks to stop routing to dead servers (Topic 15).
  • Orchestrators like Kubernetes use them to restart or replace unhealthy containers automatically.
  • Service registries use them to remove dead instances (Topic 51).
  • They enable automatic recovery — the system heals itself without humans.

Why Liveness vs Readiness Is a Real Bug Source

Mixing these up causes nasty production issues, so understand the difference concretely. Liveness = "is the process broken beyond repair?" If it fails, the orchestrator kills and restarts the container. Readiness = "can it serve traffic right now?" If it fails, the orchestrator stops sending traffic but does not restart it. The classic mistake: putting "is my database reachable?" in the liveness check. Now if the database has a brief blip, every app instance is declared dead and restarted simultaneously — turning a small DB hiccup into a full app outage. The right design: liveness checks only "am I, the process, fundamentally alive?"; readiness checks dependencies like the DB so a struggling instance is quietly taken out of rotation until it recovers, without a restart storm.

A Good Health Check Is "Shallow"; a Deep One Is Dangerous

It's tempting to make /health check everything (DB, cache, every downstream service). But a "deep" health check means one slow dependency makes all your servers report unhealthy at once — and the load balancer pulls every server out, taking you fully down even though the servers are fine. Prefer shallow liveness checks plus targeted readiness checks, and let circuit breakers/graceful degradation handle dependency problems rather than failing the whole health check.

The Timing Trade-off

How often to check, and how many misses = "dead"? Too aggressive (check every 1s, 1 miss = dead) causes false positives — a momentary GC pause kills a healthy server. Too lax (check every 60s, 5 misses) means a truly dead server keeps receiving traffic for minutes. Real systems tune this balance (e.g. check every 5–10s, mark dead after 2–3 misses).

Common misconceptions, cleared

"Liveness and readiness are the same check." No — failing liveness restarts the instance; failing readiness just removes it from traffic. Confusing them causes restart storms.

"Health checks should verify every dependency." Risky — deep checks let one shared dependency mark every server unhealthy at once. Keep liveness shallow.

Interview tip

Show you know the distinction: "shallow liveness so a DB blip doesn't restart every pod, plus readiness that gates traffic on dependencies." Mentioning false-positive tuning (interval × failure threshold) and the deep-health-check trap demonstrates real operational experience.

Key Takeaways

  • Health checks = a monitor asks a service "are you OK / ready?".
  • Heartbeats = a component regularly signals "I'm alive"; silence means it is presumed dead.
  • Liveness = alive? (fail → restart). Readiness = can take traffic? (fail → remove from rotation, no restart).
  • Keep liveness shallow — deep checks let one dependency mark every server unhealthy and cause an outage.
  • Tune interval × failure-threshold to balance fast detection vs false positives; they power self-healing.

#60Graceful Degradation

One-line definition

Graceful degradation means that when part of a system fails or is overloaded, the system drops non-essential features but keeps the core working — instead of crashing completely.

The opposite of graceful degradation is total collapse: one feature breaks and the whole app goes down. Graceful degradation keeps the most important things alive even when things go wrong.

Real-life Analogy: A Car in "Limp Mode"

Reduced power, but you still get home

When a car detects an engine problem, it does not just stop dead on the highway. It enters "limp mode" — reduced power, some features off — but still lets you drive slowly to safety. Graceful degradation is your system's limp mode: keep the essentials running.

How It Looks in Practice

  • Show stale/cached data when the live database is down, instead of an error.
  • Disable a heavy feature (like personalized recommendations) under high load, while core browsing still works.
  • Serve a simpler page (no images/video) if those services fail.
  • Return a friendly fallback message instead of a crash.
Real example: an e-commerce site on sale day

During a massive sale, the recommendation engine ("you may also like") might be turned off to save capacity — but the critical path (browse, add to cart, checkout) keeps working perfectly. Users may not even notice the missing recommendations, and sales continue.

Why It Matters

  • Better user experience — a partly-working app beats a dead one.
  • Protects revenue — core money-making features stay up.
  • Buys time — the system survives the spike or failure while engineers fix the rest.
Decide priorities in advance

Graceful degradation works only if you have decided ahead of time which features are essential (must never fail) and which can be sacrificed. Identify your "core path" and protect it.

Load Shedding: Degrading on Purpose

A powerful form of graceful degradation is load shedding — when overloaded, the system deliberately rejects some work to protect the rest. It sounds harsh, but the alternative is worse: if you accept everything during a flood, every request gets slower until the whole system collapses and nobody gets served. By shedding (e.g. rejecting low-priority requests with a quick 503, or serving a lighter version), the system keeps the most important traffic fast and stays alive. The principle: it is better to serve 80% of users well than 100% of users a crash. This is why ticket sites show a queue page, and why APIs return 429s under extreme load — controlled degradation beats uncontrolled collapse.

Real-world Examples of Degradation in Action

  • Netflix: if the personalised-recommendations service is down, it falls back to showing popular/unpersonalised rows — you still get something to watch instead of an error.
  • Amazon on sale day: non-essential widgets (detailed reviews, "customers also bought") may be dropped to protect browse → cart → checkout.
  • Maps/ride apps: if live traffic data fails, show the route without traffic colours rather than no map.
Common misconceptions, cleared

"Degradation just happens automatically." No — it must be designed: you have to build fallbacks, feature toggles, and decide the core path in advance. By default, software fails hard (one error crashes the request).

"Showing stale data is always bad." For non-critical data it's far better than an error. (But never silently show stale money/inventory — there, fail clearly instead.)

Interview tip

For any design, name the core path that must never fail and the nice-to-haves that can be shed: "checkout must stay up; recommendations and reviews can degrade." Pair it with circuit breakers (open → serve fallback) and load shedding under overload. This shows you optimise for user/business value during failures, not just uptime.

Key Takeaways

  • Graceful degradation = drop non-essential features to keep the core running under failure/load.
  • Like a car's limp mode — reduced, but still functional.
  • Load shedding = deliberately reject excess/low-priority work so the important traffic survives.
  • Must be designed (fallbacks, toggles); it doesn't happen by default. Pairs with circuit breakers.
  • Decide your essential "core path" in advance and protect it — serve 80% well over crashing for all.

#61Strong vs Eventual Consistency

One-line definition

When data is copied across many servers, consistency is about whether everyone sees the same value. Strong consistency = everyone always sees the latest value immediately. Eventual consistency = copies may briefly differ but will all agree soon.

This builds directly on ACID/BASE and CAP. When you replicate data, you face a choice: guarantee everyone sees the newest value right away (slower), or let copies catch up over time (faster, more available).

Strong Consistency

The moment a write succeeds, every read anywhere returns that new value. The system waits until all (or enough) copies are updated before confirming. Correct and predictable, but slower and less available (it may block during network issues).

  • Use for: bank balances, inventory, anything where a stale value is dangerous.

Eventual Consistency

A write updates one copy and confirms quickly; other copies catch up shortly after. For a brief window, different users might see different values, but eventually all copies converge to the same value.

  • Use for: like counts, social feeds, view counts — where a slightly old value is harmless.
Analogy: telling friends news

Strong: you gather all your friends in one room and announce the news at once — everyone knows simultaneously, but it took effort to assemble them. Eventual: you tell one friend, who tells another, and so on — fast and easy, but for a while some know and some do not. Soon, everyone knows.

PointStrongEventual
ReadsAlways latestMay be briefly stale
SpeedSlowerFaster
AvailabilityLowerHigher
Best forMoney, inventoryLikes, feeds, counts

There are also middle grounds like read-your-own-writes (you always see your own changes immediately, even if others see them slightly later).

The Consistency Spectrum (it's not just two options)

"Strong" and "eventual" are the two ends, but real systems offer a whole spectrum of useful in-between guarantees. Knowing the named ones makes you sound precise:

  • Read-your-own-writes: you always see your own updates immediately (others may lag). Fixes the "I edited my profile but it shows the old one" bug from Topic 26.
  • Monotonic reads: once you've seen a value, you'll never see an older one later (no "time travel" backwards).
  • Causal consistency: things that are cause-and-effect appear in the right order everywhere (you never see a reply before the message it replies to), while unrelated events can be in any order.

These give a much better user experience than raw eventual consistency, without the full cost of strong consistency — which is why apps like social networks lean on them.

Why Strong Consistency Costs Latency (and availability)

The reason strong consistency is "slower" is concrete: to guarantee every reader sees the latest write, the system must coordinate — the write has to reach a majority of replicas (a quorum, Topic 68) before it's confirmed, and that coordination takes round-trips, especially across regions. Worse, during a network partition (CAP, Topic 29), a strongly-consistent system may have to reject requests rather than risk serving stale data — trading availability for correctness. Eventual consistency skips the wait: confirm immediately, reconcile later.

Common misconceptions, cleared

"Eventual consistency = your data is wrong / will be lost." No — it's a timing property; data is durable and converges, usually within milliseconds.

"Strong consistency means the system is always correct." It means reads are always current — but it can become unavailable during partitions, and it's slower. Correctness of reads, not a free lunch.

"It's a global setting." Often it's chosen per operation (e.g. DynamoDB lets you request a strong or eventual read on each call).

Interview tip

Pick consistency per feature and name the level: "strong for the wallet balance; read-your-own-writes for profile edits; eventual for the like counter." Mentioning intermediate levels (read-your-own-writes, causal) instead of only the two extremes signals genuine distributed-systems depth.

Key Takeaways

  • Strong consistency: everyone sees the latest value immediately — correct but slower and less available.
  • Eventual consistency: copies briefly differ, then converge — faster and more available.
  • There's a spectrum: read-your-own-writes, monotonic reads, causal consistency sit usefully in between.
  • Strong consistency costs latency because it must coordinate a quorum; often chosen per operation.
  • Choose based on whether a brief stale value is acceptable — the practical face of CAP/PACELC.

#62Distributed Transactions (Two-Phase Commit)

One-line definition

A distributed transaction spans multiple services/databases that must all succeed or all fail together. Two-Phase Commit (2PC) is a protocol to coordinate this: first everyone agrees they can do it, then everyone actually does it.

ACID transactions are easy within one database. But what if a single operation must update several separate databases (e.g. deduct stock in one, charge payment in another)? You need them all to commit together — that is a distributed transaction.

Real-life Analogy: Friends Agreeing on a Plan

"Can everyone make it?" then "OK, it's on!"

To plan an outing, a coordinator first asks everyone "Can you come Saturday?" (phase 1). Only if everyone says yes does the coordinator announce "OK, confirmed!" (phase 2). If even one says no, the plan is cancelled for all. That is exactly 2PC.

The Two Phases

  • Phase 1 — Prepare (voting): a coordinator asks every participant "are you ready to commit?" Each one does the work tentatively and replies Yes (ready) or No.
  • Phase 2 — Commit/Abort: if all said Yes, the coordinator tells everyone "Commit!" and they finalize. If any said No, it tells everyone "Abort!" and they roll back.
Coordinator
"ready?" →
All vote Yes
"commit!" →
All commit
Everyone must agree in phase 1 before anyone commits in phase 2.

The Big Problem with 2PC

It blocks, and the coordinator is a SPOF

2PC guarantees correctness but is slow and blocking: participants must lock resources and wait. Worse, if the coordinator crashes at the wrong moment, participants can be stuck waiting indefinitely. Because of these issues, large modern systems often avoid 2PC and use the Saga pattern (next topic) instead.

The Exact Moment 2PC Gets Stuck

Let's pinpoint why 2PC is feared. After phase 1, every participant has voted "Yes" and is now holding locks, waiting to be told commit or abort. If the coordinator crashes right then, the participants are stuck in limbo: they can't commit (maybe someone else voted No) and can't abort (maybe everyone voted Yes) — so they keep their locks and block, sometimes forever, until the coordinator returns. Meanwhile those locked rows can't be used by anyone else. This "blocking" property is 2PC's fundamental flaw, and it's why holding locks across network boundaries is so dangerous compared to a single-database transaction.

So When Is 2PC Still Used?

It hasn't vanished — 2PC is fine when you need true atomic correctness across a small number of tightly-coupled, reliable participants in one data center (e.g. a couple of databases, or a DB + a message broker via XA transactions). It's avoided for high-scale, geographically-spread microservices, where its locking and blocking would cripple throughput. Rule of thumb: 2PC for a few local resources that must be perfectly atomic; sagas for many distributed services.

Common misconceptions, cleared

"2PC guarantees you can never get stuck." The opposite — its known weakness is exactly that participants can block indefinitely if the coordinator fails after voting.

"2PC scales fine." No — locks held across the network plus the coordinator round-trips make it slow and poorly-scaling; it's a small-scale, strong-correctness tool.

Interview tip

If asked to keep multiple services consistent, say: "2PC gives atomicity but blocks and doesn't scale, so for microservices I'd use a Saga with compensating actions and accept eventual consistency." Knowing why 2PC is avoided (blocking + coordinator SPOF) is the key insight interviewers want.

Key Takeaways

  • A distributed transaction must succeed or fail across multiple databases together.
  • 2PC = Phase 1 "everyone prepare & vote", Phase 2 "all commit or all abort".
  • It guarantees correctness but is slow, blocking (locks held across the network), and the coordinator is a SPOF.
  • Still fine for a few reliable local resources; avoided for high-scale distributed microservices.
  • Modern large systems often prefer the Saga pattern instead.

#63Saga Pattern

One-line definition

The Saga Pattern handles transactions across microservices by breaking them into a series of small local steps, each with a matching "undo" (compensating) action. If a later step fails, the saga runs the undos to reverse the earlier steps.

2PC locks everything and is fragile. The saga takes a different approach: do each step, and if something fails midway, undo the completed steps one by one. No global locks — great for microservices.

Real-life Analogy: Booking a Trip

Book flight, hotel, car — undo if one fails

You book a flight โœ“, then a hotel โœ“, then try a rental car โœ— (sold out). You cannot leave the trip half-booked. So you undo: cancel the hotel, cancel the flight. Each booking has a matching cancellation. The saga pattern automates exactly this "do, and undo if needed".

How It Works

Each service does its own local transaction and then triggers the next step. If any step fails, compensating transactions run in reverse to undo all the previous successful steps, returning the system to a consistent state.

Charge โœ“
→
Reserve stock โœ“
→
Ship โœ— fails
↓ compensate (undo)
Release stock
→
Refund charge
On failure, run compensating actions to undo completed steps.

Two Ways to Coordinate a Saga

  • Choreography: each service listens for events and reacts, triggering the next step. No central boss — simple for few steps, but can get tangled.
  • Orchestration: a central "orchestrator" tells each service what to do and when. Easier to understand and manage for complex flows.
Saga vs 2PC

2PC: strong consistency, locks resources, blocking. Saga: eventual consistency, no locks, resilient — better for microservices. The trade-off: with sagas you accept brief inconsistency and must carefully design the undo steps.

Choreography vs Orchestration: When to Use Which

  • Choreography (event-driven): no central boss — each service emits events and others react. Great for simple, few-step flows; fully decoupled. Downside: with many steps the logic is scattered everywhere and becomes hard to follow ("where does this flow even go next?").
  • Orchestration (central controller): one orchestrator explicitly drives the steps and the undos. Easier to understand, monitor, and debug complex flows; the trade-off is the orchestrator is a component you must build and keep available. Most non-trivial sagas use orchestration.

The Hard Parts of Designing Sagas

Sagas trade locking complexity for design complexity. Things you must handle:

  • Compensations must exist and be reliable: every step needs a sensible undo — and some things can't truly be undone (you can't un-send an email; you send an apology/cancellation instead).
  • Idempotency: steps and compensations may be retried, so they must be safe to run twice (Topic 57).
  • Intermediate visibility: because there are no locks, others can see the half-finished state (e.g. an order briefly "pending"). You design the UX around this eventual consistency.
Common misconceptions, cleared

"A saga rolls back like a database transaction." No — there's no automatic rollback. You write explicit compensating actions that semantically undo each step.

"Sagas give the same guarantees as ACID." No — you get eventual consistency with a temporarily-visible in-between state, not atomic isolation.

Interview tip

For multi-service workflows (order → payment → inventory → shipping), propose a saga, pick orchestration for clarity, and explicitly list compensating actions for each step ("if shipping fails, release stock and refund payment"). Mentioning idempotent steps and the visible intermediate state shows you've thought it through.

Key Takeaways

  • The saga pattern = a sequence of local steps, each with a compensating (undo) action.
  • If a step fails, earlier steps are undone to restore consistency — no global locks.
  • Choreography (events) suits simple flows; orchestration (central controller) suits complex ones.
  • You must design reliable, idempotent compensations and accept a visible intermediate state.
  • Preferred over 2PC for microservices; trade-off is eventual consistency.

#64Consensus Algorithms (Paxos, Raft)

One-line definition

A consensus algorithm lets a group of distributed servers agree on a single value or decision, even when some servers fail or messages are delayed. Paxos and Raft are the two famous algorithms that solve this.

In a distributed system, many servers must agree on things — like "who is the leader?" or "what is the correct order of these writes?". Getting separate machines to reliably agree, despite failures, is surprisingly hard. Consensus algorithms make it possible.

Real-life Analogy: A Group Deciding Where to Eat

Reaching one decision everyone honors

A group of friends must pick one restaurant. Some are not answering their phones, messages arrive late, opinions differ. Yet they need a single, final decision everyone will follow. Consensus algorithms are a rigorous way to reach that one agreed decision despite the chaos.

The Core Idea: Majority Agreement

Consensus works by majority vote (quorum). A decision is final once more than half the servers agree. Because any two majorities overlap, you can never have two conflicting decisions accepted — this is the key trick that keeps everyone consistent.

Raft — The Understandable One

Paxos works but is famously hard to understand. Raft was designed to do the same job in a clearer way, and is widely used today. Raft breaks consensus into:

  • Leader election: the servers elect one leader (next topic).
  • Log replication: the leader receives changes and copies them to followers.
  • Safety: a change is committed once a majority of followers have stored it.
Where consensus is used

Systems like etcd, ZooKeeper, Consul, and many distributed databases use consensus (often Raft) to stay consistent — for leader election, configuration, and agreeing on the order of operations. Kubernetes stores its cluster state in etcd, which uses Raft.

Why a Majority? The Overlap Trick

The genius of quorum-based consensus is simple once you see it: any two majorities of the same group must share at least one member. With 5 servers, any 3 (a majority) and any other 3 always overlap in at least one server. That shared member can't agree to two conflicting decisions, so two contradictory values can never both be accepted. This also explains why you typically run an odd number of nodes (3, 5, 7): 5 nodes tolerate 2 failures (3 still form a majority), while 6 nodes also only tolerate 2 — the extra node buys nothing but costs more. And if too many nodes are down to form a majority, the system stops accepting writes rather than risk inconsistency (choosing consistency over availability, CAP-style).

The Cost: Consensus Is Expensive, So Use It Sparingly

Every consensus decision needs multiple network round-trips to a majority, so it's much slower than a normal write. That's why systems don't run all their data through Raft. Instead they use consensus for the small but critical things — "who is the leader?", cluster configuration, lock ownership, metadata — and let the elected leader handle the high-volume data path more cheaply. Consensus is the trustworthy-but-slow committee; you don't ask it about every tiny thing.

Common misconceptions, cleared

"Consensus needs all servers to agree." No — only a majority. That's the whole point: it keeps working even when a minority is down or unreachable.

"Raft and Paxos do different jobs." They solve the same problem (agreement under failures); Raft is just designed to be far easier to understand and implement correctly.

"More nodes = more fault tolerance always." Going from 5 to 6 nodes does not increase failures tolerated (still 2). Use odd numbers.

Interview tip

You rarely implement Raft yourself — the win is knowing where it lives: "I'll store leader/config/lock state in a consensus system like etcd/ZooKeeper (Raft), with an odd number of nodes for majority quorum." Mentioning the odd-node rule and that consensus is reserved for critical metadata (not bulk data) shows real understanding.

Key Takeaways

  • Consensus algorithms let distributed servers agree on one decision despite failures.
  • They rely on majority (quorum) agreement — any two majorities overlap, so conflicting decisions can't both win.
  • Run an odd number of nodes (3/5/7); the system stops writing rather than risk inconsistency without a majority.
  • Consensus is slow — reserve it for critical metadata (leader, config, locks), not bulk data.
  • Paxos is the classic (hard); Raft is the modern, understandable standard. Used by etcd, ZooKeeper, Consul.

#65Leader Election

One-line definition

Leader election is the process by which a group of servers chooses one of them to be the "leader" (the one in charge of coordinating), and automatically picks a new leader if the current one fails.

Many distributed tasks need a single coordinator to avoid chaos — one server to make decisions, assign work, or accept writes. Leader election is how the group fairly and reliably designates that one server.

Why You Need a Leader

Analogy: a team needs a captain

If every player on a team made independent decisions, there would be confusion. A single captain coordinates the strategy. But if the captain leaves the field, the team quickly picks a new one so play continues. Servers do the same: one leader coordinates; if it dies, a new election happens.

What the Leader Typically Does

  • Accepts all writes and decides their order (then replicates to followers).
  • Coordinates tasks so two servers do not do the same job twice.
  • Acts as the single source of truth for decisions.

How Election Works (Simply)

Followers expect regular heartbeats from the leader. If the leader's heartbeats stop, a follower suspects it is dead and starts an election — asking others to vote for it. Whoever gets a majority of votes becomes the new leader. (This is exactly how Raft does it.)

Leader dies ๐Ÿ’ฅ
→
Followers notice (no heartbeat)
→
Vote → new leader
Missing heartbeats trigger an election; a majority vote picks the new leader.
The "split-brain" danger

A risk in leader election is split-brain: two servers both think they are the leader (e.g. after a network split), causing conflicts. Requiring a majority to elect a leader prevents this — only one side of a split can have a majority.

Split-Brain, Made Concrete

Picture 5 nodes with leader L. A network partition splits them into a group of 2 (with L) and a group of 3. The group of 2 can't reach the others — but old L is still happily acting as leader on its side. Meanwhile the group of 3 sees no heartbeats, holds an election, and (having a majority) elects a new leader. Now there are two leaders accepting writes — split-brain — and their data diverges, which is a nightmare to reconcile. The majority rule is the cure: the minority side (2 nodes) can never get 3 votes, so it must step down and stop accepting writes. Only the majority side stays operational. This is why you need an odd number of nodes and majority quorum — it guarantees at most one leader.

Fencing: Stopping a "Zombie" Old Leader

One more subtlety: an old leader might be frozen (e.g. a long pause) and "wake up" still believing it's leader, after a new one was elected. To stop this zombie from corrupting data, systems use fencing tokens — each leadership term gets an increasing number, and downstream systems reject any request carrying an older token. So even if the old leader acts, its stale token is refused.

Common misconceptions, cleared

"A leader makes the system a single point of failure." No — the role is single, but it's automatically re-elected on failure, so the system keeps going. The leader is a logical role, not a fixed machine.

"Just pick the lowest-ID server as leader." Naive schemes break under partitions (split-brain). Real election needs majority quorum (and often fencing) to be safe.

Interview tip

Whenever your design has a "primary" (DB primary, coordinator, scheduler), mention "leader election via a majority quorum (e.g. Raft/ZooKeeper) with automatic failover, and fencing tokens to avoid split-brain." Naming split-brain and how majority quorum prevents it is a classic distributed-systems checkpoint.

Key Takeaways

  • Leader election picks one server to coordinate, and re-elects if it fails.
  • The leader typically orders writes and acts as the single source of truth.
  • Triggered by missing heartbeats; the winner needs a majority of votes.
  • Majority voting prevents split-brain (the minority side must stop accepting writes).
  • Fencing tokens stop a stale "zombie" old leader from corrupting data after a new election.

#66Consistent Hashing

One-line definition

Consistent hashing is a clever technique for spreading data across servers so that when you add or remove a server, only a small fraction of data needs to move — instead of almost everything. It is the backbone of distributed caches and databases.

This is a famous, must-know system design topic. It solves a real headache in sharding: what happens to your data when the number of servers changes?

The Problem with Simple Hashing

Why "hash % N" breaks

A simple way to assign data to servers is server = hash(key) % N (N = number of servers). The disaster: if N changes (you add/remove a server), the % N result changes for almost every key, so nearly all data must move to new servers at once. That causes a massive reshuffle and cache misses everywhere.

The Solution: A Hash Ring

Consistent hashing places both servers and data keys on an imaginary circle (a "ring") using a hash. Each key is stored on the next server clockwise from it. The magic: when a server is added or removed, only the keys in its immediate neighborhood move — everything else stays put.

Hash Ring ๐Ÿ”ตservers & keys placed around a circle
Add a server
→
Only nearby keys moverest unaffected
On the ring, a key goes to the next server clockwise; changes affect only a small slice.

Virtual Nodes (the practical refinement)

To keep the load even (so one server does not accidentally get a huge slice of the ring), each physical server is placed at many points on the ring as "virtual nodes". This spreads data more uniformly and makes rebalancing smoother.

Where it is used

Consistent hashing powers distributed caches and databases like Cassandra, DynamoDB, and large-scale caching layers. Whenever you need to scale a cluster up or down with minimal disruption, this is the technique.

The Numbers That Make It Click

Compare the two approaches concretely with 4 servers becoming 5. With hash % N, changing N from 4 to 5 changes the bucket for roughly 80% of all keys — a near-total reshuffle, and for a cache that means almost everything becomes a miss at once (a stampede onto your database). With consistent hashing, adding one server to a 4-server ring only moves about 1/5 of the keys (the slice now owned by the newcomer); the other ~80% stay exactly where they were. In general, adding the Nth server moves only ~1/N of the data. That difference — "move almost everything" vs "move a small slice" — is the entire reason the technique exists.

Why Virtual Nodes Are Essential, Not Optional

With only a few servers placed once each on the ring, the gaps between them are uneven by luck of hashing — one server might own a huge arc (and get hammered) while another owns a tiny one. Virtual nodes fix this by placing each physical server at many points (say 100–200) around the ring. The law of averages then makes each server's total share roughly equal, and when a server dies its load is spread across many others rather than dumped entirely on its single clockwise neighbour. Real systems always use virtual nodes for this reason.

Common misconceptions, cleared

"Consistent hashing perfectly balances load." Not by itself — plain consistent hashing can be lumpy; it's virtual nodes that smooth the distribution.

"It eliminates all data movement." No — it minimises it to ~1/N. Some keys still move when membership changes; that's unavoidable and fine.

"It's only for caches." It also underpins data placement in Cassandra/DynamoDB and request routing in many load balancers.

Interview tip

This is a top-tier favourite. When sharding a cache/DB, say: "I'll use consistent hashing with virtual nodes so scaling the cluster only remaps ~1/N of keys instead of nearly all of them, avoiding a cache-miss stampede." Contrasting it explicitly with hash % N is exactly the answer interviewers are listening for.

Key Takeaways

  • Consistent hashing spreads data so adding/removing a server moves only a small fraction (~1/N) of keys.
  • It avoids the ~80% reshuffle and cache-miss stampede caused by simple hash % N.
  • Uses a hash ring: each key goes to the next server clockwise.
  • Virtual nodes are essential for even load and smooth rebalancing on failure.
  • Used by Cassandra, DynamoDB, distributed caches, and some load balancers.

#67Vector Clocks & Logical Clocks

One-line definition

Logical clocks (and vector clocks) are ways to figure out the order of events across different servers without relying on real-world time, because clocks on different machines are never perfectly in sync.

In a distributed system you often need to know "did event A happen before event B?". You cannot trust wall-clock timestamps, because each server's clock drifts slightly. Logical clocks solve ordering using counters instead of time.

The Problem with Real Clocks

Clocks disagree

Server A's clock might say 10:00:01 and Server B's might say 10:00:00 for an event that actually happened later. Relying on these timestamps to order events leads to wrong conclusions. We need a method that does not depend on synchronized time.

Lamport (Logical) Clocks

Each server keeps a simple counter. Every time something happens, it increments. When servers send messages, they include their counter, and the receiver bumps its own counter to stay ahead. This guarantees: if A causes B, then A's number is smaller than B's. It captures "happened-before" ordering.

Limitation: Lamport clocks can tell you a possible order, but cannot always tell whether two events were truly concurrent (unrelated) or one really caused the other.

Vector Clocks

Vector clocks fix that. Instead of one counter, each server keeps a vector (a list) of counters — one slot per server. By comparing two vectors, you can tell precisely whether one event happened before another, or whether they were concurrent (a conflict that needs resolving).

Where it is used

Vector clocks help detect conflicting concurrent updates in systems like DynamoDB and Cassandra. For example, if two users edit the same item at the same time on different replicas, vector clocks reveal the conflict so the system can resolve it (e.g. keep both versions or merge them).

Why "Just Use Timestamps" Fails — the LWW Trap

A tempting shortcut is Last-Write-Wins (LWW): attach a wall-clock timestamp to each write and keep the "latest". The problem: clocks drift between servers (clock skew). So a write that truly happened later might carry an earlier timestamp and get thrown away — meaning you can silently lose data that should have won. LWW is simple and sometimes acceptable (Cassandra uses it by default), but it can quietly drop updates. Logical/vector clocks avoid this by ordering events on causality, not unreliable time.

What "Concurrent" Really Means Here

The key concept is "happened-before". If event A could have influenced B (e.g. B read A's result, or they're on the same machine in sequence), then A happened-before B. If neither could have influenced the other, they are concurrent — and concurrent writes to the same data are a genuine conflict that no ordering can resolve on its own. Vector clocks' superpower is detecting exactly this: comparing two vectors tells you "A before B", "B before A", or "concurrent — you must decide". The system then resolves it by keeping both versions (siblings, like Dynamo) for the app/user to merge, or applying a merge rule (CRDTs).

Common misconceptions, cleared

"Timestamps are good enough to order events." Not across machines — clock skew makes them unreliable, and naive LWW can lose data.

"Vector clocks resolve conflicts." They only detect conflicts (concurrency). Resolution (merge, keep-both, business rule) is a separate decision you must make.

"This is rarely relevant." Any multi-leader / multi-region writable system (Dynamo-style) hits concurrent writes — this is core to how they stay correct.

Interview tip

If a design allows concurrent writes to the same data across replicas, mention: "I can't rely on timestamps due to clock skew; I'd use vector clocks (or version vectors) to detect concurrent conflicts, then resolve via merge / keep-both / a business rule." Distinguishing detecting from resolving conflicts is the nuance that impresses.

Key Takeaways

  • Real machine clocks drift, so they cannot reliably order distributed events (and naive Last-Write-Wins can lose data).
  • Lamport clocks use a counter to capture "happened-before" ordering.
  • Vector clocks use a counter per server, so they can also detect concurrent (conflicting) events.
  • They detect conflicts; resolving them (merge / keep-both / rule) is a separate choice.
  • Used to detect and resolve conflicts in systems like DynamoDB and Cassandra.

#68Quorum-based Reads/Writes

One-line definition

A quorum is the minimum number of servers that must agree for a read or write to be accepted. By requiring a majority for both, a distributed system can stay consistent even with replicated data.

When data is replicated across many nodes, how many copies must respond before you trust a read or confirm a write? Quorums give a precise, tunable answer that balances consistency, speed, and availability.

The Formula: W + R > N

Let:

  • N = total number of replicas holding the data.
  • W = how many replicas must confirm a write before it is considered done.
  • R = how many replicas must respond to a read.

The key rule: if W + R > N, then any read is guaranteed to overlap with the latest write — so you always read fresh data. This is how you get strong consistency from replicas.

Simple example (N = 3)

With 3 replicas, set W = 2 and R = 2. Since 2 + 2 = 4 > 3, any read of 2 replicas must include at least one that has the newest write. So you never miss the latest value. You also tolerate one replica being down for both reads and writes.

Tuning the Trade-off

SettingEffect
High WSafer writes, but slower writes
High RFresher reads, but slower reads
W=1, R=1Very fast, but eventual (may read stale)
W + R > NStrong consistency guaranteed
Why this is powerful

Quorums let you dial the balance between consistency and availability per operation. Systems like Cassandra and DynamoDB let you choose W and R, so you can pick strong consistency where it matters and fast/eventual where it does not.

Picking W and R for Your Workload

The W+R>N rule is the floor for strong consistency, but how you split it tunes performance. With N=3:

  • Write-heavy, want fast writes: use W=1, R=3. Writes confirm instantly (one replica), but reads must check all 3 to be sure they catch the latest. (1+3>3 ✓)
  • Read-heavy, want fast reads: use W=3, R=1. Reads are instant (any one replica is current), but writes are slower (must hit all 3). (3+1>3 ✓)
  • Balanced: W=2, R=2 — both moderately fast, tolerates one node down. The common default.
  • Max availability, accept staleness: W=1, R=1 (1+1≤3, so not strongly consistent) — fastest, but reads can be stale.

The lesson: put the cost on the side you do least. Read-heavy systems make writes pay; write-heavy systems make reads pay.

Quorums + Consistent Hashing + Vector Clocks = "Dynamo-style"

This topic is the capstone of Part 8 because the famous Dynamo design (behind DynamoDB and Cassandra) combines them all: consistent hashing (Topic 66) decides which N replicas hold a key, quorums (this topic) decide how many must respond, and vector clocks (Topic 67) detect conflicts when you relax W/R for availability. Seeing how these three fit together is exactly the kind of synthesis senior interviews reward.

Common misconceptions, cleared

"Quorum always means a majority." Often, but not necessarily — W and R are tunable. W+R>N is what guarantees overlap; that can be achieved with non-majority splits like W=1,R=3.

"W+R>N gives full ACID consistency." It guarantees a read overlaps the latest write, but edge cases (failed writes, concurrent writes) still need conflict handling (vector clocks) — it's strong-ish, not free ACID.

Interview tip

Show you can tune it: "N=3 replicas; for this read-heavy path I'll use W=3, R=1 for fast reads; W+R>N keeps reads current." Then connect the trio — consistent hashing places replicas, quorums decide responses, vector clocks resolve conflicts. That end-to-end picture is the goal.

Key Takeaways

  • A quorum = the minimum replicas that must agree for a read/write.
  • N = replicas, W = write confirmations, R = read responses.
  • W + R > N guarantees you always read the latest write (strong consistency).
  • Tune the split (W=1/R=3 vs W=3/R=1) to make the less-frequent operation pay the cost.
  • With consistent hashing + vector clocks, quorums form the Dynamo-style design (Cassandra, DynamoDB).

#69Authentication vs Authorization

One-line definition

Authentication (AuthN) = proving who you are (login). Authorization (AuthZ) = deciding what you are allowed to do once you are identified. AuthN comes first, then AuthZ.

These two words sound alike and are constantly confused, but they are completely different steps. Getting them straight is fundamental to all security.

Real-life Analogy: Entering an Office

ID badge vs access level

Authentication = showing your ID at the entrance to prove you are really "Priya". Authorization = your badge's access level deciding which rooms you can enter. You might be authenticated (proven to be Priya) but not authorized to enter the server room. First they confirm who you are, then they check what you can access.

Authentication (Who are you?)

The user proves their identity. Common methods:

  • Something you know: password, PIN.
  • Something you have: phone (OTP), security key.
  • Something you are: fingerprint, face.
  • MFA (Multi-Factor): combine two or more of the above for stronger security.

Authorization (What can you do?)

After login, the system checks permissions. Common models:

  • RBAC (Role-Based): permissions based on roles (admin, editor, viewer).
  • ABAC (Attribute-Based): permissions based on attributes (department, location, time).
PointAuthenticationAuthorization
QuestionWho are you?What can you do?
HappensFirstAfter authentication
ExampleLogin with passwordCan you delete this file?

The Most Common Real-World Security Bug: Broken Authorization

Here's a sobering fact: the #1 item on the OWASP Top 10 (the industry's list of worst web vulnerabilities) is Broken Access Control — i.e. authorization done wrong. The classic bug is IDOR (Insecure Direct Object Reference): the API checks that you're logged in (AuthN โœ“) but forgets to check that the data belongs to you (AuthZ โœ—). So GET /orders/123 returns order 123 even if it's someone else's — just change the number and read everyone's orders. The lesson: authentication is not enough; every request must also verify the user is authorized for that specific resource. "Am I logged in?" and "Am I allowed to touch this?" are different checks, and forgetting the second is how real breaches happen.

RBAC vs ABAC, and Where to Enforce

  • RBAC (roles): simple and common — "admins can delete, viewers can only read". Easy to reason about, but role explosion happens when rules get fine-grained.
  • ABAC (attributes): flexible — "managers can approve expenses under โ‚น50k in their own department during business hours". Powerful for complex policies, harder to manage.
  • Enforce on the server, always: never trust the client. Hiding a "Delete" button in the UI is not authorization — the attacker just calls the API directly. The server must check every time.
Common misconceptions, cleared

"If a user is logged in, they're authorized." The dangerous myth behind most breaches — AuthN proves who, AuthZ must still check what they can access, per resource.

"Hiding the button is enough." No — client-side hiding is cosmetic; authorization must be enforced server-side on every request.

Interview tip

When designing any API, say it explicitly: "Each request is authenticated (who) and authorized for the specific resource (can this user access this exact object?), enforced server-side — to avoid IDOR/broken access control." This single sentence shows real security awareness.

Key Takeaways

  • Authentication = proving identity (login). Authorization = checking permissions. AuthN first, then AuthZ.
  • AuthN methods: password, OTP, biometrics, MFA. AuthZ models: RBAC (roles), ABAC (attributes).
  • Being authenticated does not mean you are authorized — check access to the specific resource every time.
  • Broken access control / IDOR is the #1 web vulnerability — always enforce AuthZ server-side.

#70OAuth 2.0 & OpenID Connect

One-line definition

OAuth 2.0 lets you grant an app limited access to your data on another service without sharing your password (this is "authorization"). OpenID Connect (OIDC) builds on OAuth to also handle login / identity (authentication) — this is "Login with Google".

Ever clicked "Sign in with Google" or let an app access your Google Photos? That is OAuth/OIDC. It lets you safely give apps access without handing over your actual password.

Real-life Analogy: A Hotel Key Card / Valet Key

Limited access, no master key

A valet key lets the parking attendant drive your car but not open the glovebox or trunk — limited access without your full key. OAuth gives apps a "valet key" to part of your data, without your password (the master key). You can also revoke it anytime.

The Key Players

  • Resource Owner: you, the user who owns the data.
  • Client: the app that wants access (e.g. a photo-printing app).
  • Authorization Server: issues tokens after you approve (e.g. Google's login).
  • Resource Server: holds your data and accepts the token (e.g. Google Photos).

How the Flow Works (Simplified)

  • 1. The app redirects you to Google to log in and approve specific permissions ("see your photos").
  • 2. You approve. Google gives the app a short-lived access token (not your password).
  • 3. The app uses that token to access only what you allowed — nothing more.
  • 4. You can revoke the token anytime, cutting off access.
You approve
→
Auth server gives token
→
App uses tokenlimited access
The app gets a limited token, never your password.

OAuth vs OpenID Connect

The key distinction

OAuth 2.0 = authorization (granting access to data). OpenID Connect = authentication (proving identity, i.e. login), added on top of OAuth. "Login with Google" uses OIDC; "let this app read my Google Drive" uses OAuth.

The Problem OAuth Actually Solves

Before OAuth, if an app wanted to access your Gmail contacts, you'd literally give it your Google password — meaning it could read your email, change your password, do anything, forever. OAuth was invented to kill that anti-pattern. It introduces delegated, scoped, revocable access: the app gets a token limited to specific permissions ("read contacts only"), never sees your password, and you can revoke it anytime without changing your password. Three properties to remember: scope (limited permissions), delegation (acting on your behalf without being you), and revocability.

Why "Access Token + Refresh Token"?

OAuth issues two tokens for a reason rooted in earlier topics. The access token is short-lived (minutes) so that if it leaks, the damage window is tiny — but it can't be easily revoked (it's just checked, not looked up). The refresh token is long-lived and stored more securely; when the access token expires, the app quietly exchanges the refresh token for a new one. This gives you both scalability (most requests just validate a token) and a revocation point (revoke the refresh token to cut off access) — exactly the trade-off from session management (Topic 48).

Common misconceptions, cleared

"OAuth logs me in." Not by itself — plain OAuth is about access to resources. Login/identity is OpenID Connect (which adds an "ID token" identifying the user). "Sign in with Google" is OIDC, not bare OAuth.

"The app sees my password." The whole point is that it never does — it only gets a scoped token.

"OAuth encrypts the connection." No — OAuth assumes you're already on HTTPS (Topic 73); it handles authorization, not transport security.

Interview tip

If a design needs "login with Google/GitHub" or third-party data access, say: "OAuth 2.0 for delegated, scoped access with access + refresh tokens; OpenID Connect on top if I need identity/login." Calling out scopes and the token pair shows you understand the why, not just the buzzword.

Key Takeaways

  • OAuth 2.0 grants apps limited (scoped), delegated, revocable access without sharing your password.
  • Uses short-lived access tokens + long-lived refresh tokens for safety plus a revocation point.
  • Players: resource owner (you), client (app), authorization server, resource server.
  • OAuth = authorization; OpenID Connect = authentication (login) built on top of OAuth.
  • It does not provide transport security — always run over HTTPS.

#71JWT (JSON Web Tokens)

One-line definition

A JWT is a compact, digitally-signed token that carries user information. After you log in, the server gives you a JWT; you send it with each request to prove who you are — without the server needing to store a session.

JWTs are the most common way to do stateless authentication (recall Topic 47/48). The token itself carries the proof, so any server can verify it without looking anything up.

Real-life Analogy: A Tamper-Proof Wristband

A festival wristband

At a festival, you show ID once and get a sealed wristband. After that, you just flash the wristband to enter areas — staff do not re-check your ID, because the wristband is hard to fake. A JWT is a digital tamper-proof wristband: show it, and any guard (server) can trust it instantly.

The Three Parts of a JWT

A JWT looks like xxxxx.yyyyy.zzzzz — three parts separated by dots:

  • Header: says what type of token and which signing algorithm.
  • Payload: the actual data (claims) — like user id, role, and an expiry time. Note: this is readable, not encrypted — never put secrets here.
  • Signature: a cryptographic signature created with a secret key. It proves the token was not tampered with.

How It Works

  • 1. You log in; the server creates a JWT, signs it with its secret key, and gives it to you.
  • 2. You send the JWT with every request (usually in the Authorization header).
  • 3. The server verifies the signature using its key. If valid and unexpired, it trusts the token — no database lookup needed.
Important cautions

The signature stops tampering, but the payload is not secret (anyone can read it). Also, JWTs are hard to revoke before they expire (since the server stores nothing) — so keep expiry times short and use refresh tokens for longer sessions.

Signed ≠ Encrypted (the #1 JWT misunderstanding)

This trips up almost everyone, so internalise it: a standard JWT is signed, not encrypted. The payload is just Base64-encoded — which is not encryption; anyone can paste the token into a decoder and read every claim. The signature only guarantees the token wasn't altered (integrity), not that it's hidden (confidentiality). So: put a user id and role in a JWT, never a password, credit card, or secret. If you genuinely need the contents hidden, you need JWE (encrypted JWT) — but that's rare.

The Revocation Problem — and Real Fixes

JWT's superpower (statelessness — no server lookup) is also its weakness: if you fire an employee or a token is stolen, you can't instantly invalidate that JWT, because the server isn't tracking it. It stays valid until it expires. Real systems handle this with:

  • Short-lived access tokens (5–15 min) + a refresh token — so the damage window is tiny and you revoke at the refresh step.
  • A token blocklist/denylist in Redis for emergency revocation (reintroduces a small bit of state — a deliberate trade-off).
  • Token versioning — bump a "token version" on the user record so old tokens fail validation.

JWT vs Server-Side Sessions (when to choose which)

Use JWTs when you want statelessness and scale across many services (e.g. microservices, mobile, third-party APIs) and can tolerate the revocation delay. Use server-side sessions (Topic 48) when you need instant revocation and full control (banking, admin panels). It's the same revocation-vs-scalability trade-off seen throughout this part.

Common misconceptions, cleared

"JWTs are encrypted/secure to store secrets in." No — signed, not encrypted; the payload is publicly readable.

"JWTs can be revoked like sessions." Not natively — you need short expiry, a blocklist, or versioning to approximate revocation.

"Use alg: none / trust the header's algorithm." A famous attack — always pin the expected algorithm server-side, or attackers forge tokens with no signature.

Interview tip

Say: "JWTs for stateless auth across services, signed (not encrypted) so no secrets in the payload, short-lived access tokens + refresh tokens, and a Redis blocklist for emergency revocation." Mentioning the signed-vs-encrypted distinction and the revocation strategy is exactly what security-minded interviewers want.

Key Takeaways

  • A JWT is a signed token carrying user info, enabling stateless authentication. Parts: Header.Payload.Signature.
  • Signed ≠ encrypted — the payload is readable (Base64), so never store secrets in it.
  • Hard to revoke early — use short expiry + refresh tokens, a Redis blocklist, or token versioning.
  • Choose JWTs for scale/statelessness; server-side sessions for instant revocation.
  • Always pin the signing algorithm server-side (beware the alg: none attack).

#72API Security

One-line definition

API security is the set of practices that protect your APIs from misuse and attacks — making sure only the right people access them, the right amount, over secure channels, with valid data.

APIs are the doors into your system. If they are not secured, attackers walk right in. This topic gathers the key defenses, most of which build on earlier topics.

The Core Practices

  • Authentication & Authorization: verify identity (JWT/OAuth) and check permissions on every request (Topics 69–71).
  • HTTPS everywhere: always encrypt traffic with TLS so data cannot be read or altered in transit (Topic 6, 73).
  • Rate limiting & throttling: cap requests to prevent abuse and DoS (Topic 49).
  • Input validation: never trust input — validate and sanitize everything to block injection attacks (Topic 75).
  • API keys / tokens: identify and control which apps can call your API.
  • Least privilege: give each client only the minimum access it needs.
  • Use an API Gateway: centralize auth, rate limiting, and logging (Topic 50).

Idempotency (a key safety idea)

Make repeats safe

Design important operations (especially payments) to be idempotent — doing the same request twice has the same effect as doing it once. Clients send an "idempotency key"; if the server sees a repeat, it ignores the duplicate. This prevents double-charges from retries (Topic 57).

More Protections

  • CORS: control which websites are allowed to call your API from browsers.
  • Avoid exposing sensitive data: return only what is needed; never leak internal details in errors.
  • Logging & monitoring: watch for suspicious patterns (Part 11).

The Golden Rule: Never Trust the Client

Almost every API vulnerability traces back to one mistake: trusting input or checks that come from the client. The browser, mobile app, and any "validation" they do can be bypassed entirely — an attacker just talks to your API directly with custom tools. So the server must re-do everything: re-validate all input, re-check authorization for the specific resource, re-enforce limits and prices. Treat every incoming request as potentially hostile, even from "your own" app. Client-side validation is for user convenience; server-side validation is for security.

Defense in Depth: Layers, Not a Wall

Good API security is many overlapping layers so that if one fails, others still protect you: TLS (transport) + gateway auth + per-request authorization + input validation + rate limiting + monitoring. This builds directly on earlier topics — the API gateway (Topic 50) is where you centralise auth, rate limiting (Topic 49), and TLS termination so individual services don't each reinvent them. Also apply least privilege everywhere: each token, service, and DB account gets only the minimum access it needs, so a single compromise has a small blast radius.

Common misconceptions, cleared

"My mobile app validates input, so the API is safe." No — attackers skip your app and call the API directly. Validate on the server.

"HTTPS alone makes my API secure." HTTPS only protects data in transit; you still need auth, authorization, validation, and rate limiting.

"Obscure endpoints won't be found." Security by obscurity fails — scanners and leaked docs find everything. Every endpoint needs real protection.

Interview tip

Summarise API security as layers: "TLS everywhere, authenticate + authorize each request at the gateway, validate all input server-side, rate limit per client, make writes idempotent, and apply least privilege." Leading with "never trust the client" frames it perfectly.

Key Takeaways

  • API security = authenticate + authorize every request, always use HTTPS, and rate limit.
  • Never trust the client — re-validate input and re-check authorization on the server every time.
  • Use defense in depth (overlapping layers) and least privilege to shrink blast radius.
  • Make critical operations idempotent so retries cannot cause double effects.
  • An API gateway helps centralize these protections.

#73Encryption (TLS/SSL)

One-line definition

Encryption scrambles data so only someone with the right key can read it. TLS/SSL is the technology that encrypts data in transit over the internet (the "S" in HTTPS).

Without encryption, anyone who intercepts your data (on public WiFi, etc.) can read it. Encryption turns readable data into scrambled nonsense for everyone except the intended recipient.

Two Types of Encryption

  • Symmetric encryption: the same key locks and unlocks the data. Very fast, but both sides must somehow share the key secretly.
  • Asymmetric encryption: uses a key pair — a public key (shared openly, locks data) and a private key (kept secret, unlocks it). Slower, but solves the key-sharing problem.
Analogy: a mailbox

Asymmetric: anyone can drop a letter through your mailbox slot (public key), but only you have the key to open it and read the letters (private key). This is how strangers can send you secret data safely without pre-sharing a key.

How TLS Works (the clever combo)

TLS uses both: it uses slow asymmetric encryption just once at the start (the "handshake") to safely agree on a shared secret key, then switches to fast symmetric encryption for the actual data. Best of both worlds — secure key exchange plus speed.

Handshakeasymmetric, share a secret key
→
Data transferfast symmetric encryption
TLS uses asymmetric to exchange a key, then symmetric for speed.

Encryption in Transit vs at Rest

  • In transit (TLS): protects data while it travels over the network.
  • At rest: protects stored data (databases, disks) so a stolen disk is useless without the key.

The Other Half: Certificates and Trust

Encryption alone isn't enough — you also need to know you're talking to the real server, not an impostor (a "man-in-the-middle"). That's what certificates and Certificate Authorities (CAs) provide. During the TLS handshake, the server presents a certificate (its public key, signed by a trusted CA like Let's Encrypt). Your browser trusts a set of CAs, so it can verify the certificate is genuine and the public key really belongs to that domain. This is why a self-signed or expired certificate triggers scary browser warnings — encryption would still work, but identity can't be trusted. So HTTPS gives you three things at once: confidentiality (encryption), integrity (tamper detection), and authenticity (you're talking to the real site).

Hashing Is Not Encryption

A crucial distinction for passwords: encryption is reversible (with the key you get the data back); hashing is one-way (you can't reverse it). You never encrypt passwords — you hash them (with a slow, salted algorithm like bcrypt/argon2), so even if the database is stolen, the original passwords can't be recovered. Encrypt data you need to read back (card numbers, messages); hash data you only need to verify (passwords).

Common misconceptions, cleared

"Encryption proves who I'm talking to." No — encryption hides data; certificates/CAs prove identity. You need both (that's full HTTPS).

"Passwords should be encrypted in the DB." No — they should be hashed and salted (one-way), never reversibly encrypted.

"Asymmetric is just better than symmetric." They're different tools — asymmetric solves key exchange/identity but is slow; symmetric is fast for bulk data. TLS uses each where it shines.

Interview tip

Say "TLS for data in transit, encryption at rest for stored data, certificates/CAs for server identity, and hashing (bcrypt/argon2) — not encryption — for passwords." Distinguishing encrypt-vs-hash and naming the CA/identity role demonstrates solid security fundamentals.

Key Takeaways

  • Encryption scrambles data so only key-holders can read it.
  • Symmetric = one shared key (fast); Asymmetric = public/private key pair (solves key sharing & identity).
  • TLS combines both: asymmetric handshake to share a key, then fast symmetric transfer; certificates/CAs prove identity.
  • Encrypt data both in transit (TLS) and at rest.
  • Hash passwords (one-way, salted) — never reversibly encrypt them.

#74DDoS Protection

One-line definition

A DDoS (Distributed Denial of Service) attack floods your system with massive fake traffic from many sources at once, trying to overwhelm it so real users cannot get in. DDoS protection is how you absorb or block that flood.

The goal of a DDoS attack is not to steal data but to make your service unavailable by exhausting its resources. "Distributed" means it comes from thousands of machines (often a botnet) at once.

Real-life Analogy: A Fake Crowd Blocking a Shop

A mob that buys nothing

Imagine thousands of fake "customers" crowding into a shop's doorway with no intention of buying — real customers cannot even get in. That is a DDoS: fake traffic packing your system so genuine users are locked out.

How to Defend Against DDoS

  • CDN & scale: a large CDN absorbs huge traffic across its global network, soaking up the flood before it reaches you (Topic 18).
  • Rate limiting: cap requests per IP/user to throttle abusive sources (Topic 49).
  • Web Application Firewall (WAF): filters out malicious traffic patterns before they hit your servers.
  • Traffic filtering / scrubbing: specialized services (Cloudflare, AWS Shield) detect attack traffic and drop it, passing only clean traffic.
  • Auto-scaling: temporarily add capacity to absorb spikes (though attackers may scale too).
  • Blackholing / IP blocking: drop traffic from known-bad sources.
Attack flood ๐ŸŒŠ
→
CDN + WAF + scrubbingabsorb & filter
clean →
Your servers
Protection layers absorb and filter attack traffic so only clean traffic reaches you.
Key idea

You cannot always stop a DDoS at the source — the defense is having more capacity and smart filtering than the attacker can overwhelm. That is why companies rely on huge providers (Cloudflare, AWS) whose networks dwarf most attacks.

Three Layers of Attack (and why the layer matters)

"DDoS" isn't one thing — attacks target different layers, and each needs a different defense:

  • Volumetric (network layer): raw flood of traffic (e.g. UDP/amplification) to saturate your bandwidth. Defense: huge-capacity CDN/scrubbing networks that simply have more pipe than the attacker.
  • Protocol attacks: exploit how connections work (e.g. a SYN flood opening half-connections to exhaust your server's connection table). Defense: SYN cookies, firewalls, load balancers that absorb this.
  • Application layer (Layer 7): the sneakiest — looks like real requests (e.g. hammering an expensive search endpoint). Low traffic volume but high server cost, so it slips past volume-based defenses. Defense: WAF, rate limiting, bot detection, CAPTCHAs.

The insight: a giant CDN stops the volumetric flood easily, but L7 attacks need smart filtering because each request looks legitimate.

Why "Distributed" Is the Whole Problem

A plain DoS comes from one source — you just block that IP. A DDoS comes from thousands of hijacked machines (a botnet) across the globe, so there's no single IP to block, and the traffic can be indistinguishable from a real traffic spike. That's exactly why the realistic defense is "out-scale and filter at the edge" rather than "find and block the attacker" — you usually can't pinpoint them.

Common misconceptions, cleared

"Auto-scaling solves DDoS." Partly — it can absorb spikes, but it also means you pay for the attacker's fake traffic (a "denial of wallet"), and attackers can scale too. Filtering bad traffic is essential, not just scaling.

"A DDoS steals data." No — its goal is unavailability, not theft (though it's sometimes a smokescreen for another attack).

"Blocking IPs stops it." Rarely — thousands of distributed, rotating IPs make manual blocking futile.

Interview tip

Mention DDoS defense for any public-facing system: "front everything with a CDN/WAF (Cloudflare/AWS Shield) to absorb volumetric attacks, plus rate limiting and bot detection for application-layer attacks." Distinguishing volumetric vs L7 attacks shows you understand it's not one problem.

Key Takeaways

  • A DDoS floods you with fake traffic from a distributed botnet to make you unavailable (not to steal data).
  • Attacks hit different layers: volumetric, protocol, and sneaky application-layer (L7) — each needs different defenses.
  • Defenses: CDN absorption, rate limiting, WAF, traffic scrubbing, bot detection, auto-scaling.
  • Core strategy: more capacity + smart filtering at the edge — you usually can't just block the source.
  • Providers like Cloudflare and AWS Shield specialize in this.

#75SQL Injection, XSS & CSRF

One-line definition

These are three of the most common web attacks. SQL Injection tricks your database. XSS injects malicious scripts into pages. CSRF tricks a logged-in user's browser into making unwanted requests. Knowing them — and their fixes — is essential.

1. SQL Injection (SQLi)

The attacker enters specially crafted text into an input (like a login box) that gets injected into your database query, letting them read, change, or delete data.

Example

If your query is built by gluing user input directly, an attacker typing ' OR '1'='1 can turn a login check into one that is always true — logging in without a password.

Fix: use parameterized queries / prepared statements (never glue user input into SQL), and validate input. This keeps data separate from the query command.

2. Cross-Site Scripting (XSS)

The attacker injects malicious JavaScript into a page (e.g. through a comment field). When other users view the page, the script runs in their browsers — stealing cookies, sessions, or data.

Fix: escape/sanitize all user-generated content before displaying it, use a Content Security Policy (CSP), and mark cookies HttpOnly so scripts cannot read them.

3. Cross-Site Request Forgery (CSRF)

The attacker tricks a logged-in user's browser into silently sending a request to a site they are authenticated on (e.g. a hidden form that transfers money), abusing the user's existing session.

Fix: use CSRF tokens (a secret value the server checks on each sensitive request), SameSite cookies, and verify the request origin.

AttackWhat it doesMain fix
SQL InjectionManipulates DB queriesParameterized queries
XSSRuns malicious scripts in users' browsersEscape/sanitize output, CSP
CSRFForces unwanted actions using a user's sessionCSRF tokens, SameSite cookies
The golden rule

Never trust user input. Validate it, sanitize it, and keep data separate from commands. Most web attacks come down to input that was trusted when it should not have been.

The One Idea Behind SQLi and XSS: Data Treated as Code

SQL Injection and XSS look different but share a single root cause: untrusted data getting interpreted as code/commands. In SQLi, your input is treated as part of the SQL command. In XSS, your input is treated as JavaScript in the page. The fix in both cases is the same principle: keep data and code strictly separate. Parameterized queries send the SQL command and the data on separate channels, so input can never become command. Output escaping turns <script> into harmless text so it can never become executable. Understand this one idea and both attacks (and many others) make sense.

How CSRF Differs (it abuses trust, not input)

CSRF is the odd one out — it doesn't inject anything into your site. Instead it exploits the fact that browsers automatically attach your cookies to requests to a site. So a malicious page can quietly make your browser fire a real, authenticated request to your bank ("transfer money"), and the bank can't tell it wasn't you. The fix works by requiring a secret the attacker can't know or send: a CSRF token (a random value embedded in your real forms) or SameSite cookies (which tell the browser not to send cookies on cross-site requests). Note: XSS can defeat CSRF protections, which is why fixing XSS is foundational.

Common misconceptions, cleared

"Hashing/escaping in the frontend is enough." No — SQLi must be prevented at the server/DB layer (parameterized queries); client checks are bypassable.

"An ORM/framework makes me immune." Mostly safe by default, but raw queries, string concatenation, or innerHTML/dangerous rendering still open holes.

"XSS and CSRF are the same." No — XSS injects code into your page; CSRF rides your existing session from another site. Different attacks, different fixes.

Interview tip

Tie them to the root cause: "SQLi and XSS both come from mixing data with code — fix with parameterized queries and output escaping/CSP; CSRF abuses auto-sent cookies — fix with CSRF tokens / SameSite." Referencing the OWASP Top 10 and the data-vs-code principle signals genuine security literacy.

Key Takeaways

  • SQL Injection: malicious input manipulates DB queries — fix with parameterized queries.
  • XSS: injected scripts run in users' browsers — fix by escaping output, CSP, and HttpOnly cookies.
  • SQLi & XSS share one root cause: data treated as code — keep data and commands strictly separate.
  • CSRF: abuses a logged-in session via auto-sent cookies — fix with CSRF tokens and SameSite cookies.
  • The overarching rule: never trust user input (these top the OWASP list).

#76Block vs Object vs File Storage

One-line definition

There are three fundamental ways to store data: Block (raw chunks, like a hard drive), File (folders and files, like your computer), and Object (data + metadata with a unique id, like cloud storage S3). Each suits different needs.

"Storage" is not one thing. Choosing the right type matters for performance, scale, and cost. Let us understand all three simply.

1. Block Storage

Data is split into fixed-size blocks, each with an address, with no built-in structure on top. It is the lowest-level, fastest storage — like a raw hard drive or SSD. Software (like a database or OS) builds structure on top of it.

  • Best for: databases, virtual machine disks — anything needing fast, low-level access.
  • Example: AWS EBS.

2. File Storage

Data is organized as files inside folders (a hierarchy), exactly like your laptop's file system. Easy for humans and many apps to use, shareable across machines.

  • Best for: shared documents, traditional apps that expect a file system.
  • Example: network drives (NFS), AWS EFS.

3. Object Storage

Data is stored as objects — each object is the data plus metadata plus a unique id, all in a flat space (no folders). It scales almost infinitely and is cheap, accessed via APIs (HTTP).

  • Best for: images, videos, backups, huge amounts of unstructured data.
  • Example: AWS S3, Google Cloud Storage.
TypeStructureBest forExample
BlockRaw blocksDatabases, VM disksAWS EBS
FileFolders & filesShared filesNFS, AWS EFS
ObjectObjects + metadata (flat)Media, backups, big dataAWS S3
Analogy

Block = raw bricks you build anything from. File = a filing cabinet with folders. Object = a valet who takes your item, gives you a ticket (id), and stores it somewhere in a giant warehouse — you just present the ticket to get it back.

The Real Decision Driver: How You Modify Data

The cleanest way to choose is by how you change data. Block lets you modify any small piece in place (overwrite byte 500 without touching the rest) — essential for a database constantly updating rows, which is why databases sit on block storage. Object storage is the opposite: you generally replace the whole object at once (you don't edit byte 500 of a photo on S3 — you re-upload it). That immutability is precisely what lets object storage scale to trillions of items and stay cheap. File sits in between — a familiar folder hierarchy with shared read/write, but it doesn't scale as limitlessly as object storage. So: frequent in-place edits → block; write-once-read-many media → object; shared documents → file.

Why "Folders" in S3 Are an Illusion

A subtle but important point: object storage has a flat namespace — there are no real folders. A key like photos/2024/beach.jpg looks like a path, but the / is just part of the name; the console fakes a folder view for your comfort. This flatness is a feature: with no directory tree to traverse or lock, the system scales horizontally without limit. Real file systems must maintain that tree, which is part of why they don't scale as far.

Common misconceptions, cleared

"Object storage is just a file system in the cloud." No — it's flat (no real folders), accessed via HTTP API (not mounted as a drive normally), and objects are typically replaced whole, not edited in place.

"Object storage is slow, so it's worse." It has higher per-request latency than block, but near-infinite scale, durability, and low cost — different tool, not worse. You wouldn't run a database on it, but it's perfect for media.

Interview tip

This appears constantly in design questions involving uploads. Say: "user-uploaded images/videos go to object storage (S3); the database stores only the metadata and the object's URL/key." Choosing object storage for blobs and keeping the DB lean is the expected, correct instinct.

Key Takeaways

  • Block = raw, fast, in-place-editable chunks — for databases and VM disks.
  • File = folders and files (a hierarchy) — for shared documents.
  • Object = data + metadata + id in a flat space, replaced whole — for media, backups, huge unstructured data (S3).
  • Choose by how you modify data: in-place edits → block; write-once media → object; shared docs → file.
  • Store blobs in object storage; keep only their metadata/URL in the database.

#77Object Storage Internals (S3)

One-line definition

Object storage (like Amazon S3) stores files as objects in buckets, identified by a unique key. It is built to be virtually unlimited in scale, highly durable, and cheap — the default home for media and backups on the web.

We met object storage in Topic 76. Now let us peek inside how something like S3 achieves its famous scale and durability.

The Building Blocks

  • Bucket: a top-level container for your objects (like a root folder).
  • Object: the actual data (a file) plus metadata (size, type, custom tags).
  • Key: the unique name used to retrieve the object (e.g. photos/2024/beach.jpg). It looks like a path but the storage is actually flat.

How It Achieves Durability & Scale

  • Replication: every object is stored as multiple copies across different machines and data centers, so hardware failures never lose data. (S3 advertises "11 nines" of durability.)
  • Flat namespace: no real folder tree to maintain, so it scales to trillions of objects easily.
  • Distributed across many servers: load and data spread out (often using consistent hashing, Topic 66).
  • Accessed via HTTP API: simple GET/PUT requests — works from anywhere.

Useful Features

  • Versioning: keep old versions of an object so you can recover from mistakes.
  • Storage classes / lifecycle: move rarely-used data to cheaper, colder storage automatically (e.g. S3 Glacier for archives).
  • Access control & encryption: fine-grained permissions and encryption at rest.
  • Static website & CDN integration: often paired with a CDN to serve files fast worldwide.
Where it is used

Netflix stores video files in object storage; Instagram and most apps store user photos there; backups, logs, and data lakes live there too. Whenever you need to store lots of files cheaply and durably, object storage is the answer.

The Killer Pattern: Pre-Signed URLs

A pattern you should know cold, because it appears in nearly every "design a system with uploads" question. When a user uploads a photo, you do not route the file through your servers (that wastes your bandwidth and CPU). Instead, your server generates a pre-signed URL — a temporary, permission-limited link — and the client uploads the file directly to S3 using it. Downloads work the same way. The benefits: your app servers stay lean (they never touch the bytes), it scales effortlessly, and access is still controlled (the URL expires and is scoped). Pair this with a CDN in front of S3 for fast global delivery.

What "11 Nines" of Durability Actually Means

S3's famous "eleven 9s" (99.999999999%) durability means that if you store 10 million objects, you'd expect to lose one roughly every 10,000 years. It achieves this by automatically storing each object redundantly across multiple devices in multiple data centers (availability zones) and constantly verifying integrity. Durability (will my data survive?) is different from availability (can I reach it right now?) — S3's durability is higher than its availability. This is why object storage is the go-to for backups and irreplaceable data.

Common misconceptions, cleared

"Upload files through my server to S3." Wasteful at scale — use pre-signed URLs so clients talk to S3 directly; your server just authorizes.

"Object storage is strongly consistent / good for live editing." It's optimized for whole-object replace and read-many; it's not a database. Don't use it for data that's constantly mutated in place.

"Durable means always available." No — durability is "won't be lost"; availability is "reachable now". They're separate guarantees.

Interview tip

For any upload/media feature, draw it as: "client gets a pre-signed URL → uploads directly to S3 → we store the key + metadata in the DB → serve via CDN." Mentioning pre-signed URLs and lifecycle policies (move cold data to Glacier) shows you know how object storage is used in practice, not just in theory.

Key Takeaways

  • Object storage uses buckets, objects (data + metadata), and unique keys.
  • Huge durability ("11 nines") via cross-data-center replication; scales via a flat namespace.
  • Use pre-signed URLs so clients upload/download directly — your servers never handle the bytes.
  • Supports versioning, lifecycle/cold storage (Glacier), and encryption; pair with a CDN for delivery.
  • Durability ≠ availability. The standard home for media, backups, and big unstructured data.

#78Distributed File Systems (HDFS)

One-line definition

A Distributed File System stores huge files by splitting them into chunks spread across many machines, while still letting you treat it like one big file system. HDFS (Hadoop Distributed File System) is the classic example, built for big data.

When a file (or dataset) is too big for any single machine — think petabytes of logs — you need to spread it across a cluster. A distributed file system makes that cluster look and behave like one giant disk.

How HDFS Works

  • Files are split into large blocks (e.g. 128 MB each), which are distributed across many machines.
  • Each block is replicated (typically 3 copies) on different machines, so a machine failure never loses data.
  • NameNode (the master): keeps the "map" of which blocks live where (metadata).
  • DataNodes (the workers): actually store the blocks and serve them.
NameNodeknows where every block is
↓
DataNode 1blocks + copies
DataNode 2
DataNode 3
The NameNode tracks metadata; DataNodes store replicated blocks.

Key Design Ideas

  • Built for huge files and throughput, not tiny files or low latency — great for batch processing (e.g. analytics).
  • Move computation to the data: instead of moving giant files to where the code is, run the code on the machine that holds the data (this is how Hadoop/MapReduce works).
  • Fault tolerant by replication — lose a DataNode, the copies elsewhere keep data safe.
Where it fits

Distributed file systems power big data analytics — processing enormous datasets across clusters. HDFS underpins the Hadoop ecosystem; similar ideas appear in Google's GFS and many data platforms.

Why "Move Compute to Data" Is the Whole Point

This is the single most important idea here. Imagine a 10 TB dataset spread across 100 machines, and you want to count something in it. The naive approach — copy all 10 TB over the network to one powerful machine — would take forever, because the network is the bottleneck, far slower than local disk. HDFS flips this: it sends the small piece of code to the 100 machines, each processes its own local block in parallel, and only the tiny results travel back. Moving a few KB of code is trillions of times cheaper than moving terabytes of data. This "data locality" principle is what makes big-data processing (MapReduce, Spark) feasible at all.

Why Big Blocks, and the NameNode Weakness

HDFS uses huge blocks (128 MB+) deliberately: it's built for a few enormous files scanned sequentially, so large blocks mean less metadata to track and efficient streaming reads. The flip side — and a classic exam point — is the "small files problem": millions of tiny files overwhelm the NameNode, because it must hold all block metadata in memory. That also makes the NameNode a single point of failure / bottleneck; production clusters run a standby NameNode (with consensus, Topic 64) for failover. So HDFS is brilliant for "few giant files, batch throughput" and poor for "many tiny files, low latency".

Common misconceptions, cleared

"HDFS is good for lots of small files." The opposite — small files crush the NameNode's memory. It's built for few huge files.

"It's for low-latency, real-time reads." No — it's optimized for high throughput batch processing, not fast individual lookups. Use a database/object store for that.

"HDFS and object storage (S3) are the same." Related but different — HDFS co-locates compute with data for analytics; modern clouds increasingly run compute against data in object storage instead.

Interview tip

Bring up data locality for analytics designs: "for petabyte-scale batch processing I'd use a distributed file system (HDFS) or object storage with a compute engine like Spark, moving compute to the data rather than data to compute." Mentioning the small-files problem and the NameNode SPOF shows real depth.

Key Takeaways

  • A distributed file system splits huge files into replicated blocks across many machines.
  • HDFS: a NameNode tracks block metadata (in memory); DataNodes store them (usually 3 copies).
  • Core idea: move compute to the data — the network, not the disk, is the bottleneck.
  • Built for few huge files + batch throughput; weak at many small files / low latency (NameNode limits).
  • The backbone of big data processing (Hadoop/Spark).

#79Time-Series Databases

One-line definition

A Time-Series Database (TSDB) is specially built to store and query data stamped with time — measurements that arrive continuously, like CPU usage every second, stock prices, or IoT sensor readings.

Lots of data is naturally about "what happened at this time": metrics, sensor readings, prices. General databases handle this poorly at scale, so TSDBs are optimized just for it.

What Makes Time-Series Data Special

  • Always time-stamped and usually arrives in time order.
  • Write-heavy: huge volumes of new points constantly flowing in.
  • Append-only: you rarely update old points — you just add new ones.
  • Queried by time ranges: "average CPU over the last hour", "price trend this week".

How TSDBs Optimize for This

  • Efficient time-based storage: data is organized by time for fast range queries.
  • Heavy compression: consecutive readings are similar, so they compress extremely well — saving huge space.
  • Downsampling & retention: keep recent data at full detail, automatically summarize and expire old data (e.g. keep per-second data for a day, per-hour for a year).
  • Fast aggregations: built-in functions for averages, min/max, percentiles over time windows.
Examples & uses

Tools: InfluxDB, Prometheus, TimescaleDB. Uses: server monitoring dashboards (Topic 82), IoT sensor data, financial tick data, application metrics. If your main question is "how did this number change over time?", a TSDB is the right tool.

Why a Normal Database Struggles Here

Picture 10,000 servers each writing a CPU reading every second — that's 10,000 inserts/second, forever, growing into billions of rows. A regular relational DB chokes on this: its B-tree indexes (Topic 23) are designed for random reads/writes and constant updates, but they get expensive to maintain under this relentless append load, and storing each tiny reading as a full row wastes enormous space. A TSDB is purpose-built for the opposite profile: append-only, time-ordered, write-heavy, rarely updated. It can therefore make assumptions a general DB can't.

The Two Tricks That Make TSDBs Special

  • Massive compression via "delta-of-delta": consecutive readings barely change (CPU goes 50%, 51%, 51%, 52%…) and timestamps tick at regular intervals. So instead of storing each full value, TSDBs store the differences — often shrinking data by 10× or more. This is the headline optimization.
  • Downsampling + retention: you don't need per-second detail from two years ago. TSDBs automatically roll up old data (per-second → per-minute → per-hour averages) and expire the rest, keeping storage bounded forever. A general DB would just grow without limit.
Common misconceptions, cleared

"Just add a timestamp column to MySQL." Works at small scale, but falls over at high ingest volume and wastes huge space without time-series compression and retention.

"TSDBs are for updating records." No — they assume append-only; updating old points is awkward and discouraged. They're write-once-by-time.

"They're general-purpose databases." They're specialized — great for time-range aggregates, poor for arbitrary relational joins/transactions.

Interview tip

When a design involves metrics, sensors, or any "value over time", say: "this is append-only time-series data, so I'd use a TSDB (Prometheus/InfluxDB) with compression, downsampling, and retention rather than overloading the main relational DB." Naming delta compression and downsampling is the differentiator.

Key Takeaways

  • A TSDB is optimized for time-stamped, write-heavy, append-only data queried by time ranges.
  • A normal DB struggles because B-tree indexes and full rows are wrong for relentless append load.
  • Two key tricks: delta compression (10×+ smaller) and downsampling + retention (bounded storage).
  • Great for metrics, monitoring, IoT, and financial data; poor for joins/transactions.
  • Tools: InfluxDB, Prometheus, TimescaleDB.

#80Search Engines (Elasticsearch)

One-line definition

A search engine (like Elasticsearch) is a system specialized for fast full-text search — finding matching documents among millions instantly, ranking them by relevance — something normal databases are slow at.

Searching free text ("find all reviews mentioning 'battery life'") across millions of records is painfully slow for a regular database. Search engines solve this with a clever data structure called an inverted index.

The Secret: The Inverted Index

Like a book index, but for every word

A normal index maps document → its words. An inverted index flips it: it maps each word → the list of documents that contain it. So to find "battery", it instantly returns every document with that word — no scanning. This is exactly how a book's index lets you jump to every page mentioning a topic.

What Search Engines Add Beyond "Find"

  • Relevance ranking: results are scored so the best matches appear first (not just any match).
  • Tokenization & analysis: text is broken into words, lowercased, stemmed ("running" → "run"), so searches are flexible.
  • Fuzzy & typo-tolerant search: finds results even with small spelling mistakes.
  • Filters & facets: narrow results (by price, category, date) like on shopping sites.
  • Autocomplete & suggestions.

How It Scales

Elasticsearch is distributed: the index is split into shards across many nodes (Topic 27) and replicated for reliability (Topic 26). This lets it search huge datasets fast and stay available.

Common pattern

Apps often keep their main data in a normal database and also index searchable fields into Elasticsearch. The database is the source of truth; the search engine powers fast search. Used by e-commerce search, log search (the "E" in the ELK stack, Topic 81), and site search everywhere.

Why a Database LIKE '%word%' Doesn't Cut It

You might think "I'll just search with SQL LIKE '%battery%'." The problem: a leading-wildcard LIKE can't use the B-tree index (Topic 23), so it does a full table scan — checking every single row. On millions of rows that's seconds per query, and it can't rank by relevance, handle typos, or understand that "running" should match "run". The inverted index makes the lookup near-instant and enables ranking, because for each word it has already precomputed exactly which documents contain it (and how often). It's the difference between scanning a whole book versus flipping to its index.

The Trade-off: Search Engines Are Not the Source of Truth

Crucial architectural point: you almost never use Elasticsearch as your primary database. It's near-real-time (newly indexed data takes a moment to appear), it's not built for strict transactions/consistency, and rebuilding an index is normal. So the standard pattern is dual storage: the relational DB (or similar) is the source of truth, and you sync searchable fields into Elasticsearch (via events/CDC, Topic 40). This is a deliberate denormalization (Topic 25) — you accept eventual consistency between the two stores in exchange for blazing search. Keeping them in sync (and handling lag) is the real engineering work.

Common misconceptions, cleared

"Just use Elasticsearch as my main database." Risky — it's optimized for search, not as a transactional system of record; use it alongside a primary DB.

"Search results are instantly up to date." It's near-real-time; there's a small indexing delay, and the search store lags the source of truth slightly.

"An inverted index is just a normal database index." It's the inverse mapping (word → documents) plus text analysis (stemming, tokenization) and relevance scoring — far more than a B-tree.

Interview tip

For any "design search" feature, say: "I'll keep the system of record in the primary DB and index searchable fields into Elasticsearch (kept in sync via events), using its inverted index for fast, ranked, typo-tolerant search." Calling out the dual-store sync and eventual consistency is exactly what's expected.

Key Takeaways

  • Search engines do fast full-text search with relevance ranking — far beyond a normal DB's slow LIKE scan.
  • The core trick is the inverted index (word → documents) plus text analysis and scoring.
  • They add ranking, typo tolerance, filters, and autocomplete.
  • Use it alongside a primary DB (the source of truth), kept in sync — accept near-real-time/eventual consistency.
  • Elasticsearch scales via sharding + replication.

#81Logging (ELK Stack)

One-line definition

Logging means recording events that happen in your system ("user logged in", "payment failed") so you can understand what occurred, especially when debugging problems. The ELK Stack is the popular toolset to collect, store, and search logs at scale.

When something goes wrong in production, logs are your detective's notebook. But with hundreds of servers, you cannot read log files one by one — you need to centralize and search them. That is what logging systems do.

Real-life Analogy: A Diary of Everything

A record you can look back on

Logs are like a detailed diary your system writes constantly. When a problem happens at 3 AM, you flip back through the diary to see exactly what led up to it. Without it, you are guessing in the dark.

The ELK Stack (three tools working together)

  • E — Elasticsearch: stores and searches the logs fast (the search engine from Topic 80).
  • L — Logstash (or Beats): collects logs from all servers, processes them, and ships them to Elasticsearch.
  • K — Kibana: the dashboard to visualize and explore logs with charts and searches.
Servers' logs
→
Logstashcollect
→
Elasticsearchstore & search
→
Kibanavisualize
Logs flow from all servers into one searchable, visual place.

Good Logging Practices

  • Use log levels: DEBUG, INFO, WARN, ERROR — so you can filter by severity.
  • Structured logs (JSON): machine-readable logs are far easier to search and analyze.
  • Include context: request id, user id, timestamp — so you can trace one request's journey.
  • Never log secrets (passwords, tokens, card numbers).
  • Centralize: ship all logs to one place, not scattered on servers.

Why Structured Logs Change Everything

The single biggest upgrade you can make is moving from plain-text logs to structured (JSON) logs. A line like "User 5 failed login from 1.2.3.4" reads fine to a human but is a nightmare to query across millions of lines. Logged as {"event":"login_failed","user_id":5,"ip":"1.2.3.4"}, you can instantly run "show all login_failed events for user 5 in the last hour" or chart failures by IP. Structure turns your logs from a wall of text into a queryable database — which is exactly what makes the ELK stack powerful.

The Correlation ID: Tracing One Request Across Many Services

In microservices, a single user click might pass through 8 services, each logging on its own. When something fails, how do you find all the logs for that one request? The answer is a correlation ID (trace ID): generate a unique id at the entry point (the API gateway) and pass it through every downstream call, attaching it to every log line. Now you filter by that one id and see the request's entire journey across all services in order. This is the bridge to distributed tracing (Topic 83) and is indispensable in any distributed system.

Logs Aren't Free: Sampling and Cost

Logging everything at full volume gets expensive fast — storage, indexing, and ingestion costs balloon, and noisy DEBUG logs can even slow the app. Real systems manage this: log at INFO in production (DEBUG only when investigating), sample very high-volume events (keep 1 in N), and set retention (keep detailed logs days/weeks, not forever). The goal is enough signal to debug without drowning in (or paying for) noise.

Common misconceptions, cleared

"Log everything, just in case." Expensive and counterproductive — noise hides signal. Use levels, sampling, and retention deliberately.

"Logs replace metrics." No (Topic 82) — logs are detailed individual events (great for "why did this fail?"); metrics are aggregated numbers (great for "how's overall health?"). You need both.

"Plain text is fine." It severely limits querying — structured JSON is the standard for searchable logs.

Interview tip

For any distributed design, mention: "structured (JSON) logs with a correlation/trace ID propagated across services, centralized in something like ELK, with levels, sampling, and retention to control cost." The correlation ID detail signals you've actually debugged microservices.

Key Takeaways

  • Logging records events so you can debug and understand what happened.
  • The ELK stack = Elasticsearch (search), Logstash (collect), Kibana (visualize).
  • Structured JSON logs + a correlation/trace ID let you query and follow one request across all services.
  • Use log levels, sampling, and retention to control cost; never log secrets.
  • Centralize logs from all servers into one searchable place.

#82Metrics (Prometheus, Grafana)

One-line definition

Metrics are numerical measurements of your system over time — CPU usage, request rate, error rate, response time. Prometheus collects them and Grafana displays them on beautiful dashboards.

Logs tell you about individual events; metrics tell you about the overall health and trends as numbers. Together they give a full picture. Metrics answer "how is the system doing right now, and how is it trending?".

Logs vs Metrics

Different questions

Logs = "what exactly happened in this one request?" (detailed, text). Metrics = "what is the average response time and error rate over the last hour?" (aggregated numbers, great for trends and alerts). You need both.

The "Golden Signals" to Monitor

  • Latency: how long requests take.
  • Traffic: how many requests per second.
  • Errors: rate of failed requests.
  • Saturation: how "full" your resources are (CPU, memory, disk).

The Tools

  • Prometheus: a time-series database (Topic 79) that regularly "scrapes" metrics from your services and stores them.
  • Grafana: connects to Prometheus (and others) to build dashboards — graphs of all your metrics, viewable at a glance.
Services expose metrics
→
Prometheusscrape & store
→
Grafanadashboards
Prometheus collects numeric metrics; Grafana turns them into live dashboards.
Why metrics power alerts

Because metrics are numbers over time, you can set thresholds on them ("alert if error rate > 5%") — which feeds directly into alerting (Topic 84). Dashboards show health; alerts catch problems automatically.

Why Averages Lie: Always Look at Percentiles

The most important practical lesson in metrics: never trust the average response time. Suppose your average latency is a healthy 100ms. That can still hide that 1% of users wait 5 seconds — and 1% of a million requests is 10,000 furious users. Averages get washed out by the bulk of fast requests. Instead, watch percentiles: p50 (median — typical user), p95 / p99 (the slowest 5% / 1% — your worst experiences), and p99.9 for the tail. Big companies obsess over the "tail latency" (p99) because at scale, a tiny percentage is a huge number of real people. When someone reports "the site feels slow" but your average looks fine, the percentiles reveal the truth.

Push vs Pull, and Cardinality (the silent killer)

Prometheus uses a pull model: it periodically "scrapes" a /metrics endpoint each service exposes. The advantage is Prometheus controls timing and can detect a service that's down (the scrape fails). The big operational gotcha is cardinality: each unique combination of metric labels creates a separate time series. Adding a label like user_id (millions of values) to a metric can explode into millions of series and crash your monitoring. Rule: labels should be low-cardinality (status code, endpoint, region) — never unbounded values like user IDs or request IDs (those belong in logs/traces).

Common misconceptions, cleared

"Average latency tells me how users feel." No — averages hide the slow tail; use p95/p99 percentiles.

"More labels/dimensions are always better." Dangerous — high-cardinality labels (user_id, request_id) explode the number of time series and can take down your metrics system.

"Metrics can tell me exactly why one request failed." No — metrics are aggregates; for a specific request you need logs (Topic 81) or a trace (Topic 83).

Interview tip

When asked how you'd monitor a system, name the golden signals and specify percentiles: "I'd track p99 latency, error rate, throughput, and saturation in Prometheus/Grafana, and alert on p99 and error-rate thresholds." Mentioning p99 tail latency and label cardinality marks you as someone who's run production systems.

Key Takeaways

  • Metrics = numeric measurements over time (latency, traffic, errors, saturation).
  • Logs = individual events; metrics = aggregated trends. Use both.
  • Watch the golden signals and use percentiles (p95/p99), not averages — the tail is where users suffer.
  • Keep metric labels low-cardinality — user/request IDs explode the series count.
  • Prometheus (pull/scrape) collects metrics; Grafana visualizes them and they drive alerts.

#83Distributed Tracing

One-line definition

Distributed tracing follows a single request as it travels through many microservices, recording how long each step took. It lets you see the full journey and pinpoint exactly where things slowed down or failed.

In microservices, one user request might pass through 10 services. If it is slow, which service caused it? Logs and metrics alone cannot easily show the end-to-end path. Distributed tracing solves this.

Real-life Analogy: Package Tracking

Tracking a parcel across hubs

When you track a package, you see every hub it passed through and how long it waited at each. If it is delayed, you see exactly where. Distributed tracing is package tracking for a request — you see each service it visited and the time spent at each.

Key Concepts

  • Trace: the entire journey of one request through the system.
  • Span: one step within the trace (one service's work). A trace is made of many spans.
  • Trace ID: a unique id attached to the request and passed along to every service, so all the spans can be stitched together.
Gatewayspan 5ms
→
Authspan 20ms
→
DBspan 300ms โš 
One trace, many spans — instantly revealing the slow step (the DB here).

Why It Is Essential for Microservices

  • Find bottlenecks: see exactly which service or call is slow.
  • Debug failures: follow where in the chain an error happened.
  • Understand dependencies: visualize how services call each other.

Tools: Jaeger, Zipkin, and the OpenTelemetry standard for instrumenting code.

The three pillars of observability

Logs (what happened), Metrics (the numbers/trends), and Traces (the request journey) are the "three pillars of observability". Together they let you fully understand and debug complex systems.

The Exact Problem Tracing Solves

Here's the scenario that makes tracing click. A user complains "the page took 8 seconds." Your metrics confirm p99 latency spiked — but across 12 microservices, which one was slow? Your logs have the clues, but they're scattered across 12 services with no easy way to connect them. Tracing ties it all together: every span carries the shared trace ID, and the trace view shows a waterfall — Gateway 5ms, Auth 20ms, Product 30ms, and then the Database span at 7,800ms. Instantly you know where. Metrics tell you something is wrong, logs tell you what, traces tell you where. That "where, across many services" is the gap only tracing fills.

How Spans Form a Tree (parent/child)

Spans aren't just a flat list — they nest. The root span (the whole request) has child spans for each downstream call, which can have their own children. This parent-child structure is what reveals not just total time but causality and concurrency: you can see that Service A called B and C in parallel, that C was the slow one, and that C's slowness came from its database call. Context propagation (passing the trace ID + parent span ID on every call, usually via HTTP headers) is what lets the tracer reconstruct this tree.

You Can't Trace Everything: Sampling

Tracing every single request at high traffic is expensive (storage + overhead), so systems sample — e.g. keep 1% of traces, or use "tail-based" sampling that keeps all the slow or errored traces (the interesting ones) and a sample of the rest. The art is capturing enough to debug without drowning in data. This mirrors the cost trade-offs in logging and metrics.

Common misconceptions, cleared

"Logs and metrics are enough; tracing is redundant." No — in microservices only tracing shows the end-to-end path and per-service timing of one request. The three pillars are complementary.

"Tracing happens automatically." It requires context propagation — the trace ID must be passed through every service call. Standards like OpenTelemetry and service meshes (Topic 52) help automate this.

"Trace 100% of requests." Usually impractical at scale — sample (ideally keeping all errors/slow requests).

Interview tip

For microservices designs, name all three pillars and tracing's unique role: "metrics for health/alerts, logs for detail, and distributed tracing (OpenTelemetry + Jaeger) with a propagated trace ID to pinpoint which service is slow." Mentioning context propagation and sampling shows hands-on understanding.

Key Takeaways

  • Distributed tracing follows one request across many services to find slow or failing steps — the "where".
  • A trace is a tree of spans (parent/child), tied together by a shared trace ID via context propagation.
  • Use sampling (ideally keep all errors/slow traces) to control cost at scale.
  • Essential for debugging microservices; tools: Jaeger, Zipkin, OpenTelemetry.
  • Logs + Metrics + Traces = the three complementary pillars of observability.

#84Alerting & On-call

One-line definition

Alerting automatically notifies engineers when something goes wrong (based on metrics/logs), and on-call is the rotation of engineers responsible for responding to those alerts — so problems get fixed fast, day or night.

Monitoring is useless if no one notices a problem. Alerting watches your metrics and logs and wakes someone up when things break, so issues are caught before (or right as) users are affected.

How Alerting Works

You define rules/thresholds on your metrics (e.g. "error rate > 5% for 5 minutes" or "disk > 90% full"). When a rule trips, an alert fires and notifies the on-call engineer via tools like PagerDuty, Opsgenie, or Slack.

The Golden Rule: Avoid Alert Fatigue

Too many alerts = ignored alerts

If you alert on everything, engineers get flooded and start ignoring alerts — so the real one gets missed. Alert only on things that need human action, ideally symptoms users feel (high latency, errors), not every minor blip. A good alert is urgent, actionable, and rare.

Severity Levels

  • Critical (page now): users are impacted — wake someone up immediately.
  • Warning: something to look at soon, but not an emergency.
  • Info: logged for awareness, no action needed.

On-call & Incident Response

  • On-call rotation: engineers take turns being responsible, so the load is shared fairly.
  • Runbooks: step-by-step guides for handling common alerts quickly.
  • Escalation: if the first responder does not react, the alert escalates to others.
  • Postmortems: after an incident, the team reviews what happened and how to prevent it — blameless, focused on improving the system.

Alert on Symptoms, Not Causes

The most valuable alerting principle: alert on what the user feels (symptoms), not on every internal cause. If you alert on every possible cause — "CPU high on server 7", "cache miss rate up", "queue depth growing" — you get a flood of pages, many of which don't actually matter (the system might be coping fine). Instead, alert on the symptom: "checkout error rate > 2%" or "p99 latency > 2s". If users aren't affected, it can usually wait until morning. When a symptom alert fires, you then use dashboards, logs, and traces to find the cause. This single shift dramatically cuts noise while catching everything that matters. (This is why metrics' golden signals from Topic 82 make the best alerts.)

SLO-Based Alerting and Error Budgets

Mature teams go further and alert against SLOs (Service Level Objectives) — a target like "99.9% of requests succeed." That 0.1% of allowed failures is your error budget. Instead of paging on every blip, you page when you're burning the error budget too fast (e.g. "at this rate we'll blow the monthly budget in 2 hours"). This ties alerts directly to user-facing reliability goals (recall the "nines" from Topic 53) and gives an objective answer to "is this worth waking someone up?". It also balances reliability vs shipping features: if you have budget to spare, move fast; if you've burned it, slow down and stabilize.

Why Blameless Postmortems Matter

After an incident, the goal is to fix the system, not punish a person. If engineers fear blame, they hide mistakes and you never learn the real cause. A blameless postmortem assumes everyone acted reasonably with the information they had, and asks "what about the system let this happen, and how do we prevent it?" The output is concrete action items (better alerts, guardrails, runbooks). This culture is what turns each outage into a permanent improvement instead of a repeat.

Common misconceptions, cleared

"More alerts = safer." The opposite — too many alerts cause alert fatigue, and the one that matters gets ignored. Fewer, symptom-based alerts are safer.

"Alert on every cause (CPU, memory, etc.)." Most causes don't warrant a page if users are unaffected. Alert on symptoms; investigate causes after.

"Postmortems are for finding who to blame." No — effective ones are blameless and focus on systemic fixes.

Interview tip

When discussing operations, say: "I'd alert on user-facing symptoms (error rate, p99 latency) against SLOs/error budgets — not every internal cause — with severity levels, runbooks, escalation, and blameless postmortems." Mentioning symptom-based alerting and error budgets signals real SRE maturity.

Key Takeaways

  • Alerting notifies engineers automatically when metrics/logs cross thresholds; on-call responds around the clock.
  • Avoid alert fatigue: alerts should be urgent, actionable, and rare.
  • Alert on symptoms (what users feel), not every internal cause — then investigate with logs/traces.
  • Mature teams alert on SLOs/error budgets, tying pages to real reliability goals.
  • Use severity levels, runbooks, escalation, and blameless postmortems to keep improving.

#85CI/CD Pipeline

One-line definition

CI/CD automates how code goes from a developer's laptop to production. CI (Continuous Integration) = automatically build and test every code change. CD (Continuous Delivery/Deployment) = automatically release that tested code to users.

Manually testing and deploying code is slow and error-prone. CI/CD builds an automated "assembly line" so changes ship quickly, safely, and frequently.

Real-life Analogy: A Factory Assembly Line

Automated checks at every station

A car factory does not hand-build each car randomly. It runs an assembly line where each station does a step and quality checks happen automatically. CI/CD is your software assembly line: code moves through build → test → deploy, with automatic checks at each stage.

Continuous Integration (CI)

Every time a developer pushes code, an automated pipeline:

  • Builds the code.
  • Runs tests automatically (unit, integration).
  • Reports pass/fail immediately, so bugs are caught early — before merging.

This means problems are found in minutes, not weeks, and the main branch stays healthy.

Continuous Delivery vs Continuous Deployment (CD)

  • Continuous Delivery: code is automatically tested and prepared for release, but a human clicks the final "deploy" button.
  • Continuous Deployment: goes one step further — every change that passes tests is automatically released to production, no human step.
Push code
→
Build
→
Test
→
Deploy
An automated pipeline from code push to production, with checks at each step.

Why It Matters

  • Faster, more frequent releases — ship improvements daily, not yearly.
  • Fewer bugs reach users (caught by automated tests).
  • Easy rollback — small frequent changes are easy to undo.
  • Less manual, error-prone work.

Tools: GitHub Actions, GitLab CI, Jenkins, CircleCI.

The Deep "Why": Small Batches Reduce Risk

The counterintuitive heart of CI/CD: deploying more often is safer, not riskier. If you batch up 3 months of changes into one giant release, and something breaks, you're hunting through hundreds of changes to find the culprit — and the blast radius is huge. If you deploy one small change at a time, and something breaks, you know exactly what caused it (the thing you just shipped) and rollback is trivial. Frequent small deploys turn deployment from a scary quarterly event into a boring daily non-event. This is why elite teams deploy dozens of times a day — not because they're reckless, but because small batches make each deploy low-risk.

The Pipeline Is Your Quality Gate

A real pipeline is more than "build → test → deploy"; each stage is a gate that bad code can't pass: compile/build, unit tests, integration tests, security scanning (dependencies, secrets), linting, and often deploy-to-staging with smoke tests before production. The principle is fail fast and fail cheap — catch the bug in a 2-minute CI run, not in production at 3 AM. A golden rule: a red build blocks the merge; keeping the main branch always-deployable is the whole point of CI.

Common misconceptions, cleared

"Continuous Delivery and Continuous Deployment are the same." Close but not identical — Delivery keeps production deploys one manual click away; Deployment ships automatically with no human step. The difference is that final gate.

"CI/CD means no testing needed." The opposite — CI/CD depends on strong automated tests; the pipeline is only as trustworthy as its test suite.

"Deploying often is risky." Small, frequent deploys are less risky than rare big-bang releases (smaller blast radius, easier rollback).

Interview tip

Connect CI/CD to safe releases: "every change runs through an automated pipeline (build, test, scan) that gates the merge, and we deploy small batches frequently with automated rollback" — then tie it to deployment strategies (Topic 88: blue-green/canary). Emphasizing small batches + fast rollback shows you understand the why.

Key Takeaways

  • CI = automatically build and test every code change to catch bugs early; a red build blocks the merge.
  • CD = automatically deliver (human clicks deploy) or deploy (fully automatic) tested code.
  • Small, frequent deploys are safer — tiny blast radius and trivial rollback.
  • The pipeline is a series of quality gates; it's only as good as your automated tests.
  • Tools: GitHub Actions, GitLab CI, Jenkins, CircleCI.

#86Containerization (Docker)

One-line definition

A container packages your application together with everything it needs to run (code, libraries, settings) into one portable unit, so it runs identically everywhere. Docker is the most popular tool for building and running containers.

"It works on my machine but not in production" is a classic nightmare caused by environment differences. Containers eliminate it by bundling the app and its environment together.

Real-life Analogy: A Shipping Container

Standardized boxes that fit any ship

Before shipping containers, loading goods onto ships was chaotic. Standardized metal containers changed everything — any container fits any ship, truck, or crane, regardless of what is inside. Software containers do the same: a standard unit that runs the same on any machine, laptop or cloud.

Containers vs Virtual Machines

Both isolate apps, but differently:

  • Virtual Machine (VM): includes a full guest operating system — heavy (gigabytes), slow to start.
  • Container: shares the host's OS kernel and includes only the app and its dependencies — lightweight (megabytes), starts in seconds.
VMfull OS each, heavy
vs
Containershares OS, light & fast
Containers are far lighter and faster than VMs because they share the host OS.

Key Docker Concepts

  • Image: a read-only blueprint of your app and its environment (built from a Dockerfile).
  • Container: a running instance of an image.
  • Registry: where images are stored and shared (e.g. Docker Hub).

Why Containers Are Everywhere

  • Consistency: runs the same on every machine — no "works on my machine".
  • Portability: move between laptop, server, and any cloud easily.
  • Efficiency: lightweight, so you can pack many on one machine.
  • Perfect for microservices — each service in its own container.

How Containers Are "Light": Sharing the Kernel

The reason a container is megabytes and starts in milliseconds, while a VM is gigabytes and takes minutes, comes down to one thing: a VM virtualizes hardware and runs a full guest OS on top, whereas a container just shares the host's OS kernel and isolates the process using Linux features (namespaces for isolation, cgroups for resource limits). So a container is really "just a process on the host, fenced off to think it's alone." No duplicate OS means tiny size and instant startup — which is exactly what lets you pack hundreds onto one machine and spin them up/down for autoscaling. The trade-off: containers share the host kernel, so isolation is slightly weaker than a VM's (which matters for multi-tenant security — some run containers inside lightweight VMs to get both).

The Layered Image (and why it's efficient)

A Docker image is built in layers from the Dockerfile (base OS layer, then dependencies, then your code). Layers are cached and shared: if ten images use the same base layer, it's stored once. When you change only your code, only that top layer rebuilds — making builds and deployments fast. This is also why image order matters: put rarely-changing things (dependencies) early and frequently-changing things (your code) last, to maximize cache reuse.

Common misconceptions, cleared

"A container is a lightweight VM." Conceptually similar (isolation) but fundamentally different — no guest OS; it shares the host kernel. That's why it's far lighter.

"Containers are fully isolated like VMs." Not quite — sharing the kernel means weaker isolation; for hostile multi-tenant workloads you may need VM-level boundaries.

"Docker scales my app automatically." No — Docker packages and runs containers; scaling/self-healing across machines is the orchestrator's job (Kubernetes, Topic 87).

Interview tip

Frame containers as the unit of deployment: "each service ships as a container image, guaranteeing identical environments from laptop to prod, and orchestrated by Kubernetes." Knowing the kernel-sharing distinction from VMs (and its isolation trade-off) is the depth interviewers look for.

Key Takeaways

  • A container bundles an app with everything it needs, so it runs identically everywhere.
  • Docker is the leading container tool (image = layered blueprint, container = running instance).
  • Containers are lighter/faster than VMs because they share the host kernel (no guest OS) — at the cost of weaker isolation.
  • Layered images cache and share layers, making builds and deploys fast.
  • They solve "works on my machine" and are ideal for microservices; scaling them is the orchestrator's job.

#87Orchestration (Kubernetes)

One-line definition

Kubernetes (K8s) is a system that automatically manages lots of containers across many machines — deploying them, scaling them up/down, restarting failed ones, and load balancing — so you do not have to do it by hand.

One or two containers are easy to run yourself. But hundreds of containers across dozens of servers? You need an automatic "manager". That manager is Kubernetes — the industry standard for container orchestration.

Real-life Analogy: An Orchestra Conductor

Coordinating many players

An orchestra has many musicians (containers). A conductor (Kubernetes) makes sure the right ones play at the right time, keeps everyone in sync, and brings in a replacement if someone stops. You describe the music you want; the conductor handles the coordination.

What Kubernetes Does Automatically

  • Scheduling: decides which machine each container runs on.
  • Self-healing: if a container crashes, it restarts or replaces it automatically (using health checks, Topic 59).
  • Auto-scaling: adds more containers when load rises, removes them when it falls.
  • Load balancing & service discovery: spreads traffic and lets containers find each other.
  • Rolling updates & rollbacks: deploy new versions gradually, roll back if something breaks.

Key Concepts (simply)

  • Pod: the smallest unit — one or a few tightly-coupled containers.
  • Node: a machine that runs pods.
  • Cluster: all the nodes together, managed by the control plane.
  • Deployment: a declaration of how many copies of an app you want; K8s makes reality match it.
  • Service: a stable address to reach a set of pods.
The big idea: declarative

You tell Kubernetes the desired state ("I want 5 copies of this app running"), and it constantly works to make the actual state match — restarting, rescheduling, and scaling as needed. You declare the goal; K8s figures out how.

The Reconciliation Loop — Kubernetes' Beating Heart

Understand this one concept and Kubernetes makes sense. K8s constantly runs a control loop: compare desired state ("5 replicas") with actual state ("4 running"), and take action to close the gap (start 1 more). It never stops doing this. A node dies and takes 2 pods with it? The loop notices actual=3, desired=5, and reschedules 2 pods elsewhere — self-healing with no human involved. This is declarative management: you don't write "start this container on that server" (imperative); you declare the end goal and K8s figures out the steps and maintains it forever. It's the same idea as the health-check/heartbeat self-healing from Topic 59, applied to the whole cluster.

So You Get Most of Parts 6–7 "For Free"

Notice how Kubernetes bundles concepts you already learned: Service = load balancing + service discovery (Topics 15, 51); liveness/readiness probes = health checks (Topic 59); Horizontal Pod Autoscaler = auto-scaling (Topic 46); Deployment = rolling updates with rollback (Topic 88). That's why it became the standard — it operationalizes the reliability/scalability patterns from this whole course into one declarative platform.

Common misconceptions, cleared

"Kubernetes makes my app scalable by itself." Only if your app is stateless (Topic 47) — K8s can freely create/kill/move stateless pods, but stateful apps need extra care (StatefulSets, external state stores).

"K8s replaces Docker." No — it orchestrates containers (built by Docker/equivalents); they're complementary layers.

"You always need Kubernetes." It's powerful but complex — for a small app, it's often overkill (recall "don't over-engineer", Topic 46). Use it when you genuinely have many services/scale.

Interview tip

Explain K8s via the loop: "I declare desired state (replicas, resources, probes) and Kubernetes' reconciliation loop maintains it — self-healing, autoscaling, rolling updates — as long as services are stateless." Linking pods/services/probes back to load balancing, discovery, and health checks shows you understand what it automates, not just the name.

Key Takeaways

  • Kubernetes automatically manages many containers across many machines via a reconciliation loop.
  • It continuously makes actual state match desired state — scheduling, self-healing, auto-scaling, load balancing, rolling updates.
  • Concepts: Pod, Node, Cluster, Deployment, Service; it bundles many Part 6–7 patterns into one platform.
  • It's declarative: you state the goal, K8s figures out how — works best with stateless apps.
  • Powerful but complex — don't reach for it on small systems.

#88Blue-Green Deployment

One-line definition

Blue-Green Deployment is a release strategy where you run two identical environments — "Blue" (current live version) and "Green" (new version). You deploy and test on Green, then switch all traffic to it instantly. If anything breaks, switch back instantly.

Deploying a new version directly onto live servers is risky — if it is broken, users suffer while you scramble. Blue-green removes this risk with an instant, reversible switch.

How It Works

  • Blue is the current version serving all users.
  • You deploy the new version to Green (an identical but idle environment) and test it thoroughly.
  • When ready, you flip the load balancer to send all traffic to Green. The switch is instant.
  • If a problem appears, you flip back to Blue immediately — near-zero downtime rollback.
Blue (live)old version
switch →
Green (new)tested version
Flip traffic from Blue to Green instantly — and back if needed.

Pros and Cons

Pros

  • Zero/near-zero downtime deployments.
  • Instant rollback — just switch back.
  • Test the new version in a real environment before going live.

Cons

  • Costly — you run two full environments.
  • Database changes can be tricky to keep compatible across both.
  • All users switch at once (no gradual exposure — see canary, next).

The Database: Blue-Green's Hardest Problem

Switching app servers is easy; the catch almost everyone misses is the shared database. Blue and Green usually point at the same database, so the new version's schema changes must work with the old version too — otherwise instant rollback is impossible (you'd flip back to Blue, but the schema already changed under it). The professional technique is backward-compatible, expand-then-contract migrations: first add the new column/table (both versions still work), deploy and verify the new code, and only later remove the old column in a separate step. Never rename/drop in the same release you deploy — that's what breaks rollback. This decoupling of schema changes from code changes is the real skill behind safe deployments.

Common misconceptions, cleared

"Blue-green means truly zero risk." Only for the app tier — the shared database and in-flight sessions still need backward-compatible handling, or rollback breaks.

"It's the same as canary." No — blue-green switches everyone at once (fast, all-or-nothing); canary exposes a small slice gradually (Topic 89). Different risk profiles.

"You always need two full environments forever." The idle environment is mainly needed around release time; cloud autoscaling makes spinning Green up/down cheaper than a permanent double.

Interview tip

When proposing blue-green, preempt the DB question: "blue-green for instant rollback, with backward-compatible (expand/contract) DB migrations so I can switch back safely." Raising the schema-compatibility issue unprompted is a strong senior signal.

Key Takeaways

  • Blue-green = two identical environments; deploy/test on the idle one, then switch traffic instantly.
  • Gives near-zero downtime and instant rollback.
  • Cost: running two full environments; all users switch at once.
  • The hard part is the shared database — use backward-compatible expand/contract migrations to keep rollback safe.
  • Great when you need safe, reversible, all-at-once releases (vs canary's gradual approach).

#89Canary Deployment

One-line definition

Canary Deployment releases a new version to a small percentage of users first (the "canary" group). If it behaves well, you gradually roll it out to everyone. If it breaks, only a few users were affected.

Blue-green switches everyone at once. Canary is more cautious: expose the new version to a tiny slice of real users, watch closely, then ramp up only if all looks healthy.

Why "Canary"?

The canary in the coal mine

Miners once brought canary birds underground. If dangerous gas built up, the sensitive canary reacted first, warning miners before they were harmed. A canary deployment is your early-warning system: a small group of users hits the new version first, revealing problems before everyone is exposed.

How It Works

  • Deploy the new version alongside the old one.
  • Route a small slice of traffic (e.g. 1%, then 5%, then 25%) to the new version.
  • Monitor metrics (errors, latency) for the canary closely.
  • If healthy, gradually increase the percentage to 100%. If not, route everyone back to the old version.
95% → old version
5% → new (canary)watch closely
A small slice tries the new version first; ramp up only if it stays healthy.

Canary vs Blue-Green

PointBlue-GreenCanary
RolloutAll users at onceGradual (few → all)
Risk exposureEveryone if brokenOnly the small group
RollbackInstant switch backStop ramp, route back
Best forFast, clean switchesCautious, data-driven rollouts

Canary Only Works If You Can Measure It

The make-or-break dependency: canary is only as good as your observability (Part 11). The whole point is to compare the canary's health against the old version — error rate, p99 latency, key business metrics — and decide automatically or quickly whether to ramp up or abort. Without solid metrics (Topic 82) and alerting (Topic 84), you're rolling out blind and the canary gives a false sense of safety. Mature setups do automated canary analysis: the deploy tool watches the metrics and auto-promotes or auto-rolls-back. This is the concrete payoff of having built good monitoring.

Who Gets to Be the Canary?

A subtlety: you must pick the canary slice thoughtfully. A purely random 1% is common, but you might route internal employees or beta users first (they tolerate bugs better), or a single region (to contain blast radius geographically). You also want the slice to be representative — if your canary only gets low-traffic users, it won't surface load-related bugs. And give it enough time/traffic to be statistically meaningful before promoting.

Common misconceptions, cleared

"Canary is just a slow blue-green." No — blue-green runs two environments and flips all at once; canary runs versions side by side and shifts traffic gradually, watching metrics at each step.

"1% for a minute is enough." Often not — too little traffic/time hides issues. Size and duration must be statistically meaningful.

"Canary removes the need for testing." No — it's a final safety net in production, not a replacement for CI tests.

Interview tip

Tie canary to observability: "canary release to 1→5→25→100%, with automated canary analysis on error rate and p99 latency to auto-rollback — limiting blast radius to a small slice." Mentioning automated analysis and a representative slice shows real deployment maturity.

Key Takeaways

  • Canary = release to a small % of users first, monitor, then gradually roll out to all.
  • It limits the blast radius — only a few users hit a bad release.
  • It's only as good as your monitoring — ideally automated canary analysis drives promote/rollback.
  • Choose a representative canary slice and give it enough traffic/time to be meaningful.
  • Blue-green = switch all at once; canary = gradual, cautious, metrics-driven rollout.

#90Feature Flags

One-line definition

Feature flags (feature toggles) are switches in your code that let you turn features on or off without deploying new code. You can release a feature to specific users, or instantly disable it if it misbehaves.

Normally, changing what users see means deploying new code. Feature flags decouple deploying code from releasing a feature — giving you a remote-control switch over functionality.

Real-life Analogy: A Light Switch

Wired in, but controllable

The wiring for a light is installed in the wall (code is deployed), but the switch decides whether the light is on. Feature flags are switches for features: the code ships dark, and you flip it on when ready — or off the moment there is trouble.

What Feature Flags Enable

  • Gradual rollouts: enable a feature for 5% of users, then more (like canary, but controlled in code).
  • Instant kill switch: if a feature breaks, turn it off immediately — no redeploy needed.
  • A/B testing: show version A to some users and B to others to compare results.
  • Targeting: enable features for specific users (beta testers, a region, paying customers).
  • Decouple deploy from release: merge code anytime, flip it on later.
Code deployed (flag OFF)
flip ON →
Feature livefor chosen users
The feature ships hidden behind a flag, then is switched on remotely.
Keep flags tidy

Old, unused flags pile up and clutter code ("flag debt"). Remove flags once a feature is fully rolled out and stable. Tools like LaunchDarkly help manage flags at scale.

The Big Idea: Decouple Deploy from Release

This is the concept that elevates feature flags from "a config switch" to a core engineering practice. Normally "deploy" (ship code to servers) and "release" (turn it on for users) happen together — which is risky and forces big-bang launches. Flags split them: you can deploy code continuously (merged, tested, in production but dark) and release independently (flip the flag whenever business wants). This unlocks trunk-based development — engineers merge small changes daily behind flags instead of maintaining long, painful feature branches. The unfinished feature is in production but invisible until ready. Deploy becomes an engineering event; release becomes a business decision.

Flags vs Canary/Blue-Green (they complement, not compete)

These overlap, so know the distinction. Blue-green and canary (Topics 88–89) operate at the infrastructure/traffic level — routing requests between versions of the whole app. Feature flags operate at the application/code level — toggling individual features for chosen users within a single deployed version, with no redeploy and no infra changes. Crucially, a flag gives an instant kill switch: if a feature misbehaves, you flip it off in seconds, faster than even a rollback. Real teams use both: deploy safely with canary, then control feature exposure with flags.

Common misconceptions, cleared

"Feature flags are the same as canary deployments." No — canary routes traffic between app versions (infra level); flags toggle features in code (app level) without redeploying.

"Flags are free." They add complexity — every flag is a branch in your code (and a test combination). Untracked flags become "flag debt" and a source of bugs; clean them up.

"Flags are just for rollouts." Also used for A/B experiments, user/region targeting, kill switches, and selling tiered plans (entitlements).

Interview tip

Frame flags around decoupling: "feature flags decouple deploy from release — ship code dark via CI/CD, then release gradually or instantly kill it, enabling trunk-based development and A/B tests." Distinguishing app-level flags from infra-level canary/blue-green shows you understand the full release toolbox.

Key Takeaways

  • Feature flags turn features on/off without deploying code.
  • They enable gradual rollouts, instant kill switches, A/B testing, and user targeting.
  • The big idea: decouple deploying code from releasing a feature — enabling trunk-based development.
  • They work at the app/code level (vs canary/blue-green at the infra/traffic level) and complement them.
  • Clean up old flags to avoid "flag debt"; tools like LaunchDarkly help manage them.

#91Design a URL Shortener (like bit.ly / TinyURL)

The goal

Build a service that turns a long URL into a short one (e.g. bit.ly/3xYz9), and redirects anyone who visits the short link to the original long URL.

This is the classic "first" system design problem. It is simple enough to finish, yet touches hashing, databases, caching, and scale. We will use a clear framework you can reuse for every case study.

Step 1: Requirements

  • Functional: shorten a long URL; redirect a short URL to the original; (optional) custom aliases, expiry, click analytics.
  • Non-functional: very low latency on redirects, high availability, scale to billions of URLs. Reads >> writes (far more clicks than new links).

Step 2: Scale Estimate (back-of-envelope)

Say 100 million new URLs/month and a 100:1 read:write ratio. That is ~40 writes/sec and ~4,000 reads/sec — clearly read-heavy, so caching will matter a lot. Over years, billions of records → we will likely need to shard the database.

Step 3: The Core Problem — Generating the Short Key

The short code (e.g. 3xYz9) must be unique and short. Two common approaches:

  • Base-62 of a counter: keep a global incrementing number and encode it in base-62 (a–z, A–Z, 0–9). 7 characters of base-62 give ~3.5 trillion combinations. Guaranteed unique, but the counter must be distributed (e.g. each server gets a range of IDs).
  • Hash the URL (e.g. MD5) and take the first 7 chars: simple, but can collide — you must check and retry on collision.

Base-62 of a unique ID is the cleaner, collision-free choice.

Step 4: High-Level Design

Client
→
Load Balancer
→
App servers
→
Cache (Redis)
miss →
Database (key → long URL)
Redirects check the cache first; the database is the source of truth.
  • Write path: generate a unique ID → base-62 encode → store shortKey → longURL in the database.
  • Read path (redirect): look up shortKey in cache first (most clicks are for popular links); on a miss, read from DB and cache it. Return an HTTP 301/302 redirect to the long URL.

Step 5: Scaling the Pieces

  • Database: a key-value store (the lookup is just shortKey → longURL) — easy to shard by shortKey. NoSQL fits well.
  • Caching: Redis holds hot links — this absorbs the read-heavy traffic.
  • CDN/edge: can cache redirects close to users.
  • Analytics: log clicks asynchronously (to a queue) so they never slow the redirect.

Deep Dive: 301 vs 302 (a favourite follow-up)

A subtle but classic probe: should the redirect be 301 (permanent) or 302 (temporary)? It's a real trade-off. A 301 tells the browser "remember this forever" — so the browser caches it and future clicks skip your server entirely. Great for latency and load, terrible if you want click analytics (you never see the repeat clicks) or ever need to change/disable the link. A 302 forces the browser to come back to you each time — so you can count every click and control the link, at the cost of more traffic. Most analytics-driven shorteners (bit.ly) choose 302 precisely so they can track clicks. Naming this trade-off out loud is exactly the senior signal interviewers want.

Deep Dive: Distributed Unique ID Generation

The base-62 approach needs a globally unique number — but a single auto-increment counter is a bottleneck and SPOF. Real answers: (a) a ID-range/ticket server where each app server grabs a block of 1,000 IDs at a time (few coordination calls); (b) Snowflake-style IDs (timestamp + machine id + sequence) that each server generates independently with no coordination; or (c) Zookeeper-managed ranges. This ties back to leader election / coordination (Topics 64–65) and is the most common deep-dive after "how do you make the key."

Interview tip โ€” what they probe

Expect: "How do you guarantee uniqueness across servers?" (distributed IDs), "301 vs 302?" (analytics trade-off), "What if a custom alias is taken?" (check-and-reject), and "How do you handle the read-heavy load?" (cache + CDN). Lead with the read:write ratio — it justifies every later decision.

Key Takeaways

  • Follow the framework: requirements → scale estimate → core problem → high-level design → scaling.
  • The core trick is generating a short, unique key — base-62 of a unique ID avoids collisions (use ID ranges or Snowflake for distributed uniqueness).
  • 301 vs 302 is a real trade-off: 302 lets you keep click analytics and control; 301 is faster but uncacheable to track.
  • It is read-heavy, so caching (Redis) is the key performance lever.
  • Use a key-value store sharded by short key, and log analytics asynchronously.

#92Design Twitter / Instagram Feed

The goal

Build a social feed: users post content, follow others, and see a timeline of posts from people they follow — fast, at huge scale.

Step 1: Requirements

  • Functional: post a tweet/photo; follow users; view a home timeline (recent posts from those you follow).
  • Non-functional: timeline must load very fast, scale to hundreds of millions of users, highly available. Eventual consistency is fine (a post appearing a second late is OK).

Step 2: The Core Challenge — Building the Timeline

The hard part is generating each user's home timeline efficiently. There are two strategies:

Fan-out on Write (Push model)

When a user posts, immediately push that post into the precomputed timeline of every follower (stored in a cache like Redis). Reading the timeline is then instant — just read the precomputed list.

  • Pro: reads are super fast (most important, since reads dominate).
  • Con: a celebrity with 100M followers causes a huge write fan-out ("the celebrity problem").

Fan-out on Read (Pull model)

Do nothing special on write. When a user opens their timeline, fetch and merge the latest posts from everyone they follow on the fly.

  • Pro: cheap writes, no fan-out explosion.
  • Con: reads are slow and expensive (merging many users' posts each time).

The Hybrid Solution (what real systems do)

Best of both

Use fan-out on write for normal users (fast reads), but for celebrities with millions of followers, use fan-out on read (pull their posts at read time). This avoids the massive write storm while keeping most timelines fast. This hybrid is how Twitter/Instagram actually do it.

User posts
→
Fan-out service
→
Followers' timeline caches
Posts are pushed into followers' precomputed timelines (except for celebrities).

Other Components

  • Media (photos/videos): stored in object storage (S3) + served via CDN.
  • Databases: posts and the social graph (who follows whom) — sharded NoSQL/SQL.
  • Cache (Redis): holds timelines and hot data.
  • Async processing: fan-out and notifications via message queues.

Deep Dive: The Celebrity Problem, Solved Precisely

This is the defining challenge of the feed and the most probed part. Plain fan-out-on-write breaks for celebrities: when someone with 100M followers tweets, you'd do 100M timeline writes — a massive, slow spike, and most of those followers may not even be online. The elegant fix is the hybrid model: use fan-out on write (push) for normal users (fast reads for the common case), but for the handful of celebrity accounts, fan-out on read (pull) — don't push their tweets at all. When you load your timeline, the system merges your precomputed push-timeline with a live fetch of the few celebrities you follow. This caps the write explosion while keeping reads fast for everyone. Explaining this hybrid (and why each side is chosen) is what separates a strong answer from a textbook one.

Deep Dive: Timeline Storage and Merging

Each user's home timeline is a precomputed list of tweet IDs in Redis (not full tweets — store IDs and hydrate from a cache/DB to save memory). You cap it (e.g. latest ~800 entries) since nobody scrolls forever. On read, you merge push-timeline IDs with freshly-pulled celebrity tweet IDs, sort by time (or a ranking score), then hydrate. This is a concrete application of denormalization/precomputed feeds (Topic 25) and caching (Part 4).

Interview tip โ€” what they probe

They'll push on: "What happens when a celebrity posts?" (hybrid push/pull), "How do you store the timeline?" (IDs in Redis, capped, hydrate later), "Chronological or ranked feed?" (ranking adds an ML scoring step), and "Is eventual consistency OK?" (yes — a 1-second delay is fine). Always state reads >> writes up front.

Key Takeaways

  • The core challenge is the timeline: fan-out on write (fast reads) vs fan-out on read (cheap writes).
  • Real systems use a hybrid: push for normal users, pull for celebrities — merged at read time.
  • Store tweet IDs in a capped Redis timeline and hydrate full content from cache/DB.
  • Media goes to object storage + CDN; it is read-heavy and tolerates eventual consistency.

#93Design WhatsApp / Chat System

The goal

Build a real-time messaging app: one-to-one and group chats, instant delivery, online/last-seen status, delivery and read receipts, and offline message delivery.

Step 1: Requirements

  • Functional: send/receive messages in real time; group chats; delivery + read receipts (ticks); online status; deliver messages sent while you were offline.
  • Non-functional: low latency, highly available, messages must never be lost, scale to billions of messages/day.

Step 2: The Core — Real-Time Delivery

Messages must arrive instantly, and the server must be able to push to clients. So we use persistent connections — WebSockets (Topic 42) — not request/response polling.

  • Each online user holds a WebSocket connection to a chat/gateway server.
  • A session service tracks which server each user is connected to (so we know where to route a message).
User A
↔ WS
Chat serversroute via session registry
↔ WS
User B
Persistent WebSocket connections let the server push messages instantly.

Step 3: Message Flow

  • A sends a message → chat server stores it and looks up where B is connected.
  • If B is online: push it over B's WebSocket instantly → send "delivered" receipt back to A.
  • If B is offline: store the message in a per-user inbox/queue; deliver when B reconnects (offline delivery).
  • Read receipts: when B opens the chat, send a "read" event back to A.

Step 4: Storage & Scale

  • Messages DB: a write-heavy, huge-volume store — a column-family NoSQL like Cassandra fits well, sharded by chat/user.
  • Group chats: the server fans out one message to all group members' connections/inboxes.
  • Queues: ensure reliable delivery and handle offline users.
  • End-to-end encryption: messages encrypted on the device so even servers cannot read them.

Deep Dive: How "Never Lost" Actually Works

The strict requirement "messages must never be lost" forces a specific design. WebSockets alone aren't enough — a push can fail mid-flight. The pattern: when A sends a message, the server first persists it durably (to the messages DB / a queue) and acknowledges to A, then attempts delivery to B. This is essentially at-least-once delivery (Topic 38): the message survives even if B is offline or the push fails. Each message gets a unique ID so the client can deduplicate if it's delivered twice (idempotency, Topic 57). The ticks map cleanly to states: one tick = stored on server, two ticks = delivered to B's device, blue ticks = B read it — each is just an event flowing back to A.

Deep Dive: Connection Routing at Scale

With hundreds of millions of users, you can't hold all WebSockets on one server. So you have a fleet of gateway servers, and a session registry (e.g. Redis) maps user → which gateway they're connected to. To deliver to B, the server looks up B's gateway and forwards the message there (often via an internal pub/sub backplane, Topic 39). This is the stateful-connection scaling challenge from Topic 42 made concrete. Group chats fan out: look up each member's gateway and deliver — the same fan-out trade-off as the Twitter feed, just smaller groups.

Interview tip โ€” what they probe

Expect: "How do you guarantee no message loss?" (persist-then-deliver, at-least-once + dedup), "How do you find which server B is on?" (session registry + backplane), "How do groups scale?" (fan-out, and huge groups need care), and "E2E encryption?" (encrypt on-device; server stores ciphertext). Emphasize persist-before-ack — it's the heart of the answer.

Key Takeaways

  • Use WebSockets for instant, server-pushed delivery; a session registry tracks where each user is connected.
  • Persist-then-deliver (at-least-once) + message IDs for dedup = messages are never lost, even offline.
  • Store huge message volumes in a write-optimized NoSQL (Cassandra), sharded.
  • Route across gateways via a session registry + pub/sub backplane; receipts are events; groups fan-out.
  • Add E2E encryption — the server only ever stores ciphertext.

#94Design Netflix / YouTube (Video Streaming)

The goal

Build a video platform: upload videos, process them, and stream smoothly to millions of viewers worldwide on any device and network speed.

Step 1: Requirements

  • Functional: upload video; watch video (stream); search; recommendations.
  • Non-functional: smooth playback worldwide, handle huge read traffic, adapt to varying network speeds, highly available. Extremely read-heavy.

Step 2: The Upload & Processing Pipeline

When a video is uploaded, it cannot be streamed as-is. It goes through an asynchronous pipeline (the user does not wait):

  • Transcoding: convert the video into multiple resolutions/bitrates (240p, 480p, 720p, 1080p, 4K).
  • Segmenting: chop each version into small chunks (a few seconds each).
  • Store all chunks in object storage (S3) and distribute to the CDN.

This is done via a queue + worker fleet (Topic 38), so processing happens in the background.

Step 3: The Core — Adaptive Streaming via CDN

How smooth playback works

Video is served from CDN edge servers close to viewers (Topic 18) — this is what makes it fast and saves the origin. Playback uses adaptive bitrate streaming (HLS/DASH): the player monitors your network and automatically switches to a higher or lower quality chunk-by-chunk, so it never stalls. Slow network → lower quality but no buffering.

Upload
→
Transcode + segmentmany qualities, async
→
Object storage + CDN
→
Viewers (adaptive)
Process once into many qualities; serve globally via CDN with adaptive bitrate.

Step 4: Other Components

  • Metadata DB: titles, descriptions, user data — in a regular database.
  • Search: Elasticsearch (Topic 80).
  • Recommendations: a separate ML pipeline on viewing history (big data, Topic 78).
  • View counts/analytics: processed asynchronously via streams (Kafka).

Deep Dive: How Adaptive Bitrate Actually Works

This is the part interviewers love. Because each video is transcoded into many qualities and every quality is chopped into small segments (~2–10s each), the player can switch quality between segments without stopping. A small manifest file (HLS .m3u8 / DASH .mpd) lists all the available qualities and segment URLs. The player downloads segments into a buffer, continuously measures your bandwidth, and picks the highest quality that won't cause a stall: bandwidth drops → next segment is 480p; bandwidth recovers → back to 1080p. The intelligence lives in the client, not the server — the server just hosts every variant on the CDN. That's why your video dips in quality instead of buffering on a weak connection.

Deep Dive: Why the CDN Is the Whole Game

At streaming scale, the bottleneck isn't compute — it's bandwidth. Serving millions of concurrent streams from one origin is impossible and ruinously expensive. The CDN (Topic 18) caches segments at edge locations near viewers, so 99%+ of bytes are served from the edge, never touching your origin. Netflix goes further with Open Connect — placing its own caching appliances inside ISPs, so popular content is literally next door to you. The design insight: process the video once (expensive, async, offline) so you can serve it billions of times cheaply from cache. Compute-once-serve-many is the recurring theme.

Interview tip โ€” what they probe

They'll ask: "How does playback adapt to network speed?" (client-driven ABR + manifest + segments), "How do you serve millions of streams?" (CDN edge caching, the origin barely gets hit), "Why transcode into many versions up front?" (compute once, serve many; support all devices), and "Live vs on-demand?" (live adds real-time segmenting + tighter latency). Stress that it's bandwidth-bound and read-heavy.

Key Takeaways

  • Uploads run through an async transcoding pipeline into many resolutions and small segments.
  • Adaptive bitrate is client-driven: a manifest lists qualities/segments; the player switches per-segment to avoid stalls.
  • CDN edge caching is the hero — it's a bandwidth problem; process once, serve billions of times.
  • Store video segments in object storage; keep metadata in a database; search via Elasticsearch.

#95Design Uber / Ola (Ride-Hailing)

The goal

Build a ride-hailing system: riders request rides, the system finds nearby drivers, matches them, and tracks the trip live on a map — in real time.

Step 1: Requirements

  • Functional: riders request a ride; match with a nearby available driver; live location tracking; fare calculation; trip lifecycle.
  • Non-functional: real-time and low latency, highly available, handle constant location updates from millions of drivers.

Step 2: The Core Challenge — "Find Nearby Drivers" Fast

The hard problem is efficiently finding drivers near a rider, given drivers' locations change every few seconds. Scanning everyone is impossible at scale. The solution is geospatial indexing.

Geo-hashing / QuadTrees

Divide the world map into a grid of cells (using techniques like geohashing, QuadTree, or Uber's H3 hexagons). Each driver is indexed by which cell they are in. To find nearby drivers, you only look in the rider's cell and its neighbors — a tiny search instead of scanning the whole planet.

Step 3: Real-Time Location Updates

  • Drivers' apps send location every few seconds over a persistent connection (WebSockets).
  • A Location Service keeps current driver positions in a fast in-memory store (Redis) with the geospatial index.
  • This is write-heavy (constant updates) — an in-memory store handles the churn.
Rider requests
→
Matching servicequery nearby cells
→
Location service (Redis geo-index)
Matching only searches the rider's grid cell and its neighbors — fast.

Step 4: Matching & Trip Flow

  • Matching service finds nearby available drivers, ranks them (distance, rating), and offers the ride.
  • Once accepted, a Trip service tracks the trip state and streams live location to the rider.
  • Pricing service computes fares (including surge pricing based on demand/supply in an area).
  • Events (ride completed, payment, ratings) flow through queues/Kafka (event-driven, Topic 40).

Deep Dive: Why Geospatial Indexing Beats a Naive Query

The naive approach — "compute distance from the rider to every driver and keep the closest" — is O(N) over millions of drivers per request: impossible at scale. Geo-indexing turns this into a tiny lookup. With geohashing, every location maps to a short string where nearby places share a prefix (e.g. all drivers in one ~1km cell start with tdr1y). So "find nearby drivers" becomes "find drivers whose geohash matches the rider's cell prefix" — an index lookup, not a full scan. You check the rider's cell plus its 8 neighbours (to handle the rider being near a cell edge). Uber's H3 uses hexagons instead of squares because hexagons have uniform distance to all neighbours (no awkward corners), which makes proximity math cleaner. This is the make-or-break idea of the whole design.

Deep Dive: Handling the Write Storm

Millions of drivers each sending GPS every ~4 seconds is a relentless write-heavy firehose — far too much to put in a normal disk-based DB. The answer: keep the live geo-index in memory (Redis), which has native geospatial commands. Old positions are simply overwritten; you don't need history for matching (that goes to analytics async). This is a textbook case of choosing the right tool for a write-heavy, ephemeral workload (recall TSDB/in-memory discussions). Matching also has a concurrency catch: two riders mustn't be matched to the same driver — you need a lock/atomic claim on a driver during dispatch.

Interview tip โ€” what they probe

Expect: "How do you find nearby drivers efficiently?" (geohash/QuadTree/H3, search cell + neighbours), "How do you handle millions of location updates?" (in-memory Redis geo-index, overwrite), "How do you avoid matching one driver to two riders?" (atomic claim/lock), and "Surge pricing?" (supply/demand ratio per cell). The geo-index is the centerpiece — lead with it.

Key Takeaways

  • The core problem is finding nearby drivers fast — solved with geospatial indexing (geohash/QuadTree/H3) instead of an O(N) scan.
  • Search the rider's cell + neighbouring cells; H3 hexagons give uniform neighbour distances.
  • The location firehose is write-heavy — keep it in an in-memory geo-index (Redis), overwriting positions.
  • Use an atomic claim so one driver isn't matched twice; separate matching, trip, pricing services via events.

#96Design Swiggy / Zomato (Food Delivery)

The goal

Build a food delivery system: discover nearby restaurants, place an order, have the restaurant prepare it, assign a delivery partner, and track delivery live.

Step 1: Requirements

  • Functional: browse nearby restaurants & menus; place an order; restaurant accepts & prepares; assign a delivery partner; live order tracking.
  • Non-functional: low latency discovery, reliable orders (never lost), real-time tracking, handle peak meal-time spikes.

Step 2: Two Core Sub-problems

This combines two earlier ideas:

  • Restaurant discovery = a geospatial + search problem (find nearby open restaurants, filter by cuisine/rating) — like Uber's nearby search (Topic 95) plus Elasticsearch (Topic 80).
  • Delivery partner assignment = a matching problem (find a nearby free rider) — same geo-indexing as Uber.

Step 3: The Order Lifecycle (event-driven)

An order moves through many states, handled asynchronously so the app stays responsive:

Placed
→
Restaurant accepts
→
Preparing
→
Partner assigned & delivers
Each state transition emits events; services react (notify, assign, track).
  • The Order service is the source of truth for order state.
  • State changes emit events (Topic 40) — the notification service texts you, the assignment service finds a rider, analytics logs it.
  • Message queues ensure orders and steps are never lost, even under spikes (Topic 38).

Step 4: Components

  • Search/discovery service: Elasticsearch + geo-index for nearby restaurants.
  • Order service + reliable DB: orders need correctness (lean toward strong consistency).
  • Location/tracking service: live partner location via WebSockets + Redis (like Uber).
  • Notification service: push updates at each step (Topic 98).

Deep Dive: Why the Order Service Needs Strong Consistency

Unlike a social feed (eventual consistency is fine), an order is money and a promise — you can't lose it, double it, or show conflicting states. So the Order service uses a strongly consistent store (Topic 61) and treats each state transition as a recorded, ordered fact. The trick that keeps the app responsive and correct: the order write itself is a fast, consistent transaction, but everything that reacts to it (notify the user, find a delivery partner, update analytics) happens asynchronously via events (Topic 40). So correctness lives in the order DB; speed comes from async fan-out. This separation — consistent core, eventually-consistent reactions — is the key design judgment.

Deep Dive: Surviving the Meal-Time Spike

Food delivery has brutal, predictable spikes (lunch, dinner). Two defenses: message queues (Topic 38) absorb the burst — orders queue up and workers process at a sustainable rate, so nothing is dropped even if a downstream service is briefly overwhelmed (back-pressure rather than collapse). And the three-sided nature (customers, restaurants, delivery partners) means assignment is a live optimization — you batch nearby orders, predict prep time, and assign partners to minimize wait, all while supply (free riders) is scarce at peak. Acknowledging the three-sided marketplace and spike handling shows maturity beyond "it's just Uber + search."

Interview tip โ€” what they probe

They'll ask: "Why strong consistency for orders but not discovery?" (money vs browsing), "How do you survive dinner rush?" (queues + back-pressure + autoscale), "How is this different from Uber?" (three-sided: restaurant prep time + partner assignment), and "How do you not lose an order if a service crashes mid-flow?" (durable event log / saga). Frame it as consistent order core + async event-driven reactions.

Key Takeaways

  • Food delivery = geospatial discovery + matching + an event-driven order lifecycle.
  • The order core is strongly consistent (money), while reactions (notify, assign, analytics) are async events.
  • Reuse Uber's geo-indexing for nearby restaurants/partners; Elasticsearch for search.
  • Queues absorb meal-time spikes (back-pressure) so nothing is lost; live tracking uses WebSockets + Redis.

#97Design Google Drive / Dropbox (File Storage & Sync)

The goal

Build a cloud file storage and sync service: upload files, store them durably, sync them across all a user's devices, and support sharing.

Step 1: Requirements

  • Functional: upload/download files; sync across devices; share files; version history.
  • Non-functional: durable (never lose files), available, efficient syncing (do not re-upload whole files for small changes), handle large files.

Step 2: The Core Idea — Chunking

Split files into blocks

Files are split into fixed-size chunks (e.g. 4 MB). Each chunk is stored separately in object storage, and a metadata service records which chunks make up each file and in what order. This unlocks the key features below.

Step 3: Why Chunking Is Powerful

  • Efficient sync (delta sync): when you edit a big file, only the changed chunks are uploaded — not the whole file. Huge bandwidth savings.
  • Deduplication: identical chunks (across files or users) are stored once, saving storage.
  • Resumable uploads: if an upload drops, resume from the last chunk.
  • Parallel transfer: upload/download multiple chunks at once for speed.
File
split →
Chunks (4 MB)
→
Object storage
Metadata DBfile → ordered chunk list + versions
Chunks live in object storage; metadata maps files to chunks and versions.

Step 4: Sync & Components

  • Metadata service + DB: tracks files, chunk lists, versions, permissions (source of truth).
  • Block/object storage (S3-like): stores the actual chunks, replicated for durability.
  • Sync service: when a file changes, it notifies the user's other devices (via long polling/WebSockets) to pull the new chunks.
  • Notification + sharing services for collaboration.

Deep Dive: Content-Based Chunking and Hashing

The magic behind delta sync and dedup is hashing each chunk. Each chunk gets a hash (e.g. SHA-256) that acts as its fingerprint and storage key. Dedup: before uploading a chunk, the client asks "do you already have hash X?" — if yes (because you or anyone else stored an identical chunk), it skips the upload entirely. That's how re-uploading a file you already have is instant. Delta sync: when you edit a file, only the chunks whose hashes changed are uploaded; unchanged chunks are reused by reference. A subtlety: fixed-size chunking breaks if you insert a byte near the start (every subsequent chunk shifts and looks "changed"); production systems use content-defined (variable-size) chunking so boundaries stay stable across edits. Mentioning this nuance is a strong signal.

Deep Dive: Conflict Resolution and Sync

The hard part of "sync" is conflicts: you edit a file offline on your laptop while also editing it on your phone — now two divergent versions exist. The metadata service (source of truth) uses version vectors (Topic 67) to detect that the edits were concurrent rather than sequential, then resolves by keeping both as "conflicted copies" (Dropbox's approach) or by last-writer-wins for simple cases. Sync itself is push-based: when the server records a change, it notifies the user's other online devices (long-poll/WebSocket) to pull the new chunks — offline devices reconcile on next connect. This is the consistency/conflict material from Part 8 applied directly.

Interview tip โ€” what they probe

Expect: "How do you avoid re-uploading a whole file for a one-line change?" (chunk hashing + delta sync), "How do you save storage across users?" (dedup by chunk hash), "What if two devices edit offline?" (conflict detection via version vectors, conflicted copies), and "How do other devices learn of changes?" (sync service push). The chunk-hash idea unlocks almost every feature — lead with it.

Key Takeaways

  • Split files into chunks; store chunks in object storage keyed by content hash, map files→chunks in a metadata DB.
  • Chunk hashing enables delta sync, dedup, resumable and parallel transfers (use content-defined chunking).
  • A sync service pushes notifications so other devices pull changed chunks.
  • Handle offline conflicts with version vectors / conflicted copies; metadata is the source of truth.

#98Design a Notification System

The goal

Build a system that sends notifications to users across multiple channels — push (mobile), SMS, email, in-app — reliably and at massive scale.

Step 1: Requirements

  • Functional: send notifications via push/SMS/email/in-app; support templates; respect user preferences (opt-outs); handle priorities.
  • Non-functional: highly scalable (millions of notifications), reliable (do not lose or duplicate), handle third-party provider failures gracefully.

Step 2: The Core Design — Queues + Workers

Decouple sending from generating

Services that want to notify someone just drop a message into a queue (Topic 38) — they do not wait. Dedicated worker services pull from the queue and actually send via the right channel. This decoupling absorbs spikes and lets sending scale independently.

Services
→
Notification queue
→
Channel workers
→
Push / SMS / Email providers
Producers enqueue; workers send via external providers, scaling independently.

Step 3: Key Components

  • Notification service (API): receives requests, validates, applies templates.
  • Preference service: checks if the user has opted in for that channel/type (respect opt-outs and quiet hours).
  • Queues per channel/priority: separate queues so urgent (OTP) is not stuck behind bulk (marketing) — this is a bulkhead idea (Topic 58).
  • Channel workers: integrate with third-party providers (APNs/FCM for push, Twilio for SMS, SES for email).
  • Template service: manages reusable message templates.

Step 4: Reliability Concerns

  • Retries with backoff (Topic 57) when a provider fails; dead-letter queue for messages that keep failing.
  • Idempotency / dedup (Topic 72) so a retry does not send the same SMS twice.
  • Rate limiting to respect provider limits and avoid spamming users.
  • Provider fallback: if one SMS provider is down, switch to a backup.

Deep Dive: At-Least-Once + Idempotency (the delivery dilemma)

The core reliability tension: you must not lose a notification, but you also must not send the same OTP five times. Queues give at-least-once delivery (Topic 38) — safer than at-most-once, but it means duplicates can happen (a worker crashes after sending but before acking, so the message is redelivered). The fix is idempotency (Topic 72): each notification carries a unique ID, and the worker records "sent ID X" so a redelivery is detected and skipped. So the honest design is "at-least-once delivery made safe by idempotent workers" — stating this trade-off explicitly is exactly what's expected.

Deep Dive: Why Separate Queues = Bulkheads

A subtle but important choice: one shared queue is dangerous. If a marketing blast of 10 million emails is queued, an urgent OTP sent right after waits behind all 10 million — useless. Splitting into priority/channel queues (transactional vs bulk, and per channel) is the bulkhead pattern (Topic 58): a flood in one lane can't starve another. You give the transactional queue more workers and strict latency targets, while bulk drains slowly in the background. This is graceful prioritization, not just tidiness.

Interview tip โ€” what they probe

They'll ask: "How do you avoid sending duplicate notifications?" (idempotency keys + dedup store), "How do you stop bulk emails delaying OTPs?" (priority queues / bulkheads), "What if a provider (Twilio/FCM) is down?" (retry with backoff, DLQ, fallback provider), and "How do you respect opt-outs/quiet hours?" (preference service before sending). Frame it as decoupled queue + workers with idempotent, prioritized delivery.

Key Takeaways

  • Core design: queue + channel workers to decouple and scale sending.
  • Delivery is at-least-once made safe by idempotent workers (dedup by notification ID).
  • Use separate priority queues (bulkheads) so urgent OTPs aren't blocked by bulk marketing.
  • Check user preferences; ensure reliability with retries, DLQ, rate limits, and provider fallback.

#99Design a Rate Limiter

The goal

Build a system that limits how many requests a client can make in a time window (e.g. 100 requests/minute), works across many servers, and is fast enough to run on every request.

We covered the concept in Topic 49. Now we design it as a system — especially the tricky part: making it work consistently across a fleet of servers.

Step 1: Requirements

  • Functional: allow up to N requests per window per user/IP/API key; return 429 when exceeded; configurable limits.
  • Non-functional: very low latency (it runs on every request), accurate, and consistent across distributed servers.

Step 2: Pick an Algorithm

From Topic 49: Token Bucket is the popular choice — it allows short bursts and is simple to implement. (Sliding window log/counter gives smoother accuracy if needed.)

Step 3: The Core Challenge — Distributed Counting

Why local counters fail

If each server keeps its own count in memory, a user hitting 5 servers gets 5× their real limit. We need a shared, central counter that all servers read and update.

Use Redis

Store counters in a fast, central store — Redis. It is in-memory (fast), supports atomic increment operations (so concurrent updates are correct), and supports TTL/expiry (perfect for time windows). Each request atomically increments the user's counter in Redis and checks it against the limit.

Any server
→
Redis counteratomic incr + TTL
over limit?
Allow or 429
A shared Redis counter gives a consistent limit across all servers.

Step 4: Where to Run It & Extras

  • Run the rate limiter at the API Gateway (Topic 50) so it protects all services in one place.
  • Return helpful headers (remaining quota, retry-after) with the 429.
  • For extreme performance, a small local cache + periodic sync can reduce Redis calls (trading a little accuracy).

Deep Dive: The Race Condition (why "atomic" matters)

The classic bug: a naive limiter does count = GET key; if count < limit: SET key, count+1. Under concurrency, two requests both read count=99, both think they're under the 100 limit, and both proceed — you've allowed 101. This read-modify-write race is why you must use an atomic operation: Redis INCR (which increments and returns in one step) or a small Lua script that does the check-and-increment atomically on the server. Naming this race and fixing it with atomic ops is precisely the depth interviewers probe.

Deep Dive: Window Algorithms and Their Trade-offs

The algorithm choice has real consequences: Fixed window (count per calendar minute) is simplest but has a boundary burst bug — a user can send 100 requests at 0:59 and 100 more at 1:00, i.e. 200 in two seconds. Sliding window log (store every request timestamp) is perfectly accurate but memory-heavy. Sliding window counter (weighted blend of current + previous window) is the popular middle ground — smooth and cheap. Token bucket allows controlled bursts (good for APIs). Mentioning the fixed-window boundary problem shows you understand why the fancier algorithms exist.

Interview tip โ€” what they probe

Expect: "What if two requests hit at once?" (atomic INCR / Lua — the race), "Fixed vs sliding window?" (boundary burst trade-off), "Where do you enforce it?" (API gateway, central Redis), and "What if Redis is a bottleneck/down?" (local token cache + periodic sync; fail-open vs fail-closed decision). Lead with the distributed-counting problem and the atomicity fix.

Key Takeaways

  • Pick an algorithm: Token Bucket (bursts) or sliding window counter (smooth) are great defaults — fixed window has a boundary-burst bug.
  • The hard part is distributed counting — a central Redis counter with atomic INCR/Lua (avoids the read-modify-write race) + TTL.
  • Run it at the API Gateway to protect everything; return 429 with helpful headers.
  • It must be fast; under extreme load use a local cache + sync, and decide fail-open vs fail-closed if Redis is down.

#100Design Search Autocomplete (Typeahead)

The goal

Build the feature that suggests completions as a user types in a search box (like Google's suggestions) — instantly, and ranked by popularity.

Step 1: Requirements

  • Functional: as the user types a prefix, return the top few most-relevant suggestions; suggestions ranked by popularity/frequency.
  • Non-functional: extremely low latency (must feel instant, < ~100ms), handle huge query volumes, scale to billions of phrases.

Step 2: The Core Data Structure — The Trie

A tree of prefixes

A Trie (prefix tree) stores words by their letters: each path from the root spells a prefix. To autocomplete "ca", you walk to the "ca" node and all words below it are candidates. This makes prefix lookups extremely fast. To rank, store the top-k most popular completions at each node (precomputed), so you return them instantly without searching the whole subtree.

Type "ca"
→
Trie node "ca"precomputed top-k
→
cat, car, cake…
Walk to the prefix node and return its precomputed top suggestions instantly.

Step 3: Serving Fast

  • Keep the Trie in memory (it must be instant) and replicate it across servers.
  • Cache popular prefixes; the most common prefixes serve from cache.
  • Suggestions can also be served from the edge/CDN for popular queries.

Step 4: Updating Popularity (the write path)

  • Real search queries are logged and aggregated (via streams/Kafka, Topic 41) to count how often each phrase is searched.
  • Offline/async jobs periodically rebuild the Trie with updated top-k rankings — you do not update it on every keystroke (that would be too slow).
  • So reads are instant (in-memory Trie), and popularity updates happen in the background.

Deep Dive: The Read/Write Split Is the Whole Trick

The defining insight is separating the blazing-fast read path from the heavy write path. Reads must be <100ms, so they hit an in-memory Trie with precomputed top-k at each node — no computation, just walk to the prefix node and return its stored list. But you can not update popularity counts on every keystroke (billions of events would crush it and rebuilding rankings live is too slow). So the write path is fully offline/async: real searches are logged, streamed to Kafka, aggregated into frequency counts, and a background job periodically rebuilds the Trie (or just the top-k lists). The new Trie is then swapped in. This "precompute offline, serve from memory" pattern is the same idea as the precomputed feed (Topic 25) — recognizing the read/write asymmetry is what they're testing.

Deep Dive: Scaling and Sampling

A full Trie of all queries is huge, so you shard it (e.g. by first letters, so prefixes "a..m" on one set of nodes, "n..z" on another) and replicate each shard for read throughput. You don't need every search counted exactly — sampling a fraction of query logs gives accurate-enough popularity at a fraction of the cost. You also handle freshness (trending terms) by weighting recent queries more, and personalization/typo-tolerance as add-ons. The honest framing: approximate, eventually-updated rankings are totally fine here — nobody needs the suggestion order to be exact.

Interview tip โ€” what they probe

They'll ask: "How do you keep it under 100ms?" (in-memory Trie + precomputed top-k + cache/CDN), "How do you update popularity without slowing reads?" (offline log aggregation, periodic rebuild + swap), "How do you scale the Trie?" (shard by prefix, replicate), and "How fresh must rankings be?" (eventually — sample logs, weight recent). Emphasize the read/write split.

Key Takeaways

  • The core structure is a Trie (prefix tree) with precomputed top-k suggestions at each node.
  • The trick is the read/write split: in-memory Trie for instant reads; popularity updated offline from query logs.
  • Shard the Trie by prefix and replicate for scale; sample logs since rankings can be approximate/eventual.
  • Reads are blazing fast; the heavy lifting (aggregate, rebuild, swap) happens in the background.

#101Design a Distributed Cache

The goal

Build a caching system (like a Redis cluster) that spreads cached data across many machines, so it can hold more than one machine's memory and stay available if a node fails.

One cache server has limited memory and is a single point of failure. A distributed cache spreads data across many nodes — combining several earlier concepts.

Step 1: Requirements

  • Functional: get/set key-value pairs; support eviction (TTL/LRU); scale by adding nodes.
  • Non-functional: very low latency, high availability, scalable, handle node failures gracefully.

Step 2: Spreading Data — Consistent Hashing

The key technique

Use consistent hashing (Topic 66) to decide which node stores each key. Why? Because when you add or remove a cache node, only a small fraction of keys move — instead of remapping everything and wiping the whole cache. Virtual nodes keep the load balanced.

Client
hash(key) →
Hash ring picks node
→
Cache node N
Consistent hashing maps each key to a node with minimal reshuffling on changes.

Step 3: Availability — Replication

  • Replicate each key to one or more other nodes (Topic 26), so if a node dies, a replica still serves the data.
  • A node failure then only causes a brief blip, not data loss across the board.

Step 4: Eviction & Consistency

  • Each node uses an eviction policy like LRU (Topic 33) and TTL to manage its limited memory.
  • Choose a cache strategy (cache-aside is common, Topic 32) for how it syncs with the database.
  • Caches usually favour availability and speed over strict consistency (stale entries expire via TTL).

Deep Dive: Client-Side vs Proxy Routing

A key design question: who knows the hash ring and routes the request? Two models. (1) Smart client: the client library holds the ring and connects directly to the right node — lowest latency (no extra hop), but every client must learn ring changes (this is roughly how Redis Cluster works, with clients following redirects). (2) Proxy layer: clients hit a proxy that knows the ring and forwards (e.g. Twemproxy) — simpler clients, but an extra hop and another thing to scale. Naming this routing trade-off is a sign you've thought beyond "just use consistent hashing."

Deep Dive: The Hard Part Is Consistency, Not Storage

Storing key-values is easy; the genuinely hard problems are the failure and consistency edges. Replication lag: if you replicate for availability, a read from a replica may be slightly stale — usually fine for a cache, but worth stating. Hot keys: one wildly popular key (a celebrity, a viral item) overloads its single node regardless of hashing — you mitigate by replicating that hot key to multiple nodes or adding a small local cache in front. Thundering herd on a cache miss: when a hot key expires, thousands of requests stampede the DB at once — fix with a lock/"single-flight" so only one request recomputes while others wait (ties back to cache stampede, Topic 32). These edge cases are where interviews go deep.

Interview tip โ€” what they probe

Expect: "How do keys map to nodes, and what happens when you add one?" (consistent hashing + virtual nodes, ~1/N moves), "Who routes — client or proxy?" (the trade-off above), "What about a hot key?" (replicate it / local cache), and "Cache miss stampede?" (single-flight lock). It's a great capstone — it stitches together hashing, replication, eviction, and cache strategies.

Key Takeaways

  • A distributed cache spreads data across nodes using consistent hashing + virtual nodes (minimal reshuffle on scaling).
  • Routing is either smart-client (no extra hop) or proxy-based (simpler clients) — a latency vs simplicity trade-off.
  • Replication gives availability (with mild staleness); watch for hot keys and cache-miss stampedes (single-flight lock).
  • Each node uses LRU + TTL eviction; this problem reviews hashing + replication + eviction + cache strategies.

#102Design a Web Crawler

The goal

Build a system that automatically browses the web — starting from some URLs, downloading pages, extracting links, and following them — to collect data (e.g. for a search engine index).

Step 1: Requirements

  • Functional: start from seed URLs; download pages; extract and follow links; store page content.
  • Non-functional: massive scale (billions of pages), be polite (do not overload sites), avoid loops/duplicates, and be fault tolerant.

Step 2: The Core Loop

URL Frontier (queue)
→
Fetch page
→
Parse + extract links
add new URLs →
back to Frontier
A queue of URLs feeds fetchers; new links found are added back to the queue.
  • URL Frontier: a giant queue of URLs to visit (with priorities).
  • Fetchers (workers): download pages in parallel (massively distributed).
  • Parser: extracts content and links from each page.
  • Storage: save page content (object storage) and metadata.

Step 3: The Tricky Parts

  • Avoid duplicates / loops: keep a huge set of "already seen" URLs (often a Bloom filter — a compact structure that answers "have I seen this?" using little memory). Without this, crawlers loop forever.
  • Politeness: respect each site's robots.txt and rate-limit per domain so you do not hammer one server (a per-domain rate limiter, Topic 49).
  • Prioritization: crawl important/fresh pages more often.
  • Distributed & fault tolerant: thousands of workers; if one dies, its URLs are re-queued.
Why a Bloom filter?

Storing billions of seen URLs in full would need enormous memory. A Bloom filter tells you "definitely not seen" or "probably seen" using a tiny fraction of the space — perfect for deduplication at web scale (a small false-positive rate is acceptable).

Deep Dive: The Bloom Filter Trade-off

Why not a normal hash set of seen URLs? At billions of URLs, even storing 64-byte URLs would need terabytes of RAM — infeasible. A Bloom filter answers "have I seen this URL?" using a tiny bit-array and a few hash functions, at maybe ~10 bits per URL. The catch (and the thing to state): it has false positives but never false negatives — it may occasionally say "seen" for a URL you haven't crawled (so you skip a page you shouldn't), but it will never wrongly say "not seen." For a crawler, missing a tiny fraction of pages is an acceptable price for fitting dedup in memory. Knowing this asymmetry (and that it's a probabilistic, space-vs-accuracy trade) is the depth interviewers look for.

Deep Dive: Politeness via the URL Frontier

"Politeness" isn't a footnote — it shapes the architecture. If thousands of workers all hammer one popular domain in parallel, you'll effectively DDoS it and get banned. So the URL Frontier isn't a single queue; it's cleverly structured to enforce per-domain rate limits — e.g. a set of per-domain queues feeding workers such that any one domain is only hit every few seconds, while overall throughput across many domains stays massive. It also handles prioritization (crawl important/fresh pages more often) and respects robots.txt. This makes the frontier the brain of the crawler, not just a to-do list.

Interview tip โ€” what they probe

They'll ask: "How do you dedup billions of URLs cheaply?" (Bloom filter + its false-positive trade-off), "How do you avoid overloading a site?" (per-domain rate limiting in the frontier, robots.txt), "How do you avoid loops?" (seen-set), and "How does it stay fault tolerant at huge scale?" (distributed workers, re-queue on failure). The frontier and the Bloom filter are the two stars.

Key Takeaways

  • Core loop: URL frontier → fetch → parse → add new links, repeated by many distributed workers.
  • Use a Bloom filter to dedup seen URLs cheaply — it has false positives but never false negatives (acceptable trade).
  • The URL frontier enforces per-domain politeness (rate limits, robots.txt) and prioritization — it's the brain.
  • Store pages in object storage; make it distributed and fault tolerant (re-queue a dead worker's URLs).

#103Design a Stock Exchange

The goal

Build a trading system that matches buy and sell orders for stocks in real time — with extremely low latency, perfect correctness, and fairness.

This is one of the most demanding designs because the requirements are unusually strict: microsecond latency and zero tolerance for errors (it is money).

Step 1: Requirements

  • Functional: place/cancel orders (buy/sell at a price); match compatible orders; broadcast price updates; maintain the order book.
  • Non-functional: ultra-low latency (microseconds), strong consistency (no wrong trades), fairness (order priority), high throughput, full audit trail.

Step 2: The Core — The Matching Engine & Order Book

The order book

The heart is the order book: a sorted list of buy orders (bids) and sell orders (asks) for each stock. The matching engine continuously matches the highest bid with the lowest ask when their prices cross, following price-time priority (best price first; ties broken by who arrived first — fairness).

Step 3: Why It Is Different from Other Systems

  • In-memory, single-threaded matching: for determinism and speed, the matching engine often runs the order book in memory on a single thread per stock — no distributed locking, perfectly ordered. Speed beats horizontal sharding of a single book.
  • Strong consistency, not eventual: unlike social feeds, you can never show a wrong or out-of-order trade.
  • Low-level optimization: servers are co-located, networking is tuned to the microsecond. This is the rare case where you optimize hardware and avoid extra hops.

Step 4: Around the Engine

  • Gateway: validates and authenticates incoming orders, then forwards to the engine.
  • Sequencer / event log: every order is written to a durable, ordered log first (for replay and audit) — like Kafka, but ultra-fast.
  • Market data feed: broadcasts price/trade updates to all participants (a pub/sub fan-out, Topic 39).
  • Persistence & reconciliation: async writes of trades to databases for records (the in-memory book stays the fast path).

Deep Dive: Why Single-Threaded Beats Distributed Here

This design inverts the usual instinct ("distribute everything"), which is exactly why it's a great interview topic. For a single stock's order book, distributing it across machines would require coordination/locking on every order to preserve exact ordering and fairness — and that coordination latency would dwarf any parallelism gain. So the matching engine runs the book in memory, on a single thread per stock: deterministic, perfectly ordered, no locks, and brutally fast (microseconds). You still get scale by partitioning across stocks (AAPL on one engine, TSLA on another) — horizontal scaling at the symbol level, sequential processing within a symbol. The principle: when ordering and latency are paramount and a single book fits in memory, sequential in-memory processing wins.

Deep Dive: Durability Without Slowing the Hot Path

How do you keep an in-memory engine both fast and crash-safe? The same way databases do (Topic 21): an append-only event log (sequencer). Every incoming order is written to a durable, ordered log before matching. The in-memory book is just a fast projection of that log — if an engine crashes, you replay the log to rebuild the exact state. This is event sourcing (Topic 40) and gives you durability, a perfect audit trail, and deterministic recovery, all without putting a slow database in the matching hot path. Trades persist to databases asynchronously, off the critical path.

Interview tip โ€” what they probe

Expect: "Why single-threaded / in-memory instead of distributed?" (ordering + latency; partition by symbol), "How is it crash-safe if it's in memory?" (write-ahead event log + replay), "How do you guarantee fairness?" (price-time priority, deterministic sequencing), and "How do prices reach everyone?" (pub/sub fan-out). This is the one design where you argue against distributing the core — make that case clearly.

Key Takeaways

  • The heart is the matching engine on an order book, matching by price-time priority (fairness).
  • It favours strong consistency + ultra-low latency via a single-threaded, in-memory engine per stock; scale by partitioning across symbols.
  • An append-only event log (sequencer) gives durability, deterministic replay recovery, and audit — without a DB in the hot path.
  • Price updates fan out via pub/sub; trades persist asynchronously. (The rare case where you argue against distributing the core.)

#104Design a Payment System

The goal

Build a system that processes payments reliably — charging customers, paying merchants, and keeping every rupee accounted for, with zero tolerance for errors.

Payments are where correctness matters most. The themes here are consistency, idempotency, and auditability — not raw speed.

Step 1: Requirements

  • Functional: accept a payment; integrate with payment providers/banks; track status; handle refunds; record everything.
  • Non-functional: strong consistency (no double charges, no lost money), highly reliable, secure, fully auditable.

Step 2: The Most Important Idea — Idempotency

Never charge twice

Networks fail and clients retry (Topic 57). If a "charge โ‚น500" request is sent twice, the customer must not be charged twice. The fix: every payment request carries a unique idempotency key. The server records it; if it sees the same key again, it returns the original result instead of charging again (Topic 72). This is non-negotiable in payments.

Step 3: Tracking Money with a Ledger

  • Use a double-entry ledger: every transaction records where money came from and went to. Debits and credits must always balance — this makes errors detectable and gives a full audit trail.
  • Payment status moves through clear states: pending → authorized → captured → settled (or failed/refunded). Each transition is recorded.
  • Use ACID transactions (Topic 21) / strong consistency for balance changes — never eventual consistency for money.
Pay request+ idempotency key
→
Payment service
→
Provider / bank
→
Ledger (double-entry)
Idempotent requests flow to a provider; every movement is recorded in a balanced ledger.

Step 4: Reliability Across Services

  • A payment touches multiple services/banks — coordinate with the Saga pattern (Topic 63) and compensating actions (e.g. auto-refund if a later step fails), since 2PC across external banks is impractical.
  • Reconciliation jobs compare your records with the provider's daily to catch any mismatch.
  • Security: encrypt everything, never store raw card numbers (use tokenization / PCI-compliant providers).
  • Async notifications/webhooks handle provider callbacks about final status.

Deep Dive: The Lost-Response Problem (why idempotency is non-negotiable)

Picture the exact failure: the client sends "charge โ‚น500", the bank charges successfully, but the success response is lost on the network. The client times out, assumes failure, and retries. Without protection, the customer is charged twice. The fix: the client generates an idempotency key once and reuses it on every retry. The payment service stores "key X → result"; on seeing key X again, it returns the original result instead of charging again. Crucially, the key must be recorded atomically with the charge (same transaction), or there's a window where you charge but haven't yet saved the key. This is the single most important idea in the whole design — state it first and precisely.

Deep Dive: Why a Double-Entry Ledger, Not Just a Balance Column

It's tempting to store each account's balance as a single number you add/subtract. That's fragile: a bug or partial failure can silently make money appear or vanish with no trace. A double-entry ledger records every movement as paired entries — a debit from one account and an equal credit to another — so the books must always balance (sum of all entries = 0). This makes errors detectable (any imbalance is a red flag), gives an immutable audit trail (you append entries, never edit), and lets you reconstruct any balance by summing its entries. Combined with reconciliation jobs (compare your ledger against the bank's statement daily), it's how real money systems stay provably correct. Mentioning immutability + balancing + reconciliation is exactly the maturity they want.

Interview tip โ€” what they probe

Expect: "What happens if a charge request is retried?" (idempotency key, recorded atomically), "How do you keep money provably correct?" (immutable double-entry ledger + reconciliation), "How do you coordinate across bank/wallet/merchant when 2PC isn't possible?" (saga + compensating refunds, Topic 63), and "How do you handle card data?" (never store it — tokenize via a PCI-compliant provider). Lead with idempotency, then the ledger.

Key Takeaways

  • Payments prioritize correctness: strong consistency, never speed over accuracy.
  • Idempotency keys (recorded atomically with the charge) prevent double charges on retries — the single most important rule.
  • Use an immutable double-entry ledger (must balance) + clear status states + reconciliation for provable auditability.
  • Coordinate multi-step flows with sagas (compensating refunds); never store raw card data — tokenize.

#105How to Approach a System Design Interview

One-line definition

A system design interview is open-ended on purpose. Success is not about knowing one "right answer" — it is about following a clear, structured approach and explaining your trade-offs out loud. This topic gives you that step-by-step framework.

Most candidates fail not from lack of knowledge, but from jumping in randomly. A calm, structured method instantly makes you look senior. Here is a reliable framework you can use for any design question.

The Step-by-Step Framework

  • 1. Clarify requirements (do not skip!): ask questions to define scope. What features matter? Who are the users? What scale? Separate functional (what it does) from non-functional (how well — latency, availability, consistency). Confirm before designing.
  • 2. Estimate scale (back-of-envelope): roughly compute users, requests/sec, storage, bandwidth (Topic 106). This drives your design decisions.
  • 3. Define the API / data model: sketch the main endpoints and core entities. This grounds the discussion.
  • 4. High-level design: draw the big boxes — clients, load balancer, services, databases, cache, queues. Show the request flow. Start simple.
  • 5. Deep dive: the interviewer will pick a component to explore. Go deeper on the interesting part (e.g. the timeline, the matching engine, the sharding scheme).
  • 6. Scale & address bottlenecks: identify the weak points and apply the tools you learned — caching, replication, sharding, CDNs, async processing.
  • 7. Discuss trade-offs & wrap up: summarize choices, mention alternatives, and acknowledge limitations.
Clarify
→
Estimate
→
API + HLD
→
Deep dive + scale
A predictable flow keeps you calm and shows structured thinking.

How to Behave

  • Think out loud — the interviewer is grading your reasoning, not just the final diagram.
  • Drive the conversation, but stay collaborative — check in and adapt to their hints.
  • Justify every choice with a trade-off ("I'd use NoSQL here because we need scale over complex joins").
  • Start simple, then scale — do not over-engineer from minute one.

What's Really Being Tested (it's not the "right answer")

Internalise this and your nerves drop: there is no single correct design. The interviewer is a senior engineer imagining "would I want to build systems with this person?" They're evaluating four things: (1) Structured thinking — do you have a method or do you flail? (2) Communication — can you explain clearly and bring them along? (3) Trade-off reasoning — do you understand why, not just what? (4) Technical depth — can you go deep when pushed? Notice that depth is only one of four. A candidate with average knowledge but great structure and trade-offs often beats a walking encyclopedia who rambles. Every behaviour below maps to one of these four.

A Sample Time Budget (45 minutes)

Time management is itself a graded skill. A rough split: ~5 min clarify requirements + scope, ~5 min estimate scale and define the API/data model, ~10 min high-level design (the big boxes and request flow), ~15–20 min deep dive on the component the interviewer cares about, and ~5 min wrap-up (bottlenecks, trade-offs, what you'd improve). The classic failure is spending 25 minutes perfecting the HLD and never reaching a deep dive — cover breadth first, then go deep. Watch the clock and narrate it ("I'll lock the high-level design and dive into the timeline, since that's the crux").

The mindset shift that wins

Treat the interviewer's hints as gifts, not attacks. If they ask "what happens when this database gets too big?", they're steering you toward the topic they want to explore (sharding) — lean in, don't get defensive. The best sessions feel like two engineers whiteboarding together, not an interrogation.

Key Takeaways

  • Follow a structure: clarify → estimate → API/HLD → deep dive → scale → trade-offs.
  • You're judged on structure, communication, trade-offs, and depth — not on one "right answer".
  • Manage time: breadth first (HLD), then depth; leave room for the deep dive and wrap-up.
  • Always clarify first, think out loud, justify with trade-offs, and treat hints as guidance.

#106Back-of-the-Envelope Calculations

One-line definition

Back-of-the-envelope calculations are quick, rough estimates of scale — requests per second, storage needed, bandwidth — done in your head or on paper. They guide your design decisions and show you think quantitatively.

You do not need exact numbers; you need the right order of magnitude. Is it 100 requests/sec or 1 million? That changes everything. These estimates tell you whether you need caching, sharding, or a CDN.

Numbers Worth Memorizing

  • Time: 1 day ≈ 86,400 seconds (~100,000 for easy math).
  • Powers: 1 thousand = 10ยณ (KB), 1 million = 10โถ (MB), 1 billion = 10โน (GB), 1 trillion = 10ยนยฒ (TB).
  • Storage sizes: a char ≈ 1 byte; a typical tweet ≈ 300 bytes; a photo ≈ a few hundred KB–few MB; a long video ≈ GBs.
  • Latency intuition: memory access is nanoseconds; SSD is microseconds; a network round trip across regions is tens to hundreds of milliseconds.

The Standard Recipe

  • QPS (queries/sec): daily requests ÷ 86,400. Then estimate peak QPS as ~2–3× average.
  • Storage: (new items per day) × (size per item) × (retention in days).
  • Bandwidth: QPS × size per response.
  • Memory for cache: apply the "80/20 rule" — cache the ~20% of data that serves ~80% of requests.
Worked example: a Twitter-like service

Say 200M daily active users, each posting ~2 tweets/day = 400M tweets/day. Writes/sec = 400M ÷ 86,400 ≈ ~4,600 writes/sec (peak ~10K). Storage: 400M × 300 bytes ≈ 120 GB/day → ~44 TB/year. Reads are ~100× writes → ~500K reads/sec. Conclusion: hugely read-heavy → we definitely need heavy caching and sharding. See how the numbers directly justify the design?

Tips

  • Round aggressively — use 100,000 sec/day, powers of 10. Precision is not the point.
  • State your assumptions out loud ("assuming 1M daily users…").
  • Connect numbers to decisions — "500K reads/sec means a single DB can't handle it, so we cache and shard."

The Latency Numbers Every Engineer Should Know

Beyond throughput math, the famous "latency numbers" build the intuition for why we cache, batch, and use CDNs. Rough orders of magnitude: reading from memory โ‰ˆ 100 nanoseconds; a fast SSD read โ‰ˆ 100 microseconds (~1,000× slower than memory); a network round-trip within a data center โ‰ˆ 0.5 milliseconds; and a round-trip across continents โ‰ˆ 100+ milliseconds. The punchlines that justify whole architectures: memory is ~1,000× faster than disk (so cache in RAM), and a cross-region hop is ~100ms (so put a CDN near users and avoid chatty cross-region calls). When you say "I'll cache this", you can back it with "because RAM is three orders of magnitude faster than disk."

Why These Estimates Drive the Whole Design

The point of the math isn't the number — it's the decision the number forces. "~40 writes/sec" says a single modest database is fine; "~500K reads/sec" screams caching + read replicas + sharding because no single node serves that. "120 GB/day → 44 TB/year" says you'll outgrow one machine → plan to shard. "Peak is 3× average" sizes your autoscaling headroom. Always close the loop out loud: number → implication → design choice. That linkage is what makes estimation impressive rather than busywork.

Common pitfalls

Forgetting peak vs average (designing for the average then falling over at the dinner rush), ignoring the read:write ratio (it usually dictates caching/replication), and getting lost in precision (arguing 86,400 vs 100,000 wastes time — round and move on). Estimate to inform the design, not to be exact.

Key Takeaways

  • Estimate QPS, storage, and bandwidth to size your system — order of magnitude is enough.
  • Memorize basics: ~86,400 sec/day, powers of ten, rough item sizes, and the latency ladder (RAM โ‰ช SSD โ‰ช network โ‰ช cross-region).
  • Compute peak (~2–3× average) and apply the 80/20 rule for caching.
  • Always close the loop: number → implication → design choice (caching, sharding, CDN).

#107Common Interview Mistakes (and How to Avoid Them)

One-line definition

This final topic lists the most common mistakes that sink system design interviews — so you can consciously avoid every one of them. Knowing the traps is half the battle.

You now have the knowledge and the framework. The last step is to avoid the classic blunders. Most rejections come from these, not from missing some obscure fact.

The Top Mistakes

  • 1. Jumping into the solution without clarifying. Designing before understanding the requirements is the #1 mistake. Fix: always clarify scope and requirements first (Topic 105).
  • 2. Not asking about scale. A design for 1,000 users is totally different from one for 1 billion. Fix: estimate scale early (Topic 106).
  • 3. Over-engineering from the start. Proposing microservices, Kafka, and sharding for a tiny app shows poor judgement. Fix: start simple, scale only when needed.
  • 4. Staying silent. If you think without speaking, the interviewer cannot follow your reasoning. Fix: think out loud constantly.
  • 5. Not justifying decisions / ignoring trade-offs. Saying "I'll use NoSQL" without why is weak. Fix: explain the trade-off behind every choice.
  • 6. Ignoring non-functional requirements. Forgetting availability, latency, consistency. Fix: explicitly address them.
  • 7. No bottleneck analysis. Drawing boxes but never asking "what breaks first under load?". Fix: identify and address bottlenecks.
  • 8. Being rigid / defensive. Dismissing the interviewer's hints. Fix: treat hints as gifts and adapt collaboratively.
  • 9. Poor time management. Spending 20 minutes on one tiny detail. Fix: keep moving; cover breadth before depth.
  • 10. Saying "it depends" and stopping. Fix: say "it depends" and then explain on what, and make a reasoned choice.
The mindset that wins

There is no single perfect answer. Interviewers want to see structured thinking, clear communication, and sound trade-offs. Be the calm engineer who clarifies, reasons aloud, starts simple, scales deliberately, and explains every "why". Do that, and you will stand out.

The Two Mistakes That Sink the Most Candidates

If you fix only two things, fix these. First, jumping straight into solutioning. The moment you hear "design Twitter", the urge is to draw boxes — resist it. Spend the first few minutes clarifying scope and scale. Designing the wrong thing brilliantly still fails. Second, over-engineering. Reaching for Kafka, microservices, and global sharding for a system with 10,000 users signals poor judgment, not seniority. Seniority is shown by starting with the simplest thing that works and adding complexity only when a concrete bottleneck demands it. "I'd start with a single database and a cache; we'd shard only once we cross ~X writes/sec" is a far stronger statement than immediately proposing a distributed everything.

Turn Each Mistake into a Habit

The fix for all ten is to make the good behaviours automatic: open with clarifying questions (kills #1, #2, #6), narrate constantly (kills #4, #5), do quick estimation (kills #2, #7), and always pair a choice with its trade-off and alternative (kills #5, #10). Practise the framework on 3–4 designs until the flow — clarify, estimate, API, HLD, deep dive, scale, trade-offs — runs on autopilot. When the structure is muscle memory, your whole brain is free to reason about the actual problem, and you'll naturally avoid the traps under pressure.

Final interview tip

End strong: in the last few minutes, proactively summarize your design, name its bottlenecks, and say what you'd improve with more time ("with more time I'd add a read-replica strategy and discuss the cache-invalidation edge cases"). Self-critique demonstrates senior judgment and leaves the interviewer with the impression of a thoughtful, honest engineer — exactly who they want to hire.

Key Takeaways

  • Most failures come from process mistakes, not missing facts.
  • The two biggest: jumping in without clarifying and over-engineering — start simple, scale deliberately.
  • Always: clarify first, estimate scale, think out loud, justify with trade-offs; make the framework muscle memory.
  • End by summarizing, naming bottlenecks, and self-critiquing — it signals senior judgment.
๐ŸŽ‰ You finished all 107 topics!

From "what is system design?" all the way to designing payment systems and acing interviews — you have covered the complete journey, zero to advanced. Revisit any topic from the sidebar, replay the interactive demos, and you will have rock-solid, doubt-free foundations. All the best! ๐Ÿš€