Introduction to System Design
What Is System Design?
System design is the process of defining architecture, components, modules, interfaces, and data for a software system to satisfy specified requirements. It sits between product requirements and implementation, translating user needs into scalable, reliable, and maintainable technical structures. In interviews and real projects, you explain how millions of users can read feeds, upload media, or place orders without collapsing a single server.
A useful mindset treats the system as a graph of services, storage, caches, queues, and clients connected by networks with latency, failure, and cost. You reason about read-heavy vs write-heavy workloads, hot keys, thundering herds, and regional outages. Good designs are boring in the right places: clear ownership boundaries, well-defined APIs, and observability everywhere.
Functional vs Non-Functional Requirements
Functional requirements describe what the system must do: users can post tweets, search products, or transfer money. Non-functional requirements describe how well it must do it: latency targets, availability, durability, security, compliance, and cost. NFRs drive architecture more than feature lists because they determine replication, caching, and partitioning strategies.
Always clarify scope before drawing boxes. Ask daily active users, read/write ratio, maximum payload size, retention period, and consistency expectations. A chat app needs low latency and ordering guarantees; a reporting dashboard tolerates minutes of staleness but must handle large scans efficiently.
- Latency: p50, p95, and p99 targets for critical paths.
- Availability: acceptable downtime per month (e.g., 99.9%).
- Durability: can you lose data? For how long?
- Scalability: growth over 1–3 years.
High-Level Building Blocks
Most large systems combine clients, load balancers, application servers, caches, primary databases, search indexes, object storage, message queues, and CDNs. Clients talk to edge layers first; static assets are served from CDNs close to users. Dynamic requests hit stateless app tiers that read through caches and fall back to databases.
Stateless application servers scale horizontally behind load balancers. Stateful data lives in specialized stores chosen for access patterns: relational databases for transactions, document stores for flexible schemas, key-value stores for sessions, column stores for analytics, and graph databases for relationships.
The Design Process
Start with requirements and back-of-the-envelope estimates. Sketch a simple diagram with main components and data flow. Identify bottlenecks and scale them independently. Discuss failure modes and monitoring. Iterate by deepening the riskiest component—often the database or hot partition.
In interviews, narrate your thinking aloud. State assumptions, propose a baseline, then improve it. Interviewers reward clarity, trade-off analysis, and pragmatic choices over exotic technologies.
Common Pitfalls for Beginners
Junior designs often over-engineer microservices on day one or under-estimate data growth. Another mistake is ignoring the read path: a write-optimized schema may collapse when feeds or dashboards scan huge tables. Always ask who reads the data, how often, and with what filters.
Another pitfall is skipping failure analysis. Single-region deployments, missing backups, and no rate limits are acceptable only when explicitly scoped. Production traffic is spiky; exams, sales, and viral posts appear without warning.
- Do not cache without a defined invalidation story.
- Do not shard before exhausting simpler scale options.
- Do not hide complexity inside synchronous call chains.
Design Trade-offs in Practice
Every production system is a bundle of trade-offs rather than a perfect architecture diagram. Teams choose synchronous APIs when consistency and simplicity matter, and asynchronous pipelines when throughput and decoupling dominate. The same company may use strong consistency for billing and eventual consistency for analytics because the business cost of failure differs. Interview answers become stronger when you name the trade-off explicitly and justify it with user impact, operational cost, and recovery time.
Operational maturity also changes the right answer. A startup may accept a single database and manual failover because engineering time is scarce, while a mature product invests in multi-region replication and automated runbooks. Document assumptions about traffic shape, data growth, and compliance early so design decisions remain valid six months later.
- Latency vs throughput: optimize the metric users feel first.
- Consistency vs availability: use CAP reasoning with real failure scenarios.
- Build vs buy: managed services reduce toil but add vendor dependency.
- Cost vs resilience: duplicate components only where outages hurt revenue.
Reliability Checklist
Reliable systems define SLOs, measure them continuously, and alert on burn rate rather than single spikes. Retries must be idempotent, timeouts must be explicit, and cascading failures must be blocked with bulkheads and circuit breakers. Backups and disaster recovery drills are part of architecture, not an afterthought.
Security belongs in system design: authenticate callers, authorize actions, encrypt data in transit and at rest, and audit sensitive operations. Rate limiting protects shared components from abuse and accidental hot keys.
- Set timeouts on every outbound network call.
- Use health checks that reflect real dependencies.
- Prefer graceful degradation over total outage.
- Run game days to validate failover assumptions.
Chapter Summary
System design is iterative. Start with requirements, sketch a high-level diagram, estimate capacity, identify bottlenecks, add scaling layers, and revisit consistency and failure modes. The goal is not memorizing buzzwords but explaining how data flows, where state lives, and what breaks first under load.
Practice explaining your design to a teammate who will challenge assumptions. The best architects combine quantitative estimates with operational empathy: deploys, on-call pain, and customer trust.
Monitoring and Observability
Metrics expose RED/USE signals: rate, errors, duration for services; utilization, saturation, errors for resources. Distributed tracing follows requests across hops. Structured logs with correlation IDs tie user reports to failing spans. Dashboards without alerts are vanity; alerts without runbooks waste on-call energy.
Define golden signals per service: latency, traffic, errors, saturation. SLO dashboards show budget remaining. Post-incident reviews update architecture docs and action items—not blame slides.
- Instrument the critical user journey end to end.
- Alert on symptom-based SLO burn, not every CPU blip.
- Store logs with retention matched to compliance needs.
Cost and Capacity Planning
Cloud bills scale with egress, stored bytes, and over-provisioned idle instances. Reserved capacity and autoscaling save money when traffic is predictable. Archive cold data to cheaper tiers. Right-size databases before multiplying replicas.
Cost-aware design chooses managed services when staff time is more expensive than markup. Batch and compress data movement across regions.
Security and Compliance in Architecture
Threat modeling STRIDE identifies spoofing, tampering, repudiation, information disclosure, denial of service, and elevation of privilege. Segment networks, least-privilege IAM, secrets rotation, and audit trails are baseline—not optional extras.
Data residency and GDPR/CCPA may force regional storage and deletion workflows. Bake retention and export APIs into design early.
When two designs both work, prefer the one with fewer moving parts and clearer ownership. Operational simplicity reduces mean time to recovery and makes onboarding faster for new engineers joining the team.
Capacity planning should include failure capacity: if one availability zone disappears, remaining zones must absorb traffic without violating SLOs. Load tests should simulate dependency timeouts, not only happy paths.
Data models outlive services. Investing time in entity boundaries, id formats, and migration strategy pays off across years of feature work and acquisitions.
Every external dependency needs a fallback story: degrade features, serve cached snapshots, or queue work for later. Silent hangs hurt NPS more than explicit error messages with retry guidance.
Platform teams succeed when they offer paved roads—templates, CI/CD, observability baselines—so product squads do not reinvent shards and caches per feature.
Document ADRs (Architecture Decision Records) when you choose Kafka over RabbitMQ or SQL over NoSQL. Future you will forget the constraints that justified the choice.
Geographic expansion introduces replication lag, legal boundaries, and clock skew. Design APIs idempotently and use version vectors or timestamps consciously.
Performance testing in staging must use anonymized production-shaped data volumes. Empty tables lie about index selectivity and planner behavior.
Rollouts should be progressive: canary, region-by-region, feature flags. Instant global deploys maximize blast radius of subtle bugs.
On-call health improves when architectures expose clear health endpoints per dependency and automate rollbacks when error budgets burn too fast during a release.
When two designs both work, prefer the one with fewer moving parts and clearer ownership. Operational simplicity reduces mean time to recovery and makes onboarding faster for new engineers joining the team.
Capacity planning should include failure capacity: if one availability zone disappears, remaining zones must absorb traffic without violating SLOs. Load tests should simulate dependency timeouts, not only happy paths.
Data models outlive services. Investing time in entity boundaries, id formats, and migration strategy pays off across years of feature work and acquisitions.
Every external dependency needs a fallback story: degrade features, serve cached snapshots, or queue work for later. Silent hangs hurt NPS more than explicit error messages with retry guidance.
Platform teams succeed when they offer paved roads—templates, CI/CD, observability baselines—so product squads do not reinvent shards and caches per feature.
Document ADRs (Architecture Decision Records) when you choose Kafka over RabbitMQ or SQL over NoSQL. Future you will forget the constraints that justified the choice.
Geographic expansion introduces replication lag, legal boundaries, and clock skew. Design APIs idempotently and use version vectors or timestamps consciously.
Performance testing in staging must use anonymized production-shaped data volumes. Empty tables lie about index selectivity and planner behavior.
Rollouts should be progressive: canary, region-by-region, feature flags. Instant global deploys maximize blast radius of subtle bugs.
Chapter Practice Test
10 questions — answer all and submit to see your score.