From a74a8966711a9f7466c336922eb0007e8fb80932 Mon Sep 17 00:00:00 2001 From: Rakesh Deka <58238022+rakeshdeka@users.noreply.github.com> Date: Wed, 25 Jan 2023 11:13:22 +0530 Subject: [PATCH] update REDME.md - Added External link to the page so that we can easily navigate through the topics - Added label to the source. --- README.md | 247 +++++++++++++++++++++++++++--------------------------- 1 file changed, 124 insertions(+), 123 deletions(-) diff --git a/README.md b/README.md index 7471144..3f9daf2 100644 --- a/README.md +++ b/README.md @@ -3,358 +3,359 @@ These are the best resources for System Design on the Internet. ## ## Video Processing -Transcoding Videos at Scale: https://www.egnyte.com/blog/2018/12/transcoding-how-we-serve-videos-at-scale/ +[Transcoding Videos at Scale](https://www.egnyte.com/blog/2018/12/transcoding-how-we-serve-videos-at-scale/) `Egnyte` -Facebook Video Broadcasting: https://engineering.fb.com/ios/under-the-hood-broadcasting-live-video-to-millions/ +[Facebook Video Broadcasting](https://engineering.fb.com/ios/under-the-hood-broadcasting-live-video-to-millions/) `Meta` -Netflix Video Encoding at Scale: https://netflixtechblog.com/high-quality-video-encoding-at-scale-d159db052746 +[Netflix Video Encoding at Scale](https://netflixtechblog.com/high-quality-video-encoding-at-scale-d159db052746) `Netflix` -Netflix Shot based encoding: https://netflixtechblog.com/optimized-shot-based-encodes-now-streaming-4b9464204830 +[Netflix Shot based encoding](https://netflixtechblog.com/optimized-shot-based-encodes-now-streaming-4b9464204830) `Netflix` ## ## Cluster and Workflow Management -Facebook Cluster Management: https://engineering.fb.com/data-center-engineering/twine/ +[Facebook Cluster Management](https://engineering.fb.com/data-center-engineering/twine/) `Meta` -Google Autopilot - Autoscaling: https://dl.acm.org/doi/pdf/10.1145/3342195.3387524 +[Google Autopilot - Autoscaling](https://dl.acm.org/doi/pdf/10.1145/3342195.3387524) `Google` -Netflix Workflow Orchestration: https://netflix.github.io/conductor/ +[Netflix Workflow Orchestration](https://netflix.github.io/conductor/) `Netflix` -Opensource Workflow Management: https://github.com/spotify/luigi +[Opensource Workflow Management](https://github.com/spotify/luigi) `Spotify` -Meta Hardware Management: https://engineering.fb.com/2020/12/09/data-center-engineering/how-facebook-keeps-its-large-scale-infrastructure-hardware-up-and-running/ +[Meta Hardware Management](https://engineering.fb.com/2020/12/09/data-center-engineering/how-facebook-keeps-its-large-scale-infrastructure-hardware-up-and-running/) `Meta` -Meta Capacity Assignment: https://engineering.fb.com/2022/09/06/data-center-engineering/viewing-the-world-as-a-computer-global-capacity-management/ +[Meta Capacity Assignment](https://engineering.fb.com/2022/09/06/data-center-engineering/viewing-the-world-as-a-computer-global-capacity-management/) `Meta` -Amazon EC2: https://www.allthingsdistributed.com/2015/07/under-the-hood-of-the-amazon-ec2-container-service.html +[Amazon EC2](https://www.allthingsdistributed.com/2015/07/under-the-hood-of-the-amazon-ec2-container-service.html) `allthingsdistributed.com` ## ## Intra-Service Messaging -What is a message queue: https://www.cloudamqp.com/blog/what-is-message-queuing.html +[What is a message queue](https://www.cloudamqp.com/blog/what-is-message-queuing.html) `cloudamqp` -AirBnb Idempotency: https://medium.com/airbnb-engineering/avoiding-double-payments-in-a-distributed-payments-system-2981f6b070bb +[AirBnb Idempotency](https://medium.com/airbnb-engineering/avoiding-double-payments-in-a-distributed-payments-system-2981f6b070bb) `Airbnb` -Nginx Service Mesh: https://www.nginx.com/learn/service-mesh/ +[Nginx Service Mesh](https://www.nginx.com/learn/service-mesh/) `Nginx` ## ## Message Queue Antipattern -DB as queue Antipattern: http://blog.codepath.com/2012/11/15/asynchronous-processing-in-web-applications-part-1-a-database-is-not-a-queue/ +[DB as queue Antipattern](http://blog.codepath.com/2012/11/15/asynchronous-processing-in-web-applications-part-1-a-database-is-not-a-queue/) `codepath` -Using a database as a message queue: https://softwareengineering.stackexchange.com/questions/231410/why-database-as-queue-so-bad +[Using a database as a message queue](https://softwareengineering.stackexchange.com/questions/231410/why-database-as-queue-so-bad) `stackexchange` -Anti-pattern of DB as a queue: http://mikehadlow.blogspot.com/2012/04/database-as-queue-anti-pattern.html +[Anti-pattern of DB as a queue](http://mikehadlow.blogspot.com/2012/04/database-as-queue-anti-pattern.html) `Mikehadlow` -Drawbacks of DB as a queue: https://www.cloudamqp.com/blog/why-is-a-database-not-the-right-tool-for-a-queue-based-system.html +[Drawbacks of DB as a queue](https://www.cloudamqp.com/blog/why-is-a-database-not-the-right-tool-for-a-queue-based-system.html) `cloudamqp` ## ## Service Mesh -Kubernetes Service Mesh: https://akomljen.com/kubernetes-service-mesh/ +[Kubernetes Service Mesh](https://akomljen.com/kubernetes-service-mesh/) `akomljen` -Kubernetes Sidecar:https://www.weave.works/blog/introduction-to-service-meshes-on-kubernetes-and-progressive-delivery +[Kubernetes Sidecar](https://www.weave.works/blog/introduction-to-service-meshes-on-kubernetes-and-progressive-delivery) `weaveworks` -Service Mesh: https://www.weave.works/blog/introduction-to-service-meshes-on-kubernetes-and-progressive-delivery +[Service Mesh](https://www.weave.works/blog/introduction-to-service-meshes-on-kubernetes-and-progressive-delivery) `weaveworks` -NginX Service Mesh: https://www.nginx.com/learn/service-mesh/ +[NginX Service Mesh](https://www.nginx.com/learn/service-mesh/) `Nginx` ## ## Practical System Design -Facebook Messenger Optimisations: https://spectrum.ieee.org/how-facebooks-software-engineers-prepare-messenger-for-new-years-eve +[Facebook Messenger Optimisations](https://spectrum.ieee.org/how-facebooks-software-engineers-prepare-messenger-for-new-years-eve) `IEEE Spectrum` -YouTube Architecture: http://highscalability.com/youtube-architecture +[YouTube Architecture](http://highscalability.com/youtube-architecture) `Highscalability` -YouTube scalability 2012: https://www.youtube.com/watch?v=w5WVu624fY8 +[YouTube scalability 2012](https://www.youtube.com/watch?v=w5WVu624fY8) `Google Talks YT` -Distributed Design Patterns: http://horicky.blogspot.com/2010/10/scalable-system-design-patterns.html +[Distributed Design Patterns](http://horicky.blogspot.com/2010/10/scalable-system-design-patterns.html) `Horicky` -Monolith to Microservice: https://martinfowler.com/articles/break-monolith-into-microservices.html +[Monolith to Microservice](https://martinfowler.com/articles/break-monolith-into-microservices.html) `Martin Fowler` -Zerodha Tech Stack: https://zerodha.tech/blog/hello-world/ +[Zerodha Tech Stack](https://zerodha.tech/blog/hello-world/) `Zerodha` ## ## Distributed File System -Open Source Distributed File System: https://docs.ceph.com/en/latest/architecture/ +[Open Source Distributed File System](https://docs.ceph.com/en/latest/architecture/) `ceph` -Amazon S3 Performance hacks: https://aws.amazon.com/blogs/aws/amazon-s3-performance-tips-tricks-seattle-hiring-event/ +[Amazon S3 Performance hacks](https://aws.amazon.com/blogs/aws/amazon-s3-performance-tips-tricks-seattle-hiring-event/) `AWS` -Amazon S3 object expiration: https://aws.amazon.com/blogs/aws/amazon-s3-object-expiration/ +[Amazon S3 object expiration](https://aws.amazon.com/blogs/aws/amazon-s3-object-expiration/) `AWS` ## ## Time Series Databases -Pintrest Time Series Database: https://medium.com/pinterest-engineering/goku-building-a-scalable-and-high-performant-time-series-database-system-a8ff5758a181 +[Pintrest Time Series Database](https://medium.com/pinterest-engineering/goku-building-a-scalable-and-high-performant-time-series-database-system-a8ff5758a181) `Pinterest` -Uber Time Series DB: https://eng.uber.com/aresdb/ +[Uber Time Series DB](https://eng.uber.com/aresdb/) `Uber` -TimeSeries Relational DB: https://blog.timescale.com/blog/time-series-data-why-and-how-to-use-a-relational-database-instead-of-nosql-d0cd6975e87c/ +[TimeSeries Relational DB](https://blog.timescale.com/blog/time-series-data-why-and-how-to-use-a-relational-database-instead-of-nosql-d0cd6975e87c/) `Timescale` -Facebook Gorilla Time Series DB: http://www.vldb.org/pvldb/vol8/p1816-teller.pdf +[Facebook Gorilla Time Series DB](http://www.vldb.org/pvldb/vol8/p1816-teller.pdf) `VLDB` ## ## Rate Limiting -Circuit Breaker Algorithm: https://martinfowler.com/bliki/CircuitBreaker.html +[Circuit Breaker Algorithm](https://martinfowler.com/bliki/CircuitBreaker.html) `Martin Fowler` -Uber Rate Limiter: https://github.com/uber-go/ratelimit/blob/master/ratelimit.go +[Uber Rate Limiter](https://github.com/uber-go/ratelimit/blob/master/ratelimit.go) `Uber Go` ## ## In Memory Database - Redis -Redis Official Documentation : https://redis.com/ +[Redis Official Documentation](https://redis.com/) `Redis` -Learn Redis through Redis University : https://university.redis.com/ +[Learn Redis through Redis University](https://university.redis.com/) `Redis` -Redis Open Source Repo : https://github.com/redis/redis +[Redis Open Source Repo](https://github.com/redis/redis) `Redis` -Redis Architecture : https://medium.com/opstree-technology/redis-cluster-architecture-replication-sharding-and-failover-86871e783ac0 +[Redis Architecture](https://medium.com/opstree-technology/redis-cluster-architecture-replication-sharding-and-failover-86871e783ac0) `Sajal Jain` ## ## Network Protocols -What is HTTP: https://engineering.cred.club/head-of-line-hol-blocking-in-http-1-and-http-2-50b24e9e3372 +[What is HTTP](https://engineering.cred.club/head-of-line-hol-blocking-in-http-1-and-http-2-50b24e9e3372) `CRED` -QUIC Protocol: https://www.akamai.com/blog/performance/http3-and-quic-past-present-and-future +[QUIC Protocol](https://www.akamai.com/blog/performance/http3-and-quic-past-present-and-future) `akamai` -TCP Protocol algorithms: https://ee.lbl.gov/papers/congavoid.pdf (First 10 pages are important) +[TCP Protocol algorithms(First 10 pages are important)]( https://ee.lbl.gov/papers/congavoid.pdf) -WebRTC: https://webrtc.github.io/webrtc-org/blog/2012/07/23/a-great-introduction-to-webrtc.html +[WebRTC](https://webrtc.github.io/webrtc-org/blog/2012/07/23/a-great-introduction-to-webrtc.html) `webrtc` -WebSockets: https://datatracker.ietf.org/doc/html/rfc6455#section-1.2 +[WebSockets](https://datatracker.ietf.org/doc/html/rfc6455#section-1.2) `datatracker.itef.org` -Dynamic Source Routing using QUIC: https://fb.watch/fSEbI4KHlA/ +[Dynamic Source Routing using QUIC](https://fb.watch/fSEbI4KHlA/) `At Scale` ## ## Chess Engine Design -Chess Engine Building: https://www.youtube.com/watch?v=U4ogK0MIzqk +[Chess Engine Building](https://www.youtube.com/watch?v=U4ogK0MIzqk) `Sebastian Lague + YT` ## ## Subscription Management System -Subscription Manager: https://netflixtechblog.com/building-a-rule-based-platform-to-manage-netflix-membership-skus-at-scale-e3c0f82aa7bc +[Subscription Manager](https://netflixtechblog.com/building-a-rule-based-platform-to-manage-netflix-membership-skus-at-scale-e3c0f82aa7bc) `Netflix` ## ## Google Docs -Operational Transform: http://www.codecommit.com/blog/java/understanding-and-applying-operational-transformation +[Operational Transform](http://www.codecommit.com/blog/java/understanding-and-applying-operational-transformation) `codecommit` -Google Docs: https://www.youtube.com/watch?v=uOFzWZrsPV0&list=PLXDe3d8o9VFtydBV5biyz9iS3WqKsBMD5&index=3 +[Google Docs](https://www.youtube.com/watch?v=uOFzWZrsPV0&list=PLXDe3d8o9VFtydBV5biyz9iS3WqKsBMD5&index=3) `Google I/O 2009` ## ## API Design -API Design: https://medium.com/airbnb-engineering/building-services-at-airbnb-part-1-c4c1d8fa811b +[API Design](https://medium.com/airbnb-engineering/building-services-at-airbnb-part-1-c4c1d8fa811b) `Airbnb` -Swagger APIs: https://swagger.io/docs/specification/about/ +[Swagger APIs](https://swagger.io/docs/specification/about/) `swagger.io` ## ## NoSQL Database Internals -Cassandra Architecture: https://docs.datastax.com/en/archived/cassandra/3.0/cassandra/architecture/archIntro.html +[Cassandra Architecture](https://docs.datastax.com/en/archived/cassandra/3.0/cassandra/architecture/archIntro.html) `datatax` -Google BigTable Architecture: https://static.googleusercontent.com/media/research.google.com/en//archive/bigtable-osdi06.pdf +[Google BigTable Architecture](https://static.googleusercontent.com/media/research.google.com/en//archive/bigtable-osdi06.pdf) `Google` -Amazon Dynamo DB Internals: https://www.allthingsdistributed.com/2007/10/amazons_dynamo.html +[Amazon Dynamo DB Internals](https://www.allthingsdistributed.com/2007/10/amazons_dynamo.html) `allthingsdistributed.com` -Design Patterns in Amazon Dynamo DB: https://www.youtube.com/watch?v=HaEPXoXVf2k +[Design Patterns in Amazon Dynamo DB](https://www.youtube.com/watch?v=HaEPXoXVf2k) `AWS YT` -Internals of Amazon Dynamo DB: https://www.youtube.com/watch?v=yvBR71D0nAQ +[Internals of Amazon Dynamo DB](https://www.youtube.com/watch?v=yvBR71D0nAQ) `AWS YT` ## ## NoSQL Database Algorithms -Hyperloglog Algorithm: https://odino.org/my-favorite-data-structure-hyperloglog/ +[Hyperloglog Algorithm](https://odino.org/my-favorite-data-structure-hyperloglog/) `Odino` -Log Structured Merge Tree: https://www.cs.umb.edu/~poneil/lsmtree.pdf +[Log Structured Merge Tree](https://www.cs.umb.edu/~poneil/lsmtree.pdf) `University of Massachusetts Boston` -Sorted String Tables and Compaction Strategies: https://github.com/scylladb/scylla/wiki/SSTable-compaction-and-compaction-strategies +[Sorted String Tables and Compaction Strategies](https://github.com/scylladb/scylla/wiki/SSTable-compaction-and-compaction-strategies) `ScyllaDB` -Leveled Compaction Cassandra: https://www.datastax.com/blog/leveled-compaction-apache-cassandra +[Leveled Compaction Cassandra](https://www.datastax.com/blog/leveled-compaction-apache-cassandra) `datastax` -Scylla DB Compaction: https://github.com/scylladb/scylla/wiki/SSTable-compaction-and-compaction-strategies +[Scylla DB Compaction](https://github.com/scylladb/scylla/wiki/SSTable-compaction-and-compaction-strategies) `ScyllaDB` -Indexing in Cassandra: https://www.bmc.com/blogs/cassandra-clustering-columns-partition-composite-key/ +[Indexing in Cassandra](https://www.bmc.com/blogs/cassandra-clustering-columns-partition-composite-key/) `BMC` ## ## Database Replication -Database replication: https://dev.mysql.com/doc/refman/8.0/en/replication.html +[Database replication](https://dev.mysql.com/doc/refman/8.0/en/replication.html) `dev.mysql` -Netflix Data replication - Change Data Capture: https://netflixtechblog.com/dblog-a-generic-change-data-capture-framework-69351fb9099b +[Netflix Data replication - Change Data Capture](https://netflixtechblog.com/dblog-a-generic-change-data-capture-framework-69351fb9099b) `Netflix` -LinkedIn Logging Usecases: https://engineering.linkedin.com/distributed-systems/log-what-every-software-engineer-should-know-about-real-time-datas-unifying +[LinkedIn Logging Usecases](https://engineering.linkedin.com/distributed-systems/log-what-every-software-engineer-should-know-about-real-time-datas-unifying) `LinkedIn` ## ## Containers and Docker -Facebook Twine Containerization: https://engineering.fb.com/developer-tools/zookeeper-twine/ +[Facebook Twine Containerization](https://engineering.fb.com/developer-tools/zookeeper-twine/) `Meta` -CloudFlare Containerization: https://blog.cloudflare.com/cloud-computing-without-containers/ +[CloudFlare Containerization](https://blog.cloudflare.com/cloud-computing-without-containers/) `Cloudflare` -Docker Architecture: https://docs.docker.com/get-started/overview/#docker-architecture +[Docker Architecture](https://docs.docker.com/get-started/overview/#docker-architecture) `Docker` ## ## Capacity Estimation -Google Capacity Estimation: https://www.youtube.com/watch?v=modXC5IWTJI +[Google Capacity Estimation](https://www.youtube.com/watch?v=modXC5IWTJI) `Stanford` -Scalability at YouTube 2012: https://www.youtube.com/watch?v=G-lGCC4KKok +[Scalability at YouTube 2012](https://www.youtube.com/watch?v=G-lGCC4KKok) `Youtube` -Back of envelope Calculations at AWS: https://www.youtube.com/watch?v=-3qetLv2Yp0 +[Back of envelope Calculations at AWS](https://www.youtube.com/watch?v=-3qetLv2Yp0) `Hasgeek TV` -Capacity Estimation: http://static.googleusercontent.com/media/research.google.com/en//people/jeff/stanford-295-talk.pdf +[Capacity Estimation](http://static.googleusercontent.com/media/research.google.com/en//people/jeff/stanford-295-talk.pdf) `Google` ## ## Publisher Subscriber -Oracle Publisher Subscriber: https://docs.oracle.com/cd/B10501_01/appdev.920/a96590/adg15pub.htm +[Oracle Publisher Subscriber](https://docs.oracle.com/cd/B10501_01/appdev.920/a96590/adg15pub.htm) `Oracle` -Amazon Pub Sub Messaging: https://aws.amazon.com/pub-sub-messaging/ +[Amazon Pub Sub Messaging](https://aws.amazon.com/pub-sub-messaging/) `AWS` -Asynchronous processing: http://blog.codepath.com/2013/01/06/asynchronous-processing-in-web-applications-part-2-developers-need-to-understand-message-queues/ +[Asynchronous processing](http://blog.codepath.com/2013/01/06/asynchronous-processing-in-web-applications-part-2-developers-need-to-understand-message-queues/) `codepath` -Async Request Response: https://www.enterpriseintegrationpatterns.com/patterns/conversation/RequestResponse.html +[Async Request Response](https://www.enterpriseintegrationpatterns.com/patterns/conversation/RequestResponse.html) `enterpriseintegrationpatterns.com` ## ## Event Driven Architectures -Martin Fowler- Event Driven Architecture: https://www.youtube.com/watch?v=STKCRSUsyP0 +[Martin Fowler- Event Driven Architecture](https://www.youtube.com/watch?v=STKCRSUsyP0) `Martin Fowler` `Video` -Event Driven Architecture: https://martinfowler.com/articles/201701-event-driven.html +[Event Driven Architecture](https://martinfowler.com/articles/201701-event-driven.html) `Martin Fowler` ## ## Hexagonal Architectures -Hexagonal Architecture: https://netflixtechblog.com/ready-for-changes-with-hexagonal-architecture-b315ec967749 +[Hexagonal Architecture](https://netflixtechblog.com/ready-for-changes-with-hexagonal-architecture-b315ec967749) `Netflix` ## ## Microservices -Monolith Architecture: https://buttercms.com/books/microservices-for-startups/should-you-always-start-with-a-monolith/ +[Monolith Architecture](https://buttercms.com/books/microservices-for-startups/should-you-always-start-with-a-monolith/) `buttercms.com` -Monoliths vs Microservices: https://articles.microservices.com/monolithic-vs-microservices-architecture-5c4848858f59 +[Monoliths vs Microservices](https://articles.microservices.com/monolithic-vs-microservices-architecture-5c4848858f59) `microservices.com` -Microservices: http://highscalability.com/blog/2018/4/5/do-you-have-too-many-microservices-five-design-attributes-th.html +[Microservices](http://highscalability.com/blog/2018/4/5/do-you-have-too-many-microservices-five-design-attributes-th.html) `Highscalability` -Uber Nanoservices antipattern: https://www.youtube.com/watch?v=kb-m2fasdDY +[Uber Nanoservices antipattern](https://www.youtube.com/watch?v=kb-m2fasdDY) `GOTO Conferences` -Uber Domain oriented microservice: https://eng.uber.com/microservice-architecture/ +[Uber Domain oriented microservice](https://eng.uber.com/microservice-architecture/) `Uber` ## ## Load Balancing -Load Balancer with Sticky Sessions: https://stackoverflow.com/questions/10494431/sticky-and-non-sticky-sessions +[Load Balancer with Sticky Sessions](https://stackoverflow.com/questions/10494431/sticky-and-non-sticky-sessions) `Stackoverflow` -Citrix what is load balancing: https://www.citrix.com/en-in/solutions/app-delivery-and-security/load-balancing/what-is-load-balancing.html +[Citrix what is load balancing](https://www.citrix.com/en-in/solutions/app-delivery-and-security/load-balancing/what-is-load-balancing.html) `Citrix` -Nginx Load Balancing: https://www.nginx.com/resources/glossary/load-balancing/ +[Nginx Load Balancing](https://www.nginx.com/resources/glossary/load-balancing/) `Nginx` -Consistent hashing: https://michaelnielsen.org/blog/consistent-hashing/ +[Consistent hashing](https://michaelnielsen.org/blog/consistent-hashing/) `Michael Nielsen` ## ## Alerts and Anomaly Detection -Outlier Detection: https://towardsdatascience.com/outlier-detection-with-isolation-forest-3d190448d45e +[Outlier Detection](https://towardsdatascience.com/outlier-detection-with-isolation-forest-3d190448d45e) `Towardsdatascience` -Anomaly Detection: https://towardsdatascience.com/machine-learning-for-anomaly-detection-and-condition-monitoring-d4614e7de770 +[Anomaly Detection](https://towardsdatascience.com/machine-learning-for-anomaly-detection-and-condition-monitoring-d4614e7de770) `Towardsdatascience` -Uber Real Time Monitoring and Root Cause Analysis Argos: https://eng.uber.com/argos-real-time-alerts/ +[Uber Real Time Monitoring and Root Cause Analysis Argos](https://eng.uber.com/argos-real-time-alerts/) `Uber` -Microsoft Anomaly Detection: https://www.youtube.com/watch?v=12Xq9OLdQwQ&t=0s +[Microsoft Anomaly Detection](https://www.youtube.com/watch?v=12Xq9OLdQwQ&t=0s) `Microsoft Research` -Facebook Data Engineering: https://engineering.fb.com/2016/05/09/core-data/introducing-fblearner-flow-facebook-s-ai-backbone/ +[Facebook Data Engineering](https://engineering.fb.com/2016/05/09/core-data/introducing-fblearner-flow-facebook-s-ai-backbone/) `Meta` -LinkedIn Real Time Alerting: https://engineering.linkedin.com/blog/2019/06/smart-alerts-in-thirdeye--linkedins-real-time-monitoring-platfor +[LinkedIn Real Time Alerting](https://engineering.linkedin.com/blog/2019/06/smart-alerts-in-thirdeye--linkedins-real-time-monitoring-platfor) `LinedIn` -LinkedIn Isolation Forests: https://engineering.linkedin.com/blog/2019/isolation-forest +[LinkedIn Isolation Forests](https://engineering.linkedin.com/blog/2019/isolation-forest) `LinedIn` ## ## Distributed Logging -Uber Distributed Request Tracing: https://eng.uber.com/distributed-tracing/ +[Uber Distributed Request Tracing](https://eng.uber.com/distributed-tracing/) `Uber` -Pintrest Logging: https://medium.com/@Pinterest_Engineering/open-sourcing-singer-pinterests-performant-and-reliable-logging-agent-610fecf35566 +[Pintrest Logging](https://medium.com/@Pinterest_Engineering/open-sourcing-singer-pinterests-performant-and-reliable-logging-agent-610fecf35566) `Pinterest` -Google Monitoring Infrastructure: https://www.facebook.com/atscaleevents/videos/959344524420015/ +[Google Monitoring Infrastructure](https://www.facebook.com/atscaleevents/videos/959344524420015/) `At Scale event` ## ## Metrics and Text Search Engine -Facebook real time text search engine: https://www.facebook.com/watch/?v=432864835468 +[Facebook real time text search engine](https://www.facebook.com/watch/?v=432864835468) `Meta` -Elastic Search Time Based Querying: https://www.elastic.co/guide/en/elasticsearch/guide/current/time-based.html +[Elastic Search Time Based Querying](https://www.elastic.co/guide/en/elasticsearch/guide/current/time-based.html) `Elastic` -Elastic Search Aggregation: https://www.elastic.co/guide/en/elasticsearch/guide/current/aggregations.html +[Elastic Search Aggregation](https://www.elastic.co/guide/en/elasticsearch/guide/current/aggregations.html) `Elastic` ## ## Single Point of Failure -Avoiding Single Points of Failure: https://medium.com/the-cloud-architect/patterns-for-resilient-architecture-part-3-16e8601c488e +[Avoiding Single Points of Failure](https://medium.com/the-cloud-architect/patterns-for-resilient-architecture-part-3-16e8601c488e) `Adrian Hornsby` -Netflix Multi-Region Availability: https://netflixtechblog.com/active-active-for-multi-regional-resiliency-c47719f6685b +[Netflix Multi-Region Availability](https://netflixtechblog.com/active-active-for-multi-regional-resiliency-c47719f6685b) `Netflix` -Oracle Single Points of failure: https://docs.oracle.com/cd/E19693-01/819-0992/fjdch/index.html +[Oracle Single Points of failure](https://docs.oracle.com/cd/E19693-01/819-0992/fjdch/index.html) `Oracle` -DNS single point of failure 2004: http://www.tenereillo.com/GSLBPageOfShame.htm +[DNS single point of failure 2004](http://www.tenereillo.com/GSLBPageOfShame.htm) `Pete Tenereillo` -Sharding: https://medium.com/@jeeyoungk/how-sharding-works-b4dec46b3f6 +[Sharding](https://medium.com/@jeeyoungk/how-sharding-works-b4dec46b3f6) `Jeeyoung Kim` ## ## Location Based Services -Google S2 library: https://blog.christianperone.com/2015/08/googles-s2-geometry-on-the-sphere-cells-and-hilbert-curve/ +[Google S2 library](https://blog.christianperone.com/2015/08/googles-s2-geometry-on-the-sphere-cells-and-hilbert-curve/) `Christian S. Perone` ## ## Batch Processing -Map Reduce Architecture: https://static.googleusercontent.com/media/research.google.com/en//archive/mapreduce-osdi04.pdf +[Map Reduce Architecture](https://static.googleusercontent.com/media/research.google.com/en//archive/mapreduce-osdi04.pdf) `Google` ## ## Real Time Stream Processing -LinkedIn Brooklin- Real time data streaming: https://engineering.linkedin.com/blog/2019/brooklin-open-source +[LinkedIn Brooklin- Real time data streaming](https://engineering.linkedin.com/blog/2019/brooklin-open-source) `LinkedIn` -Netflix Real Time Stream Processing: https://netflixtechblog.com/keystone-real-time-stream-processing-platform-a3ee651812a +[Netflix Real Time Stream Processing](https://netflixtechblog.com/keystone-real-time-stream-processing-platform-a3ee651812a) `Netflix` -KSQLDB for Kafka: https://docs.ksqldb.io/en/latest/operate-and-deploy/how-it-works/ +[KSQLDB for Kafka](https://docs.ksqldb.io/en/latest/operate-and-deploy/how-it-works/) `ksqlDB` ## ## Caching -Google Guava Cache: https://github.com/google/guava/wiki/CachesExplained +[Google Guava Cache](https://github.com/google/guava/wiki/CachesExplained) `Guava` -Caching (See the README): https://github.com/ben-manes/caffeine/ +[Caching (See the README)](https://github.com/ben-manes/caffeine/) `Caffeine` -Caching: http://highscalability.com/blog/2016/1/25/design-of-a-modern-cache.html +[Caching](http://highscalability.com/blog/2016/1/25/design-of-a-modern-cache.html) `Highscalability` -Microsoft Caching Guide: https://docs.microsoft.com/en-us/previous-versions/msp-n-p/dn589802(v%3dpandp.10) +[Microsoft Caching Guide](https://docs.microsoft.com/en-us/previous-versions/msp-n-p/dn589802(v%3dpandp.10)) `Microsoft` -Caching patterns: https://hazelcast.com/blog/a-hitchhikers-guide-to-caching-patterns/ +[Caching patterns](https://hazelcast.com/blog/a-hitchhikers-guide-to-caching-patterns/) `Nicolas Frankel` ## ## Distributed concensus -Paxos: http://ifeanyi.co/posts/understanding-consensus/ +[Paxos](http://ifeanyi.co/posts/understanding-consensus/) `Ifeanyi Ubah` -Raft: https://raft.github.io/ +[Raft](https://raft.github.io/) `Raft` ## ## Authorization -Designing an Authorization Model for an Enterprise: https://cerbos.dev/blog/designing-an-authorization-model-for-an-enterprise +[Designing an Authorization Model for an Enterprise](https://cerbos.dev/blog/designing-an-authorization-model-for-an-enterprise) `Cerbos.dev` ## ## Content Delivery Network -AWS CloudFront CDN with S3: https://aws.amazon.com/blogs/networking-and-content-delivery/amazon-s3-amazon-cloudfront-a-match-made-in-the-cloud/ +[AWS CloudFront CDN with S3](https://aws.amazon.com/blogs/networking-and-content-delivery/amazon-s3-amazon-cloudfront-a-match-made-in-the-cloud/) `AWS` ## ## System Design Resources -Designing Data Intensive Applications: https://amzn.to/3SyNAOy +[Designing Data Intensive Applications](https://amzn.to/3SyNAOy) InterviewReady Videos: [https://interviewready.io](https://interviewready.io?_aff=HELLOWORLD&source=github)