How Uber Serves Over 40 Million Reads Per Second From Online Storage Using An Integrated Cache
tl;dr: “Docstore is Uber’s in-house, distributed database built on top of MySQL. Storing tens of PBs of data and serving tens of millions of requests/second, it is one of the largest database engines at Uber used by microservices from all business verticals. Docstore users and use cases are growing, and so are the request volume and data footprint. This post discusses the challenges serving applications that require low-latency read access and high scalability.featured in #491
Data-Caching Techniques For 1.2 Billion Daily API Requests
- Guillermo Pérez tl;dr: The cache needs to achieve three things: (1) Low latency: It needs to be fast. If a cache server has issues, you can’t retry. (2) Up and warm: It needs to hold the majority of the critical data. If you lose it, it would surely bring down the backend systems with too much load. (3) Consistency: It should never hold stale or incorrect data. “A lot of the techniques mentioned in this article are supported by our open source meta-memcache cache client.”featured in #486
How DoorDash Standardized And Improved Microservices Caching
- Jason Fan Lev Neiman tl;dr: DoorDash's expanding microservices architecture led to challenges in interservice traffic and caching. The article details how DoorDash addressed these challenges by developing a library to standardize caching, enhancing performance without altering existing business logic. Key features include layered caches, runtime feature flag control, and observability with cache shadowing. The authors also provides guidance on when to use caching.featured in #459
FIFO Queues Are All You Need For Cache Eviction
- Juncheng Yang tl;dr: “I will describe a simple, scalable eviction algorithm with three static FIFO queues. Evaluated on 6594 cache traces from 14 companies, we show that S3-FIFO has lower miss ratios than 12 state-of-the-art algorithms designed in the past decades. Moreover, S3-FIFO’s efficiency is robust — it has the lowest mean miss ratio on 10 of the 14 datasets. The use of FIFO queues enables S3-FIFO to achieve good scalability with 6× higher throughput compared to optimized LRU in cachelib at 16 threads.”featured in #443
Architecture Patterns: Caching
- Kislay Verma tl;dr: "Depending on the type of application, the type of data, and the expectation of failure, there are several strategies that can be applied for caching." Kislay discusses the levels in a systems architecture where caching commonly occurs and various caching strategies, such as read through, write through, write behind.featured in #302
New HTTP Standards For Caching On The Modern Web
- Tim Perry tl;dr: The Internet Engineering Task Force (IETF) is working to define new HTTP standards to reduce complexity around caching, launching 2 new HTTP header draft standards intended to make debugging caching easier, and provide more control over configuration. Standards are The Cache-Status Header and Targeted Cache-Control Headers, outlined here.featured in #262
Caches, Modes, And Unstable Systems
- Marc Brooker tl;dr: "Good caches have feedback loops. Like back pressure, and limited concurrency. Bad caches are typically open-loop. This starts to give us a hint about how we may use caches safely, and points to some of the safe patterns for distributed systems caching."featured in #250
Why We Started Putting Unpopular Assets In Memory
- Yuchen Wu tl;dr: Placing assets in memory, there's a trade-off between "memory usage, tail hit latency and SSD lifetimes."featured in #178
featured in #133