Modern web applications widely deploy cache across the stack to speed up data access and improve throughput.
In this talk, I will discuss three trends in hardware, workload, and cache usage that shape the design of modern caches.
The slow down of memory capacity scaling, the exploding data creation rate, and the wide adoption of Time-to-live (TTL) motivated the design of Segcache (https://paper.segcache.com and https://open-source.segcache.com).
Segcache is a new cache storage design that provides high memory efficiency and high throughput via metadata sharing to save space wasted on metadata, approximate indexing to efficiently remove short-lived data, and macro management for high throughput and linear scalability.
The design of Segcache is driven by the insight that key-value cache is different from key-value store + eviction, and is the outcome of exploring different trade-offs between caching and storage.
We deploy Segcache as standalone distributed caches and embedded off-heap caches for JVM-based microservices at Twitter. The distributed cache deployment significantly reduces the resources (CPU/DRAM) provisioned for cache clusters; the embedded deployment allows us to speed up data access and reduce GC pause time.
Speaker
Juncheng Yang
Ph.D. student @CarnegieMellon, Focus on Efficiency and Performance, Previously @Twitter & @Cloudflare, Facebook Fellow
As a 5th-year Ph.D. student at Carnegie Mellon University and part of Parallel Data Lab, Juncheng's research studies and improves the efficiency, performance, and reliability of large-scale web applications with a current focus on caching systems.
His works in collaboration with Twitter, Meta, Google, and Akamai have been published at various academic conferences such as OSDI, NSDI, SOSP, and SOCC and have won multiple best paper awards.
One of the designs, Segcache, a high-throughput, space-efficient in-memory cache, has been in production at Twitter.
Besides being a Ph.D. student, he worked at Twitter and Cloudflare, improving the cache infrastructure's efficiency. Moreover, his research is sponsored by Meta, and he is a Facebook Fellow.
