Cache Implementation Considerations

• Is it safe to cache data? Data maybe out of date, eventually consistent

• Is caching effective for that data?

  • Pattern: data changing slowly, few keys are frequently needed
  • Anti Patterns: data changing rapidly, all large key space frequently needed

• Is data structured well for caching?

  • ex: key value caching, or caching of aggregations results

• Which caching design pattern is the most appropriate?

Lazy Loading / Cache Aside / Lazy Population

• PROS:

  • Only requested data is cached (the cache is not filled up with unused data)
  • Node failures are not fatal (just increased latency to warm the cache)

• CONS:

  • Cache miss penalty that results in 3 round trips, noticeable delay for that request
  • Stale data: data can be updated in database and outdate in the cache

Write Through - Add or Update cache when database is updated

• PROS:

  • Data in cache never stale, reads are quick
  • Write penalty vs Read penalty (each write requires 2 calls)

• CONS:

  • Missing Data until it is added/updated in the DB. Mitigation is to implement Lazy Loading strategy as well
  • Cache churn - a lot of the data will never read

Cache Evictions and Time-To-Live (TTL)

• Cache Evictions can occur in three ways:

  • You delete the item explicitly in the cache
  • Item is evicted because the memory is full and it is not recently used (LRU)
  • You set an item time-to-live (TTL)

• TTL are helpful for any kind of data:

  • Leaderboards
  • Comments
  • Activity Stream

• TTL can range from few seconds to hours or days