>_
GolangStepByStep
Software Engineer

Caching Strategies

In-memory vs distributed, invalidation patterns, design trade-offs

# The Refrigerator Analogy

Imagine you want to make a sandwich. You need tomatoes.

The Database is the grocery store 10 miles away. It has absolutely everything, it never runs out, and it's perfectly reliable. But driving there takes 20 minutes. If you have to drive to the grocery store every single time a customer orders a sandwich, your restaurant will fail.

The Cache is the refrigerator under your kitchen counter. It holds very little data, but accessing it takes 2 seconds. Caching simply means: The first time someone orders a tomato sandwich, you drive to the store, buy 10 tomatoes, use 1, and put the other 9 in the fridge (The Cache). The next 9 orders will be blindingly fast.

# Level 1: In-Memory Caching (Beginner)

The absolute fastest way to cache data in Go is to store it directly in the running application's RAM using a map.

Because traffic comes in via concurrent Goroutines, if two requests try to read and write to the map at exactly the same time, Go will panic and crash. We must protect our fridge using a Read-Write Mutex.

type SimpleCache struct {
    mu   sync.RWMutex
    data map[string]string // The Fridge!
}

// Write to Cache
func (c *SimpleCache) Set(key, value string) {
    c.mu.Lock()         // Lock the door completely. No one can enter.
    c.data[key] = value // Stock the fridge
    c.mu.Unlock()       // Open the door
}

// Read from Cache
func (c *SimpleCache) Get(key string) (string, bool) {
    c.mu.RLock()        // Read-Lock: Multiple readers allowed! Writers must wait.
    val, found := c.data[key]
    c.mu.RUnlock()
    return val, found
}

# Level 2: Expiration / TTL (Intermediate)

Our fridge has a massive flaw: we never throw anything away. Eventually, the tomatoes rot (the data becomes stale), or the fridge literally explodes (the server runs out of RAM and OOM crashes).

To fix this, caches introduce Time-to-Live (TTL). Every piece of data is stamped with an expiration date.

type Item struct {
    Value      string
    Expiration time.Time // When does this rot?
}

// ... inside the Get function ...
item, found := c.data[key]

// If it's found, check if it's expired!
if found && time.Now().After(item.Expiration) {
    // The data is rotten! Pretend we didn't find it.
    // (A background goroutine should technically delete it soon)
    return "", false 
}

return item.Value, true

This forces your application to experience a Cache Miss, triggering the application to drive back to the Grocery Store (Database), fetch the fresh data, and restock the cache with a shiny new Expiration date!

# Level 3: Distributed Caches (Advanced)

As your restaurant becomes famous, you buy 10 separate buildings (Servers behind a Load Balancer).

If a customer updates their phone number on Server A, Server A updates its Local In-Memory Fridge. But Servers B through J have no idea. Their fridges still have the old phone number! If the customer refreshes their page and the Load Balancer sends them to Server B, they will see their old data. This is horrible user experience.

To survive, enterprise systems use Distributed Caching (like Redis or Memcached).

  • Redis is a separate, dedicated server whose only job is holding memory.
  • Servers A through J entirely stop using their local fridges.
  • Instead, they ALL connect over the network to the massive, shared Redis Fridge.
  • When Server A updates a phone number in Redis, Server B instantly sees it!

Trade-off: Reading from local RAM takes ~50 nanoseconds. Reading from a Redis cache over the network takes ~1 millisecond. Redis is radically faster than a Database query (100ms), but slower than Local RAM. We trade speed for state consistency.

# Level 4: The Thundering Herd Problem (Expert)

Imagine you have a complex query that takes 10 seconds to generate the "Trending Front Page News" on your app. You cache it in Redis for 5 minutes.

Your app has 5,000 active users. Exactly at the 5-minute mark, the cache gracefully expires. Suddenly, 1,000 users refresh the page at the exact same second.

1,000 Goroutines check Redis. All 1,000 get a Cache Miss. All 1,000 launch the 10-second Database Query simultaneously. Your Database's CPU hits 100% and violently crashes.

This is called a Cache Stampede or Thundering Herd. Go solves this elegantly using the singleflight package.

// Shared singleflight group
var group singleflight.Group

func GetNewsFeed() string {
    // 1. Check Redis Cache
    if val := checkRedis("news_key"); val != "" { return val }

    // 2. CACHE MISS! Thousands of Goroutines hit this line!
    // But 'group.Do' guarantees that any requests for "news_key"
    // will just PAUSE and wait for the very first Goroutine to finish!
    result, err, _ := group.Do("news_key", func() (any, error) {
        
        // This slow 10-second DB call is executed EXACTLY ONCE!
        slowData := fetchFromDatabase() 
        saveToRedis("news_key", slowData)
        return slowData, nil
        
    })

    // 3. The data is instantly shared to all 1,000 waiting Goroutines at once!
    return result.(string)
}

singleflight mathematically guarantees that no matter how enormous your concurrent traffic spike is, your database will only ever receive exactly one query for a missing cache key. This pattern is the holy grail of highly-scaled Go architectures!

practice & review