Caching: It Isn't Just a Performance Problem
Caching can make applications faster, but it also changes how systems handle freshness, correctness, personalization, invalidation, and failure. The real challenge isn't caching - it's deciding what can be safely reused, by whom, and for how long.
When an application gets slower, one of the first solutions we think about is -
"Let’s cache it."
And often, that’s the right answer. Instead of doing the same expensive work for every request, we can reuse work we’ve already done.
But as applications become larger, caching introduces a different set of architectural questions:
"What exactly are we caching?"
"Where should we cache it?"
"Who can safely reuse the result?"
"How long can it stay cached?"
"What happens when the underlying data changes?"
"What happens if the cache is unavailable or wrong?"
That is where caching stops being just a performance optimization and becomes a system-design problem.
Start With the Problem: Repeated Work
Consider a simple application request. A user requests a page, the application fetches data, and the server renders a response.
If ten users request the same page, the system may perform essentially the same work ten times. That could mean:
User
↓
Frontend / Server
↓
API / Database
↓
Response
If the underlying data hasn't changed, repeating that work may be unnecessary. This is the fundamental problem caching tries to solve - Can we reuse work we've already done?
The Basic Idea: Cache Hit vs. Cache Miss
A cache introduces another decision point into the request journey. If the requested response already exists, we have a cache hit. If it doesn't, we have a cache miss.
On a hit, we can return the cached response immediately. On a miss, we fetch the data from the source, store the result, and return it.
The important part is that a cache isn't doing anything magical. It is simply allowing the system to reuse previous work. But now we have a new problem:
"Where should that cached result live?"
There Isn't Just One Cache
In a modern web application, caching can happen at several layers.
Browser cache
The browser can cache things such as JavaScript, CSS, images, fonts, and some HTML responses. This avoids downloading the same resources repeatedly for the same user.
CDN or Edge cache
A CDN can cache content closer to users. For public content, this can reduce latency and reduce requests reaching the application servers.
Application cache
The application can cache computed values, fetched data, expensive transformations, and rendered or partially rendered results.
API or Data cache
The data layer can cache database query results, external API responses, CMS responses, and expensive aggregations.
Database or CMS
At the bottom of the stack is still the source of truth. Caching doesn't eliminate the need for the underlying data source. It changes how often we need to reach it. This gives us an important insight:
"Caching isn't a single feature. It's a property of the architecture."
What Are We Actually Caching?
Different kinds of content have very different caching characteristics.
A static JavaScript bundle might be safe to cache for a very long time, especially when it is versioned.
A CMS article might tolerate a cache lifetime of several minutes.
An API response might depend on how frequently the underlying data changes.
And personalized data may require user-specific caching or no shared caching at all.
So instead of asking, "Should we cache this?", I find it more useful to ask: "What exactly is this response, and who is allowed to reuse it?"
Public Content: The Easy Case
Caching becomes straightforward when everyone can safely receive the same response.
Imagine a public documentation page.
User A requests it.
User B requests it.
User C requests it.
If the content is identical for everyone, a shared cache can serve all three users.
The cache doesn't need to know much about the individual users because the response itself is not user-specific. This is where CDNs and edge caching become particularly powerful. One generated response can potentially serve thousands of users.
But the story changes completely when the response depends on context.
The Personalization Problem
In Part 1 of this series, I introduced personalization as a frontend architecture concern - where the content or experience can vary based on a user's context, such as their audience, role, or location.
Consider a page whose content changes based on the user's audience, location, role, or other context.
User A might receive: "Welcome to our Developer experience."
User B might receive: "Welcome to our Business Leader experience."
The URL could still be exactly the same.
Now imagine User A's personalized response is stored in a shared cache. User B requests the same URL.
If the cache key doesn't account for the relevant personalization context, User B could receive User A's response.
That's not simply a performance issue. It's a correctness issue. And correctness should win over performance. This is one of the most important reasons to think about caching at the system level.
Cache Keys Matter
A cache doesn't simply store "the page." It stores a value associated with some notion of a key.
Conceptually:
cache.get(key)
↓
cached value
The hard question is: "What belongs in that key?"
For public content, the URL might be enough. For contextual content, the key may need to include some representation of the context that changes the response.
For example, /products might safely map to one shared response. Whereas something conceptually like /products?persona=developer represents a different cacheable variant.
But adding more dimensions to a cache key has a cost. If the key includes too many variables, the number of unique cache entries grows, and the cache hit rate can fall.
So cache-key design itself becomes an architectural trade-off.
Freshness vs. Performance
Caching introduces another fundamental trade-off:
"How long should the cached value remain valid?"
A longer cache lifetime can provide fewer backend requests, lower latency, lower infrastructure load, and higher cache hit rates.
But it can also produce older data, delayed updates, and stale content. A shorter cache lifetime generally gives us fresher data, but we give up some of the performance benefits.
There isn't one universally correct cache duration. The right answer depends on the data.
For example:
| Data | Typical concern |
|---|---|
| Versioned static assets | Usually safe to cache for a long time |
| Documentation or CMS content | Moderate freshness |
| Product listings | Depends on update frequency |
| Search results | Depends on query and freshness |
| Stock or inventory data | Freshness can be critical |
| Personalized responses | User context and correctness |
The cache duration is therefore not just a tuning parameter. It is part of the product and system requirements.
Cache Invalidation: When Is Cached Data No Longer Valid?
This is where the famous caching problem appears:
"How do we know when the cached value is no longer correct?"
Suppose a CMS article is cached for ten minutes. The author updates the article after thirty seconds. The cache may still contain the previous version. We now need a policy.
Option 1 - TTL
Let the cached value expire after a fixed amount of time. Simple, but potentially stale.
Option 2 - Revalidation
Check whether the underlying value has changed and refresh when necessary. This can provide better freshness, but adds complexity.
Option 3 - Explicit invalidation
When the underlying content changes, actively invalidate the relevant cached value. This gives us stronger freshness guarantees, but the invalidation path itself becomes part of the system.
Option 4 - Stale-while-revalidate
Serve the existing value immediately while refreshing it in the background. This can provide a good user experience, but requires careful implementation.
There is no universally best invalidation strategy. The right choice depends on how expensive stale data is.
Failure Is Part of the Cache Design
A cache is another dependency. That means we should also ask:
"What happens if the cache isn't available?"
For some applications, a cache failure can simply mean:
Cache unavailable
↓
Fetch from source
↓
Return response
The application becomes slower, but remains functional. For other systems, bypassing the cache could create a sudden surge of requests against the database or external API.
That creates another system-design problem:
"What happens when many requests miss the cache at the same time?"
This is one reason cache behavior needs to be considered together with backend capacity, request volume, and failure modes.
A cache can reduce load dramatically when it works well. But a badly designed cache can also create surprising load patterns.
A Simple Next.js Example
In Next.js, we can cache a fetch response with a revalidation period:
const data = await fetch("/api/products", {
next: { revalidate: 60 },
});
The 60 tells the framework that this response can be revalidated after 60 seconds.
But the interesting architectural questions aren't about the number alone. They are:
"What does this response represent?"
"Can different users safely share it?"
"How fresh does it need to be?"
"What happens when the underlying data changes?"
"What happens if the cache is unavailable?"
The configuration is the implementation detail. The caching policy is the architectural decision.
The Questions I Ask Before Adding a Cache
When I see an opportunity to cache something, I now try to answer a small set of questions first.
"What exactly are we caching?"
"Where should it be cached?"
"Who can safely reuse it?"
"What should the cache key contain?"
"How long is it valid?"
"How is it invalidated or revalidated?"
"What happens when the cache is unavailable?"
"What happens if the cached value is wrong?"
"Can a cache miss create a backend load spike?"
"Does caching change the correctness of the response?"
These questions turn "let's add a cache" into an actual engineering discussion.
From Faster to Correct
Caching is one of the most powerful tools we have for improving application performance. But the interesting engineering work isn't simply deciding whether to cache.
It's deciding:
"What to cache?"
"Where to cache it?"
"Who can reuse it?"
"How long does it remain valid?"
"How does it become invalid?"
"What happens when things go wrong?"
A cache that makes an application faster but occasionally serves the wrong content isn't a successful optimization.
"A fast system isn't useful if it serves the wrong data."
The goal isn't maximum caching. The goal is safe reuse.
What's Next?
We've seen how caching becomes more complicated when responses depend on users and context. The next natural question is:
"How do you build personalization without turning every request into an expensive, uncached request?"
That's where caching, user context, and the edge start colliding.
And that's where I'm heading next.
In the next part of Beyond the Component, we'll explore what happens When Personalization Meets the Edge and how user context, caching, geography, and edge delivery come together to shape the architecture of a personalized experience.
