Every request your app serves is a chance to do less work. Caching is just the art of not redoing work you already did. Sounds simple. It isn’t — there’s a whole stack of places you can stash data, each with its own rules, and picking the wrong layer (or the wrong expiry) costs you either stale bugs or a slow site.
I’ve spent enough hours staring at a page that “should have updated” but didn’t, only to realise the answer was cached three layers deep. So let me walk through every place you can cache — from the user’s browser all the way down to compiled bytecode on your server — and where each one bites you.
Redis is one of those rare pieces of software where the source code is short enough to actually read and deep enough to teach you something new every time. I’ve spent a good amount of time going through the actual Redis source — and what strikes me every time is how deliberately every design choice was made. Nothing is accidental.
So let’s actually design one from the ground up — not a toy, but an architecture that mirrors what Redis actually does. Data structures, event loop, protocol, expiry, persistence. All of it.
“Just cache it in Redis” — you’ve probably heard that in a code review, a system design interview, or a Stack Overflow comment. It’s almost a reflex at this point. But why Redis specifically? Why not a regular database with a good index, or any other in-memory store? I dug into the history, the architecture, and the current landscape of alternatives, and the story is genuinely more interesting than the meme lets on.
You expose an API endpoint like /api/orders/1042. That integer tells anyone listening — a competitor, an attacker, a curious user — exactly how many orders you have. Change the number to 1041, you get the previous order. Change it to 1, you get the very first one. No auth bypass needed. The ID itself is the information leak.
That’s the sequential ID problem in one paragraph. UUID exists to fix it — and a few other things that matter at scale.
Kafka looks deceptively simple from the outside — you publish to a topic, someone reads from it. Under the hood it is a fairly intricate distributed system where several pieces have to agree on who owns what before a single byte gets delivered. I spent a good amount of time untangling this, and most articles stop at “partitions give you parallelism” without explaining the actual handshake. Let me go deeper.