“Just cache it in Redis” — you’ve probably heard that in a code review, a system design interview, or a Stack Overflow comment. It’s almost a reflex at this point. But why Redis specifically? Why not a regular database with a good index, or any other in-memory store? I dug into the history, the architecture, and the current landscape of alternatives, and the story is genuinely more interesting than the meme lets on.
You expose an API endpoint like /api/orders/1042. That integer tells anyone listening — a competitor, an attacker, a curious user — exactly how many orders you have. Change the number to 1041, you get the previous order. Change it to 1, you get the very first one. No auth bypass needed. The ID itself is the information leak.
That’s the sequential ID problem in one paragraph. UUID exists to fix it — and a few other things that matter at scale.
Kafka looks deceptively simple from the outside — you publish to a topic, someone reads from it. Under the hood it is a fairly intricate distributed system where several pieces have to agree on who owns what before a single byte gets delivered. I spent a good amount of time untangling this, and most articles stop at “partitions give you parallelism” without explaining the actual handshake. Let me go deeper.