Redis Internals: From Boot to Key Expiration and Recovery

Redis is a powerful in-memory key-value store that serves as a backbone for caching, session management, real-time analytics, and more. But have you ever wondered what happens under the hood when Redis starts, processes commands, handles key expiration, and ensures consistent performance? This article breaks down Redis internals in a step-by-step manner, taking you from its boot process to how it manages keys and expiration.

1. The Redis Boot Process

When Redis boots up, it follows a structured initialization process:

1.1) Configuration Loading:

Redis reads its configuration file (e.g., redis.conf) or command-line arguments.
Configurations like memory limits, eviction policies, persistence options, and port settings are loaded.

1.2) Event Loop Setup:

Redis initializes its event loop, which is responsible for handling client requests, key expiration, and other periodic tasks. This event loop is single-threaded but highly efficient.

1.3) Data Restoration (if applicable):

If persistence is enabled (via RDB or AOF), Redis loads the data back into memory. This step ensures that the system recovers its state after a restart.

1.4) Listening for Connections:

Redis binds to the configured port (default: 6379) and starts listening for client connections.

At this point, Redis is fully operational and ready to handle commands.

2. Handling Commands: From Connection to Execution

Redis handles commands in the following steps:

2.1) Connection Establishment:

A client connects to the Redis server over a TCP socket. The connection is non-blocking, allowing Redis to handle multiple clients concurrently.

2.2) Command Parsing:

The client sends commands in the Redis Serialization Protocol (RESP) format. Redis parses these commands into structured requests.

2.3) Command Execution:

The parsed command is routed to the appropriate function in Redis’s command table. For example, a SET command triggers the setGenericCommand function.

2.4) Response Sending:

After execution, Redis sends the result back to the client. This could be a confirmation (e.g., OK) or data (e.g., the value of a key).

3. Setting and Retrieving Keys

How Redis Sets a Key

When you execute a SET command, here’s what happens:

3.1) Command Parsing: Redis identifies the SET command and extracts the key, value, and optional arguments (like expiration).

3.2) Key Storage:

Redis stores the key-value pair in its in-memory dictionary (dict).
If an expiration is provided, Redis also adds an entry to the expires dictionary, mapping the key to its expiration timestamp.

3.3) Persistence (if enabled):

If Append-Only File (AOF) persistence is active, Redis logs the SET command to disk.

Example:

127.0.0.1:6379> SET user:1000 "Harsh" EX 10
OK

The key user:1000 is stored in dict, and its expiration time is recorded in expires.

How Redis Retrieves a Key

Key Lookup:

Redis checks the dict to locate the key.

2. Expiration Check:

Before returning the value, Redis verifies if the key exists in expires and whether its expiration time has passed.
If the key has expired, Redis deletes it from memory and returns nil.

4. Key Expiration in Detail

Redis handles key expiration using two complementary mechanisms:

4.1) Passive Expiration

When a client attempts to access a key, Redis checks its expiration time.
If the key has expired, Redis deletes it immediately and returns nil.

4.2) Active Expiration

Redis periodically scans a subset of keys with expiration times.
It uses a probabilistic approach to remove expired keys without consuming too many resources.

Example Workflow:

Key set with expiration:

127.0.0.1:6379> SET session:123 "active" EX 5 OK

After 5 seconds:

127.0.0.1:6379> GET session:123 (nil)

5. How Redis Ensures Expired Keys Are Not Returned

5.1) Lookup in **expires** Dictionary:

Before returning a key’s value, Redis checks if the key exists in the expires dictionary and whether the expiration time has passed.

5.2) Immediate Deletion:

Expired keys are deleted during lookup (passive expiration) or periodic scans (active expiration).

5.3) Memory Cleanup:

Expired keys are removed from memory, freeing resources for other operations.

Code Perspective:

Internally, Redis checks key expiration like this (simplified):

if (current_time > expires[key]) {
delete_key(key);
return NULL;
}

6. Memory Management and Recovery

Redis employs several strategies to manage memory:

6.1) Eviction Policies:

If Redis reaches its memory limit (maxmemory), it applies an eviction policy to free space. Policies include:
noeviction: Commands that increase memory fail.
allkeys-lru: Evicts the least recently used keys.
volatile-lru: Evicts the least recently used keys with expiration.

127.0.0.1:6379> CONFIG SET maxmemory 100mb 127.0.0.1:6379> CONFIG SET maxmemory-policy allkeys-lru

6.2) Lazy Deletion:

Expired keys are removed during lookups or periodic scans, reducing unnecessary memory usage.

6.3) Persistence Options:

Redis supports RDB snapshots and AOF logs for data recovery. During a restart, Redis loads data from these files back into memory.

7. Hands-On Example with Node.js

Here’s a practical example using the ioredis library:

Setting a Key with Expiration

const Redis = require("ioredis");
const redis = new Redis();
async function redisExample() {
// Set a key with expiration
await redis.set("session:123", "active", "EX", 5);
console.log("Key set with expiration");
// Try to get the key immediately
let value = await redis.get("session:123");
console.log("Initial Value:", value); // Should print "active"
// Wait for 6 seconds
setTimeout(async () => {
value = await redis.get("session:123");
console.log("Value after 6 seconds:", value); // Should print null
}, 6000);
}
redisExample();

Conclusion

Redis is a highly efficient, single-threaded system that combines an event loop, dictionaries for key storage, and robust expiration mechanisms to deliver low-latency performance. From booting up to managing keys and handling expiration, every component of Redis is optimized for speed and simplicity. Understanding these internals helps you use Redis more effectively and troubleshoot issues when they arise.