Redis Core Concepts: Data Structures, TTLs, and Persistence
A staff-engineer guide to Redis data structures, expiry semantics, and RDB/AOF persistence trade-offs for backend caching layers.
Redis Core Concepts
Redis is the default answer to "how do we make this fast" in almost every backend system — session storage, hot-path caching, leaderboards, rate limiters, and pub/sub fan-out all lean on the same small set of primitives. This guide covers those primitives in depth: the data structures, how expiry actually works under the hood, and the durability trade-offs you're implicitly making the moment you point production traffic at a Redis instance.
1. What Redis Actually Is
Redis is an in-memory data structure server. That phrase matters more than "cache" — Redis is not just a key-value store with a TTL bolted on. It exposes rich, typed data structures (strings, hashes, lists, sets, sorted sets, streams, bitmaps, HyperLogLogs) and operates on them with atomic, single-threaded command execution.
Single-threaded, but not slow. The core command execution loop in Redis runs on one thread, which is what makes operations like INCR and LPUSH atomic without locks. Redis 6+ added I/O threading for network reads/writes, but command execution itself stays single-threaded. This is a deliberate trade-off: predictable latency over parallel throughput.
Why teams reach for Redis
| Use case | Data structure | Why Redis fits |
|---|---|---|
| Session storage | Hash / String (JSON) | Sub-millisecond reads, native TTL |
| Cache layer in front of a DB | String | Simple get/set, TTL-based eviction |
| Rate limiting | String (counter) / Sorted Set | Atomic INCR, sliding window via ZSET |
| Leaderboards | Sorted Set | O(log N) ranked inserts, ZRANGE by score |
| Distributed locks | String (SET NX PX) | Atomic conditional set with expiry |
| Real-time fan-out | Pub/Sub | Lightweight, no persistence overhead |
| Deduplication at scale | HyperLogLog / Set | Approximate or exact cardinality |
2. Strings
The STRING type is Redis's simplest and most-used data structure — a binary-safe byte sequence up to 512MB. It's used for plain values, serialized JSON, counters, and bitmaps.
# Basic set/get
SET user:1001:name "Alicia Keys"
GET user:1001:name
# → "Alicia Keys"
# Set with expiry in one atomic command
SET session:abc123 "{\"userId\":1001}" EX 1800
# Atomic counters — no read-modify-write race
INCR page:home:views
INCRBY inventory:sku-42 -5
INCRBYFLOAT wallet:1001:balance 19.99
# Conditional writes — the basis of distributed locks
SET lock:order:5001 "worker-7" NX PX 10000With Spring Data Redis (Lettuce client, Spring Boot 3.x), the equivalent looks like:
@Service
public class ProductCacheService {
private final StringRedisTemplate redisTemplate;
public ProductCacheService(StringRedisTemplate redisTemplate) {
this.redisTemplate = redisTemplate;
}
public void cacheProduct(String productId, String json) {
redisTemplate.opsForValue()
.set("product:" + productId, json, Duration.ofMinutes(15));
}
public Optional<String> getCachedProduct(String productId) {
return Optional.ofNullable(
redisTemplate.opsForValue().get("product:" + productId));
}
public long incrementViewCount(String productId) {
return redisTemplate.opsForValue()
.increment("product:" + productId + ":views");
}
}Prefer StringRedisTemplate over the generic RedisTemplate<Object, Object> for simple string/JSON payloads. It uses StringRedisSerializer for both keys and values by default, avoiding the Java-serialization or JDK-default serializer footguns that produce unreadable binary blobs in redis-cli.
3. Hashes
A HASH maps field names to values within a single key — effectively a mini object. Hashes are the right structure when you have several related fields you want to read/write independently without deserializing an entire JSON blob.
HSET user:1001 name "Alicia Keys" email "alicia@example.com" plan "pro"
HGET user:1001 email
HGETALL user:1001
HINCRBY user:1001 loginCount 1
HDEL user:1001 plan@Service
public class UserProfileCacheService {
private final RedisTemplate<String, Object> redisTemplate;
public void cacheProfile(String userId, Map<String, Object> fields) {
String key = "user:" + userId;
redisTemplate.opsForHash().putAll(key, fields);
redisTemplate.expire(key, Duration.ofHours(6));
}
public Object getField(String userId, String field) {
return redisTemplate.opsForHash().get("user:" + userId, field);
}
public Long incrementLoginCount(String userId) {
return redisTemplate.opsForHash()
.increment("user:" + userId, "loginCount", 1);
}
}HGETALL on a large hash blocks other clients for the duration of the response serialization, since Redis is single-threaded. Avoid hashes with thousands of fields — if you need that, consider bucketing into multiple keys or using HSCAN for incremental iteration instead of a single HGETALL.
String (JSON blob) vs Hash — when to use which
| Aspect | String (serialized JSON) | Hash |
|---|---|---|
| Partial field update | Requires read-modify-write of whole blob | HSET updates one field atomically |
| Memory overhead | Lower for small objects | Slightly higher per-field overhead |
| Field-level TTL | Not possible (Redis < 7.4) | Not possible either — TTL is per-key |
| Read pattern | Always deserialize whole object | Fetch only needed fields with HMGET |
| Best for | Immutable or rarely-updated blobs | Objects with independently-updated fields |
4. Lists
LIST is a doubly linked list of strings, ordered by insertion. Lists are the backbone of simple queues, activity feeds, and recent-items caches.
LPUSH notifications:1001 "New comment on your post"
RPUSH activity:feed:1001 "logged in"
LRANGE activity:feed:1001 0 9 # latest 10 items
LTRIM activity:feed:1001 0 99 # keep only the newest 100
BLPOP queue:jobs 5 # blocking pop, 5s timeoutpublic void pushRecentActivity(String userId, String activity) {
String key = "activity:feed:" + userId;
redisTemplate.opsForList().leftPush(key, activity);
redisTemplate.opsForList().trim(key, 0, 99); // cap the feed size
redisTemplate.expire(key, Duration.ofDays(30));
}
public List<Object> getRecentActivity(String userId, int count) {
return redisTemplate.opsForList().range("activity:feed:" + userId, 0, count - 1);
}Redis lists are not a message broker. BLPOP/BRPOP give you a simple work queue, but there's no consumer groups, no acknowledgment/redelivery, and no persistence guarantee beyond whatever durability you've configured. For anything beyond a lightweight job queue, use a real broker (Kafka, RabbitMQ, or Redis Streams if you want to stay in the Redis ecosystem).
5. Sets and Sorted Sets
A SET is an unordered collection of unique strings — ideal for membership checks, tagging, and set algebra (union/intersection/difference). A SORTED SET (ZSET) adds a floating-point score to each member, keeping the collection ordered by score with O(log N) inserts.
# Set — membership and set algebra
SADD user:1001:roles "admin" "billing"
SISMEMBER user:1001:roles "admin" # → 1
SINTER user:1001:roles user:1002:roles # shared roles
# Sorted set — leaderboard
ZADD leaderboard:global 15420 "player:88"
ZADD leaderboard:global 18990 "player:12"
ZREVRANGE leaderboard:global 0 9 WITHSCORES # top 10
ZRANK leaderboard:global "player:88" # rank of a specific player
ZINCRBY leaderboard:global 100 "player:88" # atomic score bump@Service
public class LeaderboardService {
private final ZSetOperations<String, String> zSetOps;
public LeaderboardService(RedisTemplate<String, String> redisTemplate) {
this.zSetOps = redisTemplate.opsForZSet();
}
public void recordScore(String playerId, double score) {
zSetOps.add("leaderboard:global", playerId, score);
}
public Set<ZSetOperations.TypedTuple<String>> topPlayers(int count) {
return zSetOps.reverseRangeWithScores("leaderboard:global", 0, count - 1);
}
public Long rankOf(String playerId) {
Long rank = zSetOps.reverseRank("leaderboard:global", playerId);
return rank == null ? null : rank + 1; // 0-indexed → human rank
}
}Sorted sets are the standard building block for sliding-window rate limiters. Store request timestamps as scores, ZREMRANGEBYSCORE to evict entries older than the window, and ZCARD to count requests in the current window. We cover the full pattern in the distributed locks and rate limiting guide.
Data structure decision table
| Need | Structure | Key commands |
|---|---|---|
| Simple value / counter / lock | String | SET, GET, INCR |
| Object with multiple fields | Hash | HSET, HGETALL, HINCRBY |
| Ordered, bounded feed or queue | List | LPUSH, LRANGE, LTRIM |
| Unique tags / membership / set ops | Set | SADD, SISMEMBER, SINTER |
| Ranked data / sliding windows | Sorted Set | ZADD, ZRANGE, ZREMRANGEBYSCORE |
| Append-only event log with consumer groups | Stream | XADD, XREADGROUP, XACK |
| Approximate unique counting at scale | HyperLogLog | PFADD, PFCOUNT |
6. TTLs and Expiry Semantics
Every Redis key can carry an optional time-to-live. Understanding exactly how expiry is enforced matters — it isn't a background cron job deleting keys on a schedule; it's a hybrid of lazy and active mechanisms.
Expiry commands
EXPIRE session:abc123 1800 # set TTL in seconds
PEXPIRE lock:order:5001 10000 # set TTL in milliseconds
TTL session:abc123 # remaining seconds (-1 = no TTL, -2 = doesn't exist)
PERSIST session:abc123 # remove TTL, key lives forever
EXPIREAT session:abc123 1755200000 # expire at a specific Unix timestampA common production bug: overwriting a key with SET silently clears its TTL unless you re-specify it. SET session:abc123 newValue — with no EX/PX — turns your session key permanent. Use SET key val KEEPTTL (Redis 6+) if you want to update the value without touching the expiry, or always pass the TTL explicitly on every write.
// Spring Data Redis — TTL-safe update pattern
public void refreshSessionData(String sessionId, String newPayload) {
redisTemplate.opsForValue().set(
"session:" + sessionId,
newPayload,
Duration.ofMinutes(30) // always re-assert TTL explicitly
);
}Eviction policies when memory fills up
TTL only removes keys that have an expiry. When Redis hits maxmemory, the eviction policy decides what happens next — and the default (noeviction) is almost never what you want for a cache.
| Policy | Behavior when memory is full | Best for |
|---|---|---|
noeviction | Reject writes with an error, reads still work | Redis used as primary store (not a cache) |
allkeys-lru | Evict least-recently-used key, any key | General-purpose cache |
volatile-lru | Evict LRU among keys with a TTL only | Mixed cache + persistent-data instance |
allkeys-lfu | Evict least-frequently-used key | Cache with skewed, hot-key access patterns |
volatile-ttl | Evict the key with the nearest expiry first | Prioritizing keeping long-lived data |
volatile-random / allkeys-random | Evict a random key | Rarely used; no access-pattern data available |
# redis.conf or CONFIG SET
maxmemory 4gb
maxmemory-policy allkeys-lruIf you run Redis as a pure cache with maxmemory-policy noeviction (the default), you will eventually get OOM command not allowed errors on writes once the instance fills up — Redis will refuse new data rather than evict old data. Always set an explicit maxmemory and an eviction policy that matches your usage (allkeys-lru for a pure cache) before going to production.
7. Persistence: RDB vs AOF
Redis is in-memory, but it is not necessarily volatile — you control how much durability you want, trading it against write throughput and recovery time.
RDB (Redis Database snapshots)
RDB forks the process and writes a compact, point-in-time binary snapshot of the entire dataset to disk on a schedule (or on demand via BGSAVE).
# redis.conf — save if N changes happen within M seconds
save 900 1 # after 900s if at least 1 key changed
save 300 10 # after 300s if at least 10 keys changed
save 60 10000 # after 60s if at least 10000 keys changed
dbfilename dump.rdb
dir /var/lib/redis| Pros | Cons |
|---|---|
| Compact single file — fast to restore, fast to transfer for backups | Data since the last snapshot is lost on crash |
| Minimal runtime overhead (fork uses copy-on-write) | fork() on a large dataset can cause latency spikes |
| Great for disaster recovery / backups | Not real-time durability |
AOF (Append-Only File)
AOF logs every write command to a file, which is replayed on restart to rebuild state. It offers much stronger durability guarantees.
# redis.conf
appendonly yes
appendfsync everysec # fsync once per second — default, good balance
# appendfsync always # fsync every write — safest, slowest
# appendfsync no # let the OS decide — fastest, least safeappendfsync | Durability | Performance impact |
|---|---|---|
always | Near-zero data loss | Significant write latency cost |
everysec | Up to 1 second of writes lost on crash | Recommended default — small overhead |
no | OS-dependent, could lose seconds of data | Fastest, least safe |
Most production setups run both. RDB for fast full-restore and portable backups, AOF for point-in-time durability with everysec fsync. Redis also supports AOF rewrite (BGREWRITEAOF) which compacts the log into the minimal set of commands needed to reproduce the dataset, since a naive append-only log grows unbounded.
In-memory vs durable storage — the real trade-off
| Dimension | Pure in-memory (no persistence) | RDB only | AOF (everysec) | AOF (always) |
|---|---|---|---|---|
| Write latency | Lowest | Low (snapshot is async) | Low-moderate | Highest |
| Data loss on crash | Everything since boot | Since last snapshot | Up to 1 second | Near zero |
| Disk I/O | None | Periodic bursts | Continuous, low | Continuous, high |
| Restart time | N/A (empty) | Fast | Slower (log replay) | Slowest |
| Use as source of truth? | Never | Risky | Acceptable for some | Yes, if truly needed |
Redis is a cache by default mindset, not a database, even with AOF always enabled. Persistence reduces data loss risk but doesn't give you the transactional guarantees, backup tooling maturity, or query flexibility of a real database. If Redis becomes your system of record, budget for Redis Enterprise / Redis Cluster operational maturity, not a single-node "cache" deployment.
8. Connecting from Spring Boot
# application.yml
spring:
data:
redis:
host: redis.internal
port: 6379
password: ${REDIS_PASSWORD}
timeout: 2000ms
lettuce:
pool:
max-active: 16
max-idle: 8
min-idle: 2
shutdown-timeout: 200ms@Configuration
public class RedisConfig {
@Bean
public RedisTemplate<String, Object> redisTemplate(RedisConnectionFactory factory) {
RedisTemplate<String, Object> template = new RedisTemplate<>();
template.setConnectionFactory(factory);
template.setKeySerializer(new StringRedisSerializer());
template.setHashKeySerializer(new StringRedisSerializer());
template.setValueSerializer(new GenericJackson2JsonRedisSerializer());
template.setHashValueSerializer(new GenericJackson2JsonRedisSerializer());
template.afterPropertiesSet();
return template;
}
}Lettuce (Spring Boot's default Redis client) is thread-safe and async under the hood, backed by Netty. A single LettuceConnectionFactory and connection pool is sufficient for most services — you don't need one connection per thread the way older Jedis-based setups often required.
Key takeaways
- Redis is a data structure server, not just a key-value cache — pick the structure (string, hash, list, set, ZSET) that matches your access pattern instead of always serializing JSON into a string.
- Overwriting a key with
SETclears its TTL unless you useKEEPTTLor re-specify the expiry — this is a frequent source of "why did this session never expire" bugs. - Expiry is lazy + active-sampled, not instant — a small number of already-expired keys can briefly still occupy memory until accessed or swept.
- Always set
maxmemoryand an explicit eviction policy (allkeys-lrufor pure caches) — the defaultnoevictionwill reject writes once memory fills up. - RDB gives you fast, compact backups; AOF gives you durability. Run both in production, with
appendfsync everysecas the default durability/performance balance. - Large hashes and lists can block the single-threaded event loop during full reads (
HGETALL,LRANGEon huge collections) — useSCAN-family commands and bounded ranges. - Treat Redis as a cache, not a source of truth, unless you've deliberately invested in Redis's durability and operational tooling to make it one.
Interview Questions
- What are the core Redis data structures, and when would you choose a Hash over a String for caching an object?
- How does Redis expiry actually work — is it a background sweep, lazy deletion, or both?
- What happens if you
SETa key that already has a TTL, without specifying a new expiry? - What eviction policies does Redis support, and which one would you pick for a pure cache versus a mixed-use instance?
- What's the difference between RDB and AOF persistence? What are the trade-offs of each?
- What does
appendfsync everysecmean, and why is it the common default overalwaysorno? - Why is Redis described as "single-threaded," and how does that affect commands like
HGETALLon a very large hash? - How would you implement a leaderboard using Redis data structures?
- What's the risk of running Redis as a system of record instead of a cache?
- How does
maxmemory-policy noevictionbehave once memory is exhausted, and why is that dangerous for an unconfigured cache? - What's the difference between
EXPIRE,PEXPIRE, andEXPIREAT? - Why would you choose Lettuce over Jedis in a modern Spring Boot service?
- How would you safely migrate a large dataset from RDB-only persistence to AOF without downtime?