06-caching-locking-performance

Redis Core Concepts: Data Structures, TTLs, and Persistence

A staff-engineer guide to Redis data structures, expiry semantics, and RDB/AOF persistence trade-offs for backend caching layers.

August 14, 2026
backend-engineerredisttlcachedata-structurespersistence

Redis Core Concepts

Redis is the default answer to "how do we make this fast" in almost every backend system — session storage, hot-path caching, leaderboards, rate limiters, and pub/sub fan-out all lean on the same small set of primitives. This guide covers those primitives in depth: the data structures, how expiry actually works under the hood, and the durability trade-offs you're implicitly making the moment you point production traffic at a Redis instance.


1. What Redis Actually Is

Redis is an in-memory data structure server. That phrase matters more than "cache" — Redis is not just a key-value store with a TTL bolted on. It exposes rich, typed data structures (strings, hashes, lists, sets, sorted sets, streams, bitmaps, HyperLogLogs) and operates on them with atomic, single-threaded command execution.

💡

Single-threaded, but not slow. The core command execution loop in Redis runs on one thread, which is what makes operations like INCR and LPUSH atomic without locks. Redis 6+ added I/O threading for network reads/writes, but command execution itself stays single-threaded. This is a deliberate trade-off: predictable latency over parallel throughput.

Why teams reach for Redis

Use caseData structureWhy Redis fits
Session storageHash / String (JSON)Sub-millisecond reads, native TTL
Cache layer in front of a DBStringSimple get/set, TTL-based eviction
Rate limitingString (counter) / Sorted SetAtomic INCR, sliding window via ZSET
LeaderboardsSorted SetO(log N) ranked inserts, ZRANGE by score
Distributed locksString (SET NX PX)Atomic conditional set with expiry
Real-time fan-outPub/SubLightweight, no persistence overhead
Deduplication at scaleHyperLogLog / SetApproximate or exact cardinality

2. Strings

The STRING type is Redis's simplest and most-used data structure — a binary-safe byte sequence up to 512MB. It's used for plain values, serialized JSON, counters, and bitmaps.

bash
# Basic set/get
SET user:1001:name "Alicia Keys"
GET user:1001:name
# → "Alicia Keys"
 
# Set with expiry in one atomic command
SET session:abc123 "{\"userId\":1001}" EX 1800
 
# Atomic counters — no read-modify-write race
INCR page:home:views
INCRBY inventory:sku-42 -5
INCRBYFLOAT wallet:1001:balance 19.99
 
# Conditional writes — the basis of distributed locks
SET lock:order:5001 "worker-7" NX PX 10000

With Spring Data Redis (Lettuce client, Spring Boot 3.x), the equivalent looks like:

java
@Service
public class ProductCacheService {
 
    private final StringRedisTemplate redisTemplate;
 
    public ProductCacheService(StringRedisTemplate redisTemplate) {
        this.redisTemplate = redisTemplate;
    }
 
    public void cacheProduct(String productId, String json) {
        redisTemplate.opsForValue()
            .set("product:" + productId, json, Duration.ofMinutes(15));
    }
 
    public Optional<String> getCachedProduct(String productId) {
        return Optional.ofNullable(
            redisTemplate.opsForValue().get("product:" + productId));
    }
 
    public long incrementViewCount(String productId) {
        return redisTemplate.opsForValue()
            .increment("product:" + productId + ":views");
    }
}

Prefer StringRedisTemplate over the generic RedisTemplate<Object, Object> for simple string/JSON payloads. It uses StringRedisSerializer for both keys and values by default, avoiding the Java-serialization or JDK-default serializer footguns that produce unreadable binary blobs in redis-cli.


3. Hashes

A HASH maps field names to values within a single key — effectively a mini object. Hashes are the right structure when you have several related fields you want to read/write independently without deserializing an entire JSON blob.

bash
HSET user:1001 name "Alicia Keys" email "alicia@example.com" plan "pro"
HGET user:1001 email
HGETALL user:1001
HINCRBY user:1001 loginCount 1
HDEL user:1001 plan
java
@Service
public class UserProfileCacheService {
 
    private final RedisTemplate<String, Object> redisTemplate;
 
    public void cacheProfile(String userId, Map<String, Object> fields) {
        String key = "user:" + userId;
        redisTemplate.opsForHash().putAll(key, fields);
        redisTemplate.expire(key, Duration.ofHours(6));
    }
 
    public Object getField(String userId, String field) {
        return redisTemplate.opsForHash().get("user:" + userId, field);
    }
 
    public Long incrementLoginCount(String userId) {
        return redisTemplate.opsForHash()
            .increment("user:" + userId, "loginCount", 1);
    }
}
⚠️

HGETALL on a large hash blocks other clients for the duration of the response serialization, since Redis is single-threaded. Avoid hashes with thousands of fields — if you need that, consider bucketing into multiple keys or using HSCAN for incremental iteration instead of a single HGETALL.

String (JSON blob) vs Hash — when to use which

AspectString (serialized JSON)Hash
Partial field updateRequires read-modify-write of whole blobHSET updates one field atomically
Memory overheadLower for small objectsSlightly higher per-field overhead
Field-level TTLNot possible (Redis < 7.4)Not possible either — TTL is per-key
Read patternAlways deserialize whole objectFetch only needed fields with HMGET
Best forImmutable or rarely-updated blobsObjects with independently-updated fields

4. Lists

LIST is a doubly linked list of strings, ordered by insertion. Lists are the backbone of simple queues, activity feeds, and recent-items caches.

bash
LPUSH notifications:1001 "New comment on your post"
RPUSH activity:feed:1001 "logged in"
LRANGE activity:feed:1001 0 9    # latest 10 items
LTRIM activity:feed:1001 0 99    # keep only the newest 100
BLPOP queue:jobs 5                # blocking pop, 5s timeout
java
public void pushRecentActivity(String userId, String activity) {
    String key = "activity:feed:" + userId;
    redisTemplate.opsForList().leftPush(key, activity);
    redisTemplate.opsForList().trim(key, 0, 99);   // cap the feed size
    redisTemplate.expire(key, Duration.ofDays(30));
}
 
public List<Object> getRecentActivity(String userId, int count) {
    return redisTemplate.opsForList().range("activity:feed:" + userId, 0, count - 1);
}
⚠️

Redis lists are not a message broker. BLPOP/BRPOP give you a simple work queue, but there's no consumer groups, no acknowledgment/redelivery, and no persistence guarantee beyond whatever durability you've configured. For anything beyond a lightweight job queue, use a real broker (Kafka, RabbitMQ, or Redis Streams if you want to stay in the Redis ecosystem).


5. Sets and Sorted Sets

A SET is an unordered collection of unique strings — ideal for membership checks, tagging, and set algebra (union/intersection/difference). A SORTED SET (ZSET) adds a floating-point score to each member, keeping the collection ordered by score with O(log N) inserts.

bash
# Set — membership and set algebra
SADD user:1001:roles "admin" "billing"
SISMEMBER user:1001:roles "admin"      # → 1
SINTER user:1001:roles user:1002:roles  # shared roles
 
# Sorted set — leaderboard
ZADD leaderboard:global 15420 "player:88"
ZADD leaderboard:global 18990 "player:12"
ZREVRANGE leaderboard:global 0 9 WITHSCORES   # top 10
ZRANK leaderboard:global "player:88"           # rank of a specific player
ZINCRBY leaderboard:global 100 "player:88"     # atomic score bump
java
@Service
public class LeaderboardService {
 
    private final ZSetOperations<String, String> zSetOps;
 
    public LeaderboardService(RedisTemplate<String, String> redisTemplate) {
        this.zSetOps = redisTemplate.opsForZSet();
    }
 
    public void recordScore(String playerId, double score) {
        zSetOps.add("leaderboard:global", playerId, score);
    }
 
    public Set<ZSetOperations.TypedTuple<String>> topPlayers(int count) {
        return zSetOps.reverseRangeWithScores("leaderboard:global", 0, count - 1);
    }
 
    public Long rankOf(String playerId) {
        Long rank = zSetOps.reverseRank("leaderboard:global", playerId);
        return rank == null ? null : rank + 1;  // 0-indexed → human rank
    }
}

Sorted sets are the standard building block for sliding-window rate limiters. Store request timestamps as scores, ZREMRANGEBYSCORE to evict entries older than the window, and ZCARD to count requests in the current window. We cover the full pattern in the distributed locks and rate limiting guide.

Data structure decision table

NeedStructureKey commands
Simple value / counter / lockStringSET, GET, INCR
Object with multiple fieldsHashHSET, HGETALL, HINCRBY
Ordered, bounded feed or queueListLPUSH, LRANGE, LTRIM
Unique tags / membership / set opsSetSADD, SISMEMBER, SINTER
Ranked data / sliding windowsSorted SetZADD, ZRANGE, ZREMRANGEBYSCORE
Append-only event log with consumer groupsStreamXADD, XREADGROUP, XACK
Approximate unique counting at scaleHyperLogLogPFADD, PFCOUNT

6. TTLs and Expiry Semantics

Every Redis key can carry an optional time-to-live. Understanding exactly how expiry is enforced matters — it isn't a background cron job deleting keys on a schedule; it's a hybrid of lazy and active mechanisms.

Expiry commands

bash
EXPIRE session:abc123 1800         # set TTL in seconds
PEXPIRE lock:order:5001 10000      # set TTL in milliseconds
TTL session:abc123                 # remaining seconds (-1 = no TTL, -2 = doesn't exist)
PERSIST session:abc123             # remove TTL, key lives forever
EXPIREAT session:abc123 1755200000 # expire at a specific Unix timestamp
⚠️

A common production bug: overwriting a key with SET silently clears its TTL unless you re-specify it. SET session:abc123 newValue — with no EX/PX — turns your session key permanent. Use SET key val KEEPTTL (Redis 6+) if you want to update the value without touching the expiry, or always pass the TTL explicitly on every write.

java
// Spring Data Redis — TTL-safe update pattern
public void refreshSessionData(String sessionId, String newPayload) {
    redisTemplate.opsForValue().set(
        "session:" + sessionId,
        newPayload,
        Duration.ofMinutes(30)   // always re-assert TTL explicitly
    );
}

Eviction policies when memory fills up

TTL only removes keys that have an expiry. When Redis hits maxmemory, the eviction policy decides what happens next — and the default (noeviction) is almost never what you want for a cache.

PolicyBehavior when memory is fullBest for
noevictionReject writes with an error, reads still workRedis used as primary store (not a cache)
allkeys-lruEvict least-recently-used key, any keyGeneral-purpose cache
volatile-lruEvict LRU among keys with a TTL onlyMixed cache + persistent-data instance
allkeys-lfuEvict least-frequently-used keyCache with skewed, hot-key access patterns
volatile-ttlEvict the key with the nearest expiry firstPrioritizing keeping long-lived data
volatile-random / allkeys-randomEvict a random keyRarely used; no access-pattern data available
bash
# redis.conf or CONFIG SET
maxmemory 4gb
maxmemory-policy allkeys-lru
🚨

If you run Redis as a pure cache with maxmemory-policy noeviction (the default), you will eventually get OOM command not allowed errors on writes once the instance fills up — Redis will refuse new data rather than evict old data. Always set an explicit maxmemory and an eviction policy that matches your usage (allkeys-lru for a pure cache) before going to production.


7. Persistence: RDB vs AOF

Redis is in-memory, but it is not necessarily volatile — you control how much durability you want, trading it against write throughput and recovery time.

RDB (Redis Database snapshots)

RDB forks the process and writes a compact, point-in-time binary snapshot of the entire dataset to disk on a schedule (or on demand via BGSAVE).

bash
# redis.conf — save if N changes happen within M seconds
save 900 1        # after 900s if at least 1 key changed
save 300 10        # after 300s if at least 10 keys changed
save 60 10000       # after 60s if at least 10000 keys changed
 
dbfilename dump.rdb
dir /var/lib/redis
ProsCons
Compact single file — fast to restore, fast to transfer for backupsData since the last snapshot is lost on crash
Minimal runtime overhead (fork uses copy-on-write)fork() on a large dataset can cause latency spikes
Great for disaster recovery / backupsNot real-time durability

AOF (Append-Only File)

AOF logs every write command to a file, which is replayed on restart to rebuild state. It offers much stronger durability guarantees.

bash
# redis.conf
appendonly yes
appendfsync everysec   # fsync once per second — default, good balance
# appendfsync always   # fsync every write — safest, slowest
# appendfsync no       # let the OS decide — fastest, least safe
appendfsyncDurabilityPerformance impact
alwaysNear-zero data lossSignificant write latency cost
everysecUp to 1 second of writes lost on crashRecommended default — small overhead
noOS-dependent, could lose seconds of dataFastest, least safe
💡

Most production setups run both. RDB for fast full-restore and portable backups, AOF for point-in-time durability with everysec fsync. Redis also supports AOF rewrite (BGREWRITEAOF) which compacts the log into the minimal set of commands needed to reproduce the dataset, since a naive append-only log grows unbounded.

In-memory vs durable storage — the real trade-off

DimensionPure in-memory (no persistence)RDB onlyAOF (everysec)AOF (always)
Write latencyLowestLow (snapshot is async)Low-moderateHighest
Data loss on crashEverything since bootSince last snapshotUp to 1 secondNear zero
Disk I/ONonePeriodic burstsContinuous, lowContinuous, high
Restart timeN/A (empty)FastSlower (log replay)Slowest
Use as source of truth?NeverRiskyAcceptable for someYes, if truly needed
🚨

Redis is a cache by default mindset, not a database, even with AOF always enabled. Persistence reduces data loss risk but doesn't give you the transactional guarantees, backup tooling maturity, or query flexibility of a real database. If Redis becomes your system of record, budget for Redis Enterprise / Redis Cluster operational maturity, not a single-node "cache" deployment.


8. Connecting from Spring Boot

yaml
# application.yml
spring:
  data:
    redis:
      host: redis.internal
      port: 6379
      password: ${REDIS_PASSWORD}
      timeout: 2000ms
      lettuce:
        pool:
          max-active: 16
          max-idle: 8
          min-idle: 2
        shutdown-timeout: 200ms
java
@Configuration
public class RedisConfig {
 
    @Bean
    public RedisTemplate<String, Object> redisTemplate(RedisConnectionFactory factory) {
        RedisTemplate<String, Object> template = new RedisTemplate<>();
        template.setConnectionFactory(factory);
        template.setKeySerializer(new StringRedisSerializer());
        template.setHashKeySerializer(new StringRedisSerializer());
        template.setValueSerializer(new GenericJackson2JsonRedisSerializer());
        template.setHashValueSerializer(new GenericJackson2JsonRedisSerializer());
        template.afterPropertiesSet();
        return template;
    }
}

Lettuce (Spring Boot's default Redis client) is thread-safe and async under the hood, backed by Netty. A single LettuceConnectionFactory and connection pool is sufficient for most services — you don't need one connection per thread the way older Jedis-based setups often required.


Key takeaways

  • Redis is a data structure server, not just a key-value cache — pick the structure (string, hash, list, set, ZSET) that matches your access pattern instead of always serializing JSON into a string.
  • Overwriting a key with SET clears its TTL unless you use KEEPTTL or re-specify the expiry — this is a frequent source of "why did this session never expire" bugs.
  • Expiry is lazy + active-sampled, not instant — a small number of already-expired keys can briefly still occupy memory until accessed or swept.
  • Always set maxmemory and an explicit eviction policy (allkeys-lru for pure caches) — the default noeviction will reject writes once memory fills up.
  • RDB gives you fast, compact backups; AOF gives you durability. Run both in production, with appendfsync everysec as the default durability/performance balance.
  • Large hashes and lists can block the single-threaded event loop during full reads (HGETALL, LRANGE on huge collections) — use SCAN-family commands and bounded ranges.
  • Treat Redis as a cache, not a source of truth, unless you've deliberately invested in Redis's durability and operational tooling to make it one.

Interview Questions

  • What are the core Redis data structures, and when would you choose a Hash over a String for caching an object?
  • How does Redis expiry actually work — is it a background sweep, lazy deletion, or both?
  • What happens if you SET a key that already has a TTL, without specifying a new expiry?
  • What eviction policies does Redis support, and which one would you pick for a pure cache versus a mixed-use instance?
  • What's the difference between RDB and AOF persistence? What are the trade-offs of each?
  • What does appendfsync everysec mean, and why is it the common default over always or no?
  • Why is Redis described as "single-threaded," and how does that affect commands like HGETALL on a very large hash?
  • How would you implement a leaderboard using Redis data structures?
  • What's the risk of running Redis as a system of record instead of a cache?
  • How does maxmemory-policy noeviction behave once memory is exhausted, and why is that dangerous for an unconfigured cache?
  • What's the difference between EXPIRE, PEXPIRE, and EXPIREAT?
  • Why would you choose Lettuce over Jedis in a modern Spring Boot service?
  • How would you safely migrate a large dataset from RDB-only persistence to AOF without downtime?