Eviction Policies and Memory Management

LRU isn't kicking in because maxmemory-policy allkeys-lru only evicts when a new command tries to allocate memory and would exceed maxmemory. If memory is already at maxmemory but no new writes are attempted, no eviction occurs. The OOM error means you hit maxmemory before eviction could free space. Check CONFIG GET maxmemory-samples (default 5)—low sample sizes mean LRU candidates are chosen from a small pool, missing true LRU keys. Increase to CONFIG SET maxmemory-samples 100. Also check eviction with INFO stats: look for evicted_keys counter. If 0, eviction never triggered. To diagnose: run OBJECT IDLETIME on the 10 old keys—if all show >2592000 (30 days in seconds), they should be candidates. If they're not evicted, verify policy is actually set: CONFIG GET maxmemory-policy should return allkeys-lru. If policy is correct but keys remain, the issue is timing: eviction runs during writes, not on a background schedule. Solution: (1) increase maxmemory to 40GB to buy time, (2) set maxmemory-policy volatile-lru instead to only evict keys with TTL (prevents accidental data loss), (3) proactively delete old keys with SCAN and UNLINK for keys idle >24 hours.

Volatile-ttl is evicting keys too aggressively because most keys have short TTLs (seconds or minutes). During peak traffic, memory pressure intensifies and eviction becomes frequent, causing cache misses on keys you intended to keep. Check INFO stats: evicted_keys should show recent growth. Use MEMORY DOCTOR to analyze key distribution and estimate ideal maxmemory. Root cause analysis: (1) run RANDOMKEY in a loop and call TTL on each to see TTL distribution, (2) use SCAN and collect TTL stats: redis-cli --stat | grep evicted_keys to watch eviction in real-time. Fix: (1) increase maxmemory to reduce pressure (e.g., 64GB instead of 32GB), or (2) change policy to allkeys-lru to keep frequently-accessed keys instead of just evicting by TTL, or (3) reduce TTL values on less-critical keys to focus eviction on those, or (4) implement two-tier caching: hot keys in Redis with short TTLs, cold keys in DynamoDB/memcached. For peak traffic: use MEMORY STATS to find oversized keys (e.g., large hashes/lists) and split them, use pipelining to reduce round-trips and memory churn. Test with redis-benchmark -c 100 -t get,set -q and monitor eviction in real-time.

The issue is memory accounting vs actual allocation. Redis reports memory usage via INFO memory > used_memory_rss (resident set size), but maxmemory is checked against used_memory (logical bytes allocated). If fragmentation is high (used_memory_rss / used_memory > 1.5), Redis might think it has free memory when the OS shows it's swapping. Check CONFIG GET maxmemory-policy and verify it's not noeviction (which never evicts). If policy is correct, the problem is likely: (1) eviction threshold logic: eviction starts BEFORE maxmemory, not after. Set maxmemory-samples 100 and restart redis-server to recalculate eviction candidates. (2) Master-replica lag: if this is a replica, maxmemory enforced differently—replicas can exceed maxmemory during replication. Run INFO replication to check role. (3) Persistence overhead: if bgsave is running (INFO persistence > rdb_bgsave_in_progress: 1), memory temporarily doubles (copy-on-write). Wait for bgsave to finish. Immediate fix: (1) run CONFIG SET maxmemory 40GB to increase headroom, (2) run MEMORY PURGE to clean fragmentation, (3) add keys with expiration: instead of persistent keys, set EXPIRE on session keys (e.g., 1 hour). Long-term: monitor memory before reaching 90%: set up alert when used_memory_rss approaches 90% of maxmemory and trigger preemptive scaling.

LFU tracks frequency at 1-second granularity by default, but after 1 week, old frequency data decays (Redis implements exponential decay to avoid old keys being stuck). Check CONFIG GET lfu-decay-time (default 1): this is minutes between decay ticks. If a key hasn't been accessed in the past few minutes, its frequency counter decays significantly, making it look "cold" even if it was hot historically. Run OBJECT FREQ on multiple keys to check: if all show low freq (1-5), decay is too aggressive. Increase lfu-decay-time: CONFIG SET lfu-decay-time 5 to decay less frequently. Also check lfu-log-factor (default 10): higher values give more granularity to frequency tracking. Increase to CONFIG SET lfu-log-factor 15. These changes take effect immediately on new accesses but don't retroactively update old keys. To reset: (1) manually clear frequency by running OBJECT FREQ on all keys after CONFIG SET changes (won't clear old data though), or (2) implement key versioning: append version suffix to keys (user:123:v2) to treat them as new keys with fresh frequency. For recommendation engine: use separate cache tiers: hot keys (accessed >1/min) in Redis with LFU, medium-frequency keys in L2 cache (e.g., memcached), cold keys in DB. Test with redis-benchmark -c 1000 -t get -q --keyspace 100 to verify LFU behavior under load.

Volatile-random with mostly non-expiring keys is a bad policy—it will evict the same small subset repeatedly. First, identify which keys have TTL: use RANDOMKEY in a loop and check TTL. If >50% show -1 (no TTL), your key population is TTL-deficient. Options: (1) Change policy dynamically: CONFIG SET maxmemory-policy allkeys-lfu to evict all keys by frequency instead of just TTL-expiring ones. This takes effect immediately. (2) Backfill TTLs on existing keys: use SCAN 0 MATCH and for each key, run EXPIRE to give all keys a TTL. For high-volume (millions of keys): use Lua scripting to batch this: EVAL 'local keys = redis.call("keys", ARGV[1]); for _, key in ipairs(keys) do redis.call("expire", key, ARGV[2]) end; return #keys' 0 . (3) Immediately increase maxmemory to 80GB to reduce eviction pressure while you migrate keys. For the 50% cache misses: implement a secondary cache (e.g., Memcached) as L2 for keys evicted from Redis. Log evictions: use INFO stats > evicted_keys to detect when eviction > 10K/sec and trigger alerts. Prevention: enforce TTL policy at app layer—require all keys set with EXPIRE before inserting into Redis.

Redis doesn't natively support per-tenant memory limits. You need application-level enforcement. Implement: (1) Key namespace: prefix all keys with tenant ID (tenant:A:user:123). Use SCAN tenant:A:* and sum MEMORY USAGE for each key to track per-tenant memory. (2) Set eviction per tenant: use RANDOMKEY to find keys under tenant:B:* when Tenant B exceeds quota, then UNLINK those keys (faster than DEL). (3) Use MEMORY STATS and MEMORY USAGE to get per-key breakdown. Build a monitoring loop: every 60 seconds, scan all tenant namespaces and if any tenant exceeds quota by >5%, trigger eviction of least-recent-used keys in that namespace (implement via Lua script). (4) For hard limits: reject SET commands from Tenant B if their namespace memory >= 15GB. Return error to client: "Tenant quota exceeded". Implement via Lua: EVAL 'local tenant = ARGV[1]; local quota = tonumber(ARGV[2]); local keys = redis.call("keys", tenant..":*"); local total = 0; for _, k in ipairs(keys) do total = total + redis.call("memory", "usage", k) end; if total >= quota then return redis.error_reply("Quota exceeded") else return redis.call("set", ...) end' 0 . Better architecture: use Redis 7.0+ which supports XREAD with maxlen for bounded streams. For true multi-tenancy at scale: use separate Redis instances per tenant or Redis Enterprise with multi-tenancy isolation.

The problem is: under extreme load, eviction can't keep up with incoming writes. Each SET allocates memory first, then eviction checks maxmemory after. If allocation is very large (e.g., 1GB key) or requests are very fast (1M/sec), eviction lags behind and OOM occurs. Prevention layers: (1) Set CONFIG maxmemory-reserve 10GB as a hard stop before OOM. This tells Redis to stop accepting writes when memory reaches maxmemory - reserve. Trade-off: less cache, but prevents crash. (2) Use CONFIG SET maxmemory-samples 1000 (higher values = more accurate LRU selection, faster eviction). (3) Set max-clients 100000 to limit connections—fewer connections = fewer concurrent writes = less memory churn. (4) Configure TCP backlog: CONFIG SET tcp-backlog 1024 to queue requests and smooth spikes. (5) Implement rate-limiting at application level: reject requests if QPS > threshold to prevent DDoS amplification. For immediate DDoS mitigation: (1) use iptables to rate-limit incoming traffic (e.g., 10K pps per IP), (2) enable Redis protection: CONFIG SET protected-mode yes to reject unauthenticated clients, (3) deploy Redis behind a reverse proxy (Redis Commander, Envoy) that rate-limits per connection. Test with redis-benchmark -c 1000 -t set -q --pipe to simulate attack and monitor memory growth. Use LATENCY DOCTOR to identify eviction delays during spike.