The Key Tax

Ask a team how much Redis they need and the answer is almost always a dataset size. "We have 40 GB of sessions." That number has very little to do with the RAM that gets bought, and the gap is not a fudge factor. Redis does not store your data. It stores a data structure, one per key, and the structure carries a price that is charged before your bytes are counted at all.

The price has two parts, and they are easy to conflate. The first is bookkeeping: the pointers and headers Redis walks on every lookup, which exist whether the value is one byte or one megabyte. The second is the memory allocator's rounding, which happens after Redis has decided how much to ask for. Neither part scales with your payload, which is what makes the total behave so strangely: the toll is fixed per key, so the cost of a dataset is driven by its key count, not its size.

Ten million 100-byte values and ten thousand 100 KB values are both "one gigabyte of data". Redis will charge the first one about 1.7 GB and the second one almost exactly 1 GB, and nothing about the bytes themselves explains the difference. It is the same gigabyte, split into a thousand times as many keys, and each key pays the toll.

This is the written companion to the Redis In-Memory Key Inflation calculator, which applies that toll to a dataset rather than to a single key. What follows is the part a one-line overhead percentage hides: where the 48 bytes actually go, why the key count and not the key size sets the bill, the packing trick Redis's own manual teaches and the encoding cliff that silently undoes it, why the allocator's rounding is not a rounding error at scale, and why a Redis instance that has shed half its data still reports the same resident set size.

The Floor Under Every Key

A single SET does not create one object. On a 64-bit build of Redis 7 it creates a dictionary entry in the top-level keyspace, a string object to hold the key, and a wrapped object to hold the value — and the string representation used for both is Redis's own SDS, which carries its length and free space in a header rather than scanning for a terminator. None of that is incidental. The dictionary entry is what makes lookups O(1); the headers are what make length a constant-time operation and the strings binary-safe.

Itemised, the fixed part is smaller than most people assume and larger than most budgets allow:

Forty-eight bytes of structure, then your key name, then your value. For a 100-byte value behind a 12-character key that is 112 bytes of user data, and the entry measures around 168 bytes once the allocator has rounded it — which is where the inflation begins. For a 5-byte value behind the same key, the entry still measures about 88 bytes against 17 bytes of data. The structure did not change; the ratio did, because the same toll was divided by a much smaller payload.

This is the reconciliation that most Redis sizing advice skips. You will find the overhead quoted as 1.6× and you will find it quoted as 35%, and both figures are defensible because they describe different value sizes. Measure a 100-byte value and you get something near 1.6×. Measure a dataset of kilobyte-scale values and the same 48 bytes is a few percent, which is where the 35% rule of thumb comes from once the allocator and the small-value tail are mixed in. Neither figure is "the" overhead. The overhead is a fixed toll, and the ratio is that toll divided by whatever you happen to be storing.

The Key Count Is the Bill

Because the toll is per key, the cost of a dataset is a function of how many keys it is spread across. Take one gigabyte of application data and store it three ways — same bytes, different granularity — and the RAM Redis asks for changes by more than half:

The premium column is the toll as a share of the total, and it falls by three orders of magnitude across the table. Read it from the other direction and the same fact becomes a design instruction: the cheapest way to store a gigabyte is in as few keys as the access pattern allows. That is not a novel observation, but it is one that is routinely ignored, because application code is usually written per entity — one key per session, one key per product, one key per request — and the entity boundary becomes the key boundary without anyone choosing it.

Redis's own documentation contains a striking example of the extreme. A bitmap holding one bit per user — subscribed or not — for 100 million users occupies 12 MB. The same membership information stored as 100 million keys of one boolean each would carry 100 million tolls, several gigabytes of structure, to answer a question that a bit already answers. Bitmaps and bitfields are not always the right model, but the comparison is the point: the shape of the data structure chosen sets the key count, and the key count sets the bill.

There is a second-order effect worth naming. The toll is not only memory. Every top-level key is also an entry the server must scan during key eviction, replication, and expiry checks, and every one of them appears in the keyspace notifications stream if notifications are enabled. A dataset with ten million keys is more expensive to operate than the same bytes in ten thousand keys, in ways that show up as latency rather than as a line on a bill.

The Trick the Manual Teaches

If the toll is per key, the obvious move is to use fewer keys — and Redis's own memory-optimisation documentation demonstrates exactly that, with numbers. It benchmarks 100,001 keys in the shape object:102393, each holding a short value, twice: once as flat string keys, and once packed so that a hundred of them share a single hash key.

Two orders of magnitude fewer keys, one order of magnitude less memory, identical data. The mechanism is not compression. Redis uses a compact encoding — a listpack in Redis 7, a ziplist before it — for small hashes, in which fields and values are stored end to end in a single contiguous block with a couple of bytes of length framing per element instead of a dictionary entry, an object header and two SDS headers each. One hash key pays one toll; its hundred fields pay roughly two bytes apiece.

Redis's documentation is explicit about the payoff and the price. The payoff is "up to 10 times less memory, with 5 times less memory used being the average saving" for small aggregate types. The price is capability: because hash fields are not full objects, a hash field cannot have its own time to live, and hashes cannot nest. Packing therefore suits data with a shared lifetime — the fields of one user's session, the attributes of one product, one day's counters for one tenant — and it is the wrong tool the moment individual fields need to expire independently.

There is a third consideration that rarely gets stated: the packing and unpacking is in your application code, and it is where the bugs live. Splitting a key into a hash and a field means every read and write has to split it the same way, and a schema change to the split has to be migrated rather than simply deployed. The memory saving is real and large; so is the coupling.

The Encoding Cliff

Compact encodings are used only while the collection stays small, and the boundary is a configuration value. Cross it by one element and the entire collection converts to a full hash table or skiplist, retroactively, and the per-element cost jumps by roughly a factor of three.

Here is where the widely repeated numbers diverge from the actual defaults. Redis's documentation lists these for Redis 7.0 and later:

Note the second row from the top and the third. The hash default is 512 fields. A large share of the tutorials, cheat sheets and Stack Overflow answers you will find state 128 — and 128 is the sorted-set default, which is listed directly below the hash setting in the same block of redis.conf and in the same table of the documentation. The two numbers sit one line apart and have been transposed for years.

The transposition matters in both directions. Believe the limit is 128 and a perfectly healthy hash that converts to a hashtable at 129 fields looks like a defect. Raise the setting to 512 in the belief that you are doubling it and you have actually made no change, leaving a much larger hash in a single listpack than you intended — and a listpack that large is reallocated on write, which is a CPU cost you did not budget for.

The cliff itself is worth putting a number on. Take a hash of 512 fields holding eight-character field names and twelve-character values. In listpack form each field costs the field, the value and about three bytes of framing — roughly 23 bytes. Convert the same hash to a hashtable and each field now needs a dictionary entry, an SDS for the field and an SDS for the value, plus its share of a bucket array, which lands near 64 to 80 bytes after allocation rounding. One extra field can multiply the memory of that key by three.

Two operational notes follow. First, the conversion is one-way: a hash that has become a hashtable does not return to a listpack when fields are deleted, so a key that briefly exceeded the threshold keeps the expensive encoding until it is rewritten. Second, OBJECT ENCODING key reports the current encoding and is the only reliable way to see which side of the cliff a given key is on — the configuration tells you the threshold, not the outcome.

A version note closes the loop. Redis 7.0 renamed the ziplist directives to listpack and replaced the encoding underneath; Redis 7.2 removed the ziplist spellings entirely and added the set variants. A configuration file carried forward from Redis 6 without review will be parsed with unrecognised directives ignored or the defaults silently in force, which is the most common way to end up running thresholds nobody chose.

The Allocator's Rounding

Everything above describes what Redis asks the allocator for. What the process actually pays is what the allocator hands back, and Redis delegates that decision to jemalloc, which serves requests from a fixed ladder of size classes rather than in arbitrary sizes.

The small-object ladder runs in coarse steps below 128 bytes and finer ones above: 8, 16, 32, 48, 64, 80, 96, 112, 128, 160, 192, 224, 256 bytes, then 320, 384, 448, 512 and onward. A request is rounded up to the next class that fits it, so a 100-byte allocation is served from the 112-byte class and twelve bytes are stranded; a 200-byte allocation lands in the 224-byte class and twenty-four bytes are stranded. The internal fragmentation is about 12% at that scale and it never reaches zero, because the only request sizes that fit exactly are the class boundaries themselves.

Twelve percent sounds like the kind of thing that belongs in a footnote. It belongs in the budget instead, for the simple reason that the rounding applies to every allocation, and a Redis key with the layout described above is four to five separate allocations. A dataset whose raw bytes sum to 1.2 GB can land near 1.4 GB of resident memory before a single byte of genuine fragmentation has been accounted for, and that figure is stable rather than transient — it is not going to be reclaimed by a restart or a defragmentation pass, because nothing is wrong.

This is also why the overhead figures people measure are consistently higher than the sums of their components. A calculation that adds a dictionary entry and two object headers and stops will understate the bill, and the missing term is the allocator, not an error in the arithmetic.

Why RSS Outlives Your Data

The number a Redis process reports to the operating system and the number Redis believes it has in use are two different quantities, and confusing them is the most common cause of a mis-sized instance.

used_memory is what Redis thinks it has allocated. used_memory_rss is what the operating system has actually given the process. Their ratio is published as mem_fragmentation_ratio, and the RSS side does not come down the way the first does: when keys are deleted, the allocator marks pages free and reuses them for future writes, but it does not return them to the kernel. The documented consequence is blunt — fill an instance with 5 GB, delete 2 GB, and RSS typically stays near 5 GB while Redis reports roughly 3 GB in use.

Two practices follow, and both run against intuition. The first is that an instance must be sized for its peak, not its average. If the workload needs 10 GB at year-end and 5 GB the rest of the year, the correct instance holds 10 GB, because the process will have grown into the pages and will keep them. The second is that mem_fragmentation_ratio stops meaning what its name suggests after any peak. It is defined as RSS divided by current usage, so a period of high usage followed by deletion produces a ratio well above 1 without any fragmentation having occurred — the denominator fell, not the numerator.

Redis reports a more honest split under two less-quoted fields. used_memory_overhead is the memory consumed by the server's internal structures — the keyspace dictionaries, client buffers, replication buffers, the event loop — and used_memory_dataset is the remainder, with used_memory_dataset_perc giving the dataset's share of the whole. On a small instance that percentage is routinely in the twenties, which is the same toll described earlier seen from the inside: at low volumes, the bookkeeping is most of the process.

One more term belongs on the peak, and it is specific to persistence. Fork-based background saving relies on copy-on-write, and the worst case — every page touched while the child writes — can briefly double resident memory. An instance running at 90% of its host's RAM is therefore not a Redis instance with 10% headroom; it is an instance that can exhaust the host during an automatic save, which is a failure mode that appears in discovery precisely because it waits for the busiest hour.

Measure, Then Multiply

Sizing Redis correctly does not require a spreadsheet of struct offsets. It requires four commands and a discipline about which number to believe.

1. Measure a real key. MEMORY USAGE key returns the bytes attributable to that key, including its value and headers, sampled. Run it across a representative set rather than one example, because the size distribution is what drives the total. For structures, MEMORY USAGE key SAMPLES 0 forces a full traversal instead of a sample.
2. Confirm the encoding. OBJECT ENCODING key answers the question the configuration cannot: is this collection still compact, or has it already crossed a threshold? A hash reporting hashtable where you expected listpack is a key that is costing several times what it should.
3. Read the split, not the total. INFO memory gives used_memory, used_memory_overhead, used_memory_dataset and the RSS fields. The total tells you what you are paying; the split tells you whether you are paying for your data or for the structure around it.
4. Find the outliers before the average reaches you. A sampling scan of the keyspace surfaces the keys that dominate memory, which are usually few and usually the result of an encoding that flipped. A single large hash that converted to a hashtable can be a larger line item than a million small ones.

Then multiply. Take the peak key count, the measured entry size, and the safety margin the workload earns — more for bursty writes and replicas, less for a steady read-mostly cache — and set maxmemory below the instance size, because the alternative to a configured limit is an eviction storm or an out-of-memory write failure at the worst possible moment. The calculator paired with this briefing applies the toll to a dataset and returns the required RAM, the pure overhead, and a recommended maxmemory with headroom.

The point of the whole exercise is that the answer is not the dataset size plus a percentage pulled from an article. It is the dataset size plus a per-key toll multiplied by the number of keys you chose, rounded by an allocator that does not care what the data means. Every one of those terms is measurable, and the largest of them is usually the one nobody counts.

Frequently Asked Questions

Why does a 100-byte value take more than 100 bytes in Redis?

Because Redis allocates a structure around the value. Every top-level key carries a dictionary entry (24 bytes), a redisObject header for the value (16 bytes), an SDS header plus terminator for the key and again for the value (4 bytes each at short lengths), and a share of the bucket array. That is 48 bytes of bookkeeping before the key name or the value is counted, and the allocator then rounds each of those allocations up to a size class. A 100-byte value with a 12-character key measures roughly 168 bytes, not 112.

How much memory does a million Redis keys use?

It depends almost entirely on the value size, because the per-key toll is fixed. A million keys with 100-byte values cost approximately 168 MB, of which about 48 MB is bookkeeping. A million keys with 1 KB values cost approximately 1.08 GB, and the same 48 MB is now about 4% of the total. Ten million keys with 100-byte values cost roughly 1.7 GB for 1 GB of data; the same gigabyte as ten thousand 100 KB values costs almost exactly 1 GB. Key count is the multiplier.

Is hash-max-listpack-entries 128 or 512?

512. Redis's documentation lists hash-max-listpack-entries 512 and hash-max-listpack-value 64 for Redis 7.0 and later, and the same 512 for the Redis 6.2 and earlier ziplist spelling. The 128 that appears in a large share of tutorials is the sorted-set default, zset-max-listpack-entries, which sits directly below the hash setting in the same configuration block. Believing 128 makes a hash that converts at 129 fields look like a bug; setting 512 to "double" it changes nothing at all.

Why does Redis memory not drop after deleting keys?

Because the allocator does not return memory to the operating system. When keys are removed the pages are marked free and reused for later writes, but RSS stays where it was — a 5 GB instance that drops to 3 GB of live data usually still reports about 5 GB resident. This is why instances are sized for peak usage rather than steady state, and why mem_fragmentation_ratio becomes meaningless after any peak much larger than the current workload: it is RSS divided by current usage, and it is the denominator that fell.

Methodology & Disclosure

The per-key cost model in this briefing is stated against a 64-bit build of Redis 7 with 64-bit pointers, and its components are the documented structure sizes rather than measurements of any particular deployment: a keyspace dictEntry of 24 bytes (three pointers — key, value, chain link), a redisObject header of 16 bytes on the value, and an SDS header of 3 bytes plus a one-byte terminator for any string shorter than 256 bytes (the sdshdr8 layout introduced in Redis 3.2; earlier versions used an 8-byte header, which is the origin of the 9-byte header figure still quoted widely). String values at or below 44 bytes are stored with the object header and the string in one allocation under the embstr encoding and above it under raw; integers use the int encoding and skip the value string entirely. The hash-table bucket term is amortised at roughly 8 bytes per key, reflecting the bucket array's occupancy band rather than an exact load factor.

Allocator rounding follows jemalloc's small-object size-class ladder (8, 16, 32, 48, 64, 80, 96, 112, 128, 160, 192, 224, 256, 320, 384, 448, 512 bytes and upward at progressively coarser steps); every allocation is rounded up to the next class, so a 100-byte request consumes 112 bytes and a 200-byte request consumes 224. Per-entry figures in the tables are model outputs from those components, rounded, and are intended to be re-derived rather than quoted. Published comparisons — the 11 MB versus 1.7 MB packing benchmark, the 12 MB bitmap for 100 million users, the "up to 10 times less memory, 5 times on average" range for compact encodings, and the encoding defaults by version — are taken from Redis's own documentation and configuration reference. No figure here is a measurement of a specific production instance, and measured values differ from the model by a few percent in either direction depending on Redis version, allocator build, key-length distribution and value-size distribution. This site takes no sponsorship, no affiliate commission and no vendor payment of any kind; the projects named here have no relationship with jslet and did not review this briefing.

References & Further Reading

1. Redis. Memory optimization — the compact-encoding defaults by version (hash-max-listpack-entries 512, zset-max-listpack-entries 128, set-max-intset-entries 512, and the Redis 7.2 set listpack directives), the "up to 10 times less memory, 5 times on average" range, the 100-million-user bitmap example, and the 100,001-key packing benchmark (11 MB as string keys versus 1.7 MB packed into 100-field hashes). redis.io
2. Redis. Key eviction — maxmemory configuration, eviction policies, and the behaviour of the instance as it approaches the configured limit. redis.io
3. Redis. MEMORY USAGE — per-key memory attribution, the SAMPLES option, and what the returned byte count includes. redis.io
4. Redis. INFO — the memory section: used_memory, used_memory_rss, mem_fragmentation_ratio, used_memory_overhead, used_memory_dataset and used_memory_dataset_perc. redis.io
5. Redis. OBJECT ENCODING — reported encodings per type (listpack, hashtable, intset, skiplist, embstr, raw, int) and the one-way nature of encoding conversion. redis.io
6. Redis source. src/server.h, src/dict.h, src/sds.h — the structure definitions behind the byte figures: redisObject, dictEntry, and the sdshdr5 through sdshdr64 header layouts. github.com/redis/redis
7. Jemalloc. Size classes — the small-object allocation ladder that governs internal fragmentation for every Redis allocation. jemalloc.net
8. Related: Redis In-Memory Key Inflation Calculator · DB Instance Sizing · Database Rows to Storage · Token Size to Storage · The Attention Gap · Container Resource Limits · All Data & Storage Tools

📜 Copyright & Attribution