Cloudflare cut 100 TB of RAM out of 1.1.1.1, and the cache got faster

You have 250 billion things in memory at once. Waste one byte on each and you are down 250 gigabytes. Not on a single box. Across the whole fleet, in one go, because somebody once stored a field the wrong way around.

That is daily life for Big Pineapple, the Rust platform behind 1.1.1.1, Gateway DNS, DNS Firewall and AS112. Sebastiaan Neuteboom of Cloudflare wrote up five optimizations on 27 August 2026. All five change the same thing: how a cache entry is laid out in memory. No new algorithm, no different data structure, no extra hardware. The result is over 50% less memory per entry, roughly 100 terabytes freed across the fleet, and a cache that handles 43% more inserts per second.

That last number is why I read it twice. Saving memory normally costs you speed. Here it did the opposite.

Why does a list that never grows still pay for growth?

The first step is one you can repeat in your own code this afternoon.

A Rust Vec stores three things: a pointer to the heap, the current length, and the capacity. Capacity exists so you can push without reallocating every time. Fine, if you are still going to push.

A DNS answer that has landed in the cache is never modified again. So capacity does nothing and costs 8 bytes per field. Worse, the Vec has already reserved heap space for items that will never arrive. Room for eight, five filled, three sitting there for nothing.

Box<[T]> cannot grow, so it needs no capacity field and no reservation. Box<str> does the same for text. A cache entry had eight such fields, so that is 64 bytes per entry gone. At 250 billion entries, one search and replace is worth more than 15 terabytes.

Three lists became one list plus two 2-byte numbers

A DNS answer has three sections: answer, authority and additional. Those sat in three separate lists, and every list costs an 8-byte pointer plus an 8-byte length.

They merged them into one list with two offsets marking where the second and third section start. Record counts per section fit in a u16, so an offset costs 2 bytes instead of 16. Net saving: 28 bytes per entry.

Watch the tail end of that, because it is the kind of detail that is easy to miss on your own code. Rust inserts padding to satisfy alignment and rounds a struct up to a multiple of its alignment. Removing a small field can therefore free more than the field itself weighs. They also packed several loose booleans into a single bitflag, and the struct shrank by more than the booleans were worth.

The record's name was stored twice

Every DNS record has an owner: the domain the record belongs to. Query example.com A and you get two records with example.com as owner. That name then sits both in the cache key and next to every record.

On the wire this was solved decades ago. RFC 1035 lets you replace a repeated name with a 2-byte pointer. The catch is that following those pointers during a lookup costs time, and the lookup is exactly the path where you do not want to spend it. Cloudflare deliberately trades memory for speed there.

The way out is that the owner is usually identical to the queried domain. So drop it, and restore it from the cache key you already have in hand when you build the response. The field became Option<Box<Name>>. On None, no heap allocation. Only when the owner differs, like the A records behind a CNAME, does the full name still get stored.

The record is no longer self-contained as a result. That is a real concession, and they say so plainly.

NAPTR made every A record pay 120 bytes

This is the best part of the write-up.

A Rust enum is always as large as its biggest variant. RecordData has a variant per record type, and the biggest is NAPTR: three variable-length text fields, a domain name and two integers, 136 bytes together. Add the tag and padding and the whole enum lands on 144 bytes.

An A record needs 4 bytes. An AAAA needs 16. And A and AAAA together are over 80% of the traffic.

Work that through. For the overwhelming majority of records there was roughly 120 bytes of padding sitting ready for a variant that will almost certainly never turn up. A record type you hardly ever see was setting the price of the record type you see constantly. That bug shows up in no test and in no profiler that looks at CPU.

The fix: box the large variants. The enum then holds an 8-byte pointer and the data sits on the heap at exactly the size it needs. For A and AAAA that saves 120 bytes per record. NAPTR itself gets slightly more expensive, since it now pays for a pointer and allocation overhead on top. But NAPTR is rare, so that trade is easy.

They threw their own fix out two steps later

Then came step five, which largely undoes step four. Honestly, that is the most instructive thing in the whole post.

Boxing costs you two things. First, allocator overhead: jemalloc sorts allocations into fixed size bins. A TXT record asks for 32 bytes and fits the 32-byte bin exactly, no waste. An MX record asks for 40 and gets the 48-byte bin, so 8 bytes gone. Second, locality: one entry's data is now scattered across the heap, and every pointer you follow can cost a fresh CPU cache line. At millions of entries that stops being a detail.

So record data is now stored as a single Box<[u8]>: per record a 2-byte length prefix followed by the raw wire bytes. One allocation for all records in the entry, packed contiguously. When building a response, A, AAAA, TXT and every DNSSEC record type get copied straight in, with no field-by-field re-serialization at all. Only CNAME, NS, MX and SOA still go through the parser, because those contain names and need name compression.

The price: you can no longer index straight to the third record, you have to walk the buffer. For round-robin on A records that is extra work, but with a handful of records per entry it disappears into the noise.

They serialize records into a reusable scratch buffer that persists between inserts, then memcpy that into the final box in one go. That change alone was worth 13% more inserts per second.

953 bytes to 420, and 43% in production

The production numbers are lower, and they put that in themselves. Between 18 May and 6 July 2026, p99 resident memory per instance went from 9.3 GB to 5.3 GB, a 43% drop. p90 went from 6.5 GB to 3.8 GB. The gap with the benchmark exists because resident memory is not only the cache.

They also warn that the sharp dips right after each release are not savings but restarts with an empty cache that still has to fill. Only the flat stretches after that mean anything. That is exactly the kind of footnote that makes a blog post less impressive and more trustworthy, and I rarely see it. We went through the same read-the-actual-numbers exercise on what long-context Codex runs really cost, and the pattern is the same: the headline figure and the figure you can plan with are rarely the same number.

So do 130 servers get switched off?

No. And that is the one line in the post where I raise an eyebrow.

"100 terabytes, equivalent to the RAM in 130 Gen 13 servers" is a lovely comparison, but not a single server goes dark. Further down it says so straight out: they plan to spend the freed memory on more cache capacity. More entries, higher hit rate, fewer queries to upstream servers.

That is the better call on the merits. A higher hit rate means faster answers for you and less traffic hitting authoritative servers for everyone else. But "100 terabytes saved" reads like a cost saving, and it is not one. It is the same bill with more cache in it.

Grep your own structs for fields that never grow

The concrete action is small. Walk your hot data structure and look for two things.

One: a Vec or String inside something that never changes after creation. That is 8 dead bytes per field plus unused heap space, and Box<[T]> or Box<str> removes both. Two: an enum whose largest variant is rare. Print std::mem::size_of::<YourEnum>() and compare it to what your most common variant actually needs. If there is a factor of ten between them, your entire hot path is paying for a case that almost never happens.

This is not a Rust problem and not a 250-billion-entries problem. The same mistake lives in every language where you reach for a general-purpose container to hold data that is immutable.

Cloudflare says they are looking at the cache itself next. The moment to watch is the first post that names the new cache size, because only then does it show whether those 100 terabytes bought a higher hit rate or just a nicer memory graph.