mightsleep

Two dimensions, one disk

2026-10-25T00:00:00+00:00

This series started as a side question. I had just written an allocator for GPU memory, which packs buffers into one long line, and wondered why the line had to be one-dimensional. Two dimensions led to Morton order, Morton picked up a few ideas from image recognition, and soon I was solving Morton with bit tricks instead of packing anything. The packing can wait.

Take a table with two columns you filter on: a position on a map, or a time and a sensor id. The question is always the same: give me everything inside this rectangle. Memory, a file and an index all have one dimension, so before anything can be stored, someone has to decide in which order the cells of a two-dimensional grid go into a line. This post is about that decision. It opens a short series; the posts after it compute the orders fast and measure what they cost.

Rows

The obvious order is the one every array uses, row after row. On an 8 × 8 grid, cell (x, y) gets number y · 8 + x.

Row by row, with a 4 × 4 query marked. The line follows the numbers; it jumps back at the end of every row.

The marked square holds numbers 18 to 21, 26 to 29, 34 to 37 and 42 to 45: four contiguous stretches of the index. I will call such a stretch a run. Each run means jumping to a new place in the index and reading from there, so a run is roughly a seek. In this order a rectangle costs one run per row it covers, whatever its width, so a query one column wide and a hundred tall costs a hundred.

That is only half of the cost, though. A disk hands out pages, blocks of consecutive keys, and a database reads them whole. In rows, a page is a strip, a piece of one row. A square query cuts through as many strips as it has rows and uses only the few cells of each strip that fall inside it.

A curve

A space-filling curve is just another order: a path through the grid that visits every cell once, chosen so that cells close in the order are also close on the grid. The name sounds like geometry, but all that matters here is the numbering.

Z-order

The cheapest such order interleaves the bits of the two coordinates like a zipper: the lowest bit of x, then the lowest bit of y, then the next bit of x, and so on. The cell x = 2, y = 1 is x = 10 and y = 01 in binary; interleaved, that is 0110, so the cell gets number 6. This is Z-order, also called Morton order after G. M. Morton, who described it for a map database at IBM in 1966.

Z-order on a 4 × 4 grid. Every 2 × 2 block is a small Z, and the four blocks make a large one.

Every 2 × 2 block gets four consecutive numbers, every aligned 4 × 4 block sixteen, and so on up. That is the whole point: an aligned square of any size is one contiguous stretch of the index. The price is the jumps between blocks, the diagonal strokes of the Z, and some of them are long.

Hilbert

David Hilbert's curve, from 1891, is built from the same blocks, but each block is turned so that the path leaves it right next to where the next block starts. It never jumps: consecutive numbers are always neighbouring cells.

Hilbert's curve on the same grid. Each quarter is a turned copy of the 2 × 2 pattern, so the path never lifts.

It is also harder to compute. Z-order is a bit shuffle, while Hilbert has to know at every level how the block it is in is turned, and that is a chain of decisions, each depending on the last. Making that cheap is the subject of the next post.

What a square costs

Here are both numbers for a setup that looks more like a database than a toy: a 256 × 256 grid, pages of 256 cells, and a 16 × 16 square query, averaged over all 58 081 positions the square can take.

order	runs	pages read	cells read for 256 wanted
rows	16.0	16.0	4096
Z-order	30.0	3.7	957
Hilbert	15.9	3.7	957

Pages are where the curves win, and both win by the same margin. On a curve a page is a block close to a square, because an aligned square of the grid is a contiguous stretch of keys, and a square query cuts through only a few such blocks. Reading a quarter of the data is a real saving.

Pages of 16 cells, outlined: whole rows on the left, 4 × 4 blocks on Hilbert's curve on the right (Z-order's pages are the same blocks). The 8 × 8 query touches the shaded pages, eight against four. It is aligned to the blocks here, which flatters the curve; the table averages every position.

Runs are a different story. Hilbert only ties with plain rows, and Z-order needs almost twice as many: every jump between blocks starts a new run. For a square, a curve does not mean fewer seeks. Whether runs or pages decide what a query really costs, and whether something between Z-order and Hilbert does better, is what the rest of this series measures.

Try it

Below are three orders: Z-order, Hilbert, and one between them that gets a post of its own. Drag a rectangle across the grid; the curve outside it dims and the runs inside stay lit, so you can count them.

Morton (Z-order), Inside-out and Hilbert, computed in Rust compiled to WebAssembly. Drag to select a rectangle; the curve outside it dims, so the runs stay lit.

Where this lives

Geohash is Z-order: the bits of latitude and longitude, interleaved and written in base 32, so that nearby places share a prefix. Delta Lake can sort the files of a table along it, OPTIMIZE ... ZORDER BY, so that a filter on either column skips most files. Google's S2 library numbers cells on the faces of a cube along Hilbert's curve. And GPUs lay textures out in tiles much like Z-order, because a triangle reads a small square of texels, not a row.

Three questions for the posts that follow:

How to compute Hilbert almost as cheaply as Z-order: one constant, and sixty-four keys in one instruction.
A curve between the two that takes away a third of Z-order's runs for one shift and two XORs.
Whether fewer runs make a cheaper query. They do, by less than the count suggests.

Method

The numbers in the table come from a short script that numbers the grid in each order, collects the keys of every 16 × 16 square, sorts them, and counts runs as gaps between consecutive keys and pages as distinct values of key / 256. The same script draws the figures, so the pictures and the table cannot disagree. Hilbert is the classic construction, the same one the demo uses.

Four lanes, one CAS

2026-10-11T00:00:00+00:00

The first post mentioned a router I wrote over a weekend: it sits on port 443, reads the server name from the TLS handshake and passes the connection on without decrypting anything. A week later it has grown IPv6, a plaintext port, the PROXY protocol, metrics, and three kinds of connection limits, two of which are small sketches packed four lanes to a 64-bit word. This post goes through all of it, including the parts that broke.

Why a router at all

The server has one public address. Two things want its port 443: this site, and a tunnel that exists because some networks I sit in let out TCP on 80 and 443 and nothing else. Whoever connects to 443 says, in the first message of the TLS handshake, which name it wants (Server Name Indication, SNI). That is enough to tell the two apart before any cryptography happens. Unknown names get the website, so somebody poking at the tunnel's address sees an ordinary web server.

The first post already told how this almost went wrong: the web server behind the router picks a virtual host by the HTTP Host header, not by SNI, so "public site" in the handshake and "internal dashboard" in the request would have served the dashboard to the internet. The public site now runs on its own instance that knows nothing else.

Where is layer 4, anyway

Load balancer vendors sort themselves into "L4" and "L7", and this router would be filed under L4. I have never found the layers much use, and not only because the network ignores them.

Start with the hardware. A network card does the physical layer and the MAC, computes IP and TCP checksums, cuts TCP segments for the kernel (TSO), hashes the addresses and ports to spread packets over its queues (RSS), and newer ones encrypt TLS records. One chip, five layers.

Then debugging, where problems refuse to stay on one floor. A DPI-evasion tool on my laptop once split the ClientHello across TCP segments, which is layer 4, and the TLS library on the other end refused the handshake, which is layer 7. A firewall that drops ICMP, layer 3, makes a TCP connection hang the first time it sends a large packet. Further down this post a web server's HTTP redirect depends on a port number from a header in front of the TCP stream. You switch between layers all the time, so they do not even work as a filing system for bugs.

And the stack is not seven floors. The tunnel behind this router carries IP inside WireGuard inside WebSocket inside TLS inside TCP inside IP: layer 3 inside layer 7. A fixed list of seven floors cannot say that. What the wire carries is envelopes inside envelopes: a frame is a header followed either by data or by another frame, as deep as anyone likes. A "layer" is just how many envelopes deep you are.

What the layers are good for is a contract about encapsulation: who owns which header and who may change it. That contract is why IPv6 could slide in under an unchanged TCP and why QUIC could be built on UDP. It is a model for designing protocols, not for where the work happens or where the failures hide. Even as a contract it gets broken: middleboxes read into TCP, so new transport protocols like SCTP never made it through the internet, and QUIC encrypts its own headers so that nobody can touch them.

The model I actually use is where on the wire is the packet, and what happens to it there. A packet travels a chain of places, and each place is a queue, a transformation and a decision:

application buffer → socket send queue → qdisc → driver ring
  → NIC (TSO, checksums) → wire → switches → … → NIC (RSS)
  → driver ring → NAPI/GRO → TCP receive queue → application

The tools are built for this view: ss for socket queues, tc -s for qdiscs, ethtool -S for the card's rings and drops, perf trace -e skb:kfree_skb for where the kernel threw a packet away and why. "Where are we losing time" is a question about a queue, not about a layer.

It needs one more axis: which bytes a place reads. A layer is then a property of a decision, not of a box. RSS reads TCP ports from the bottom of the stack. This router sits at the end of a TCP connection and decides on bytes of TLS. It gets called "L4" for where it sits: it terminates TCP, holds two sockets per connection, and never terminates TLS. Packet-level balancers such as Maglev or Katran are "L4" too, and never terminate even TCP: they only rewrite packets.

Put together: a packet's trip is a chain of steps, and each step looks at some bytes at the front (headers) or at the end (the Ethernet checksum) and acts on them. The layered picture is the special case where every step opens exactly one envelope. Almost nothing behaves that way.

Reading one message

The ClientHello is the only thing the router ever parses, and the only code here that reads bytes chosen by strangers. TLS wraps it in records: five bytes of header (type 22, version, length), then the payload. A hello can be split across several records, and DPI-evasion tools split it on purpose, sometimes in the middle of the name, to confuse middleboxes exactly like this one.

So the parser does not wait for "enough bytes". It is a function of the prefix read so far, answering one of four things: the name, no name, not TLS at all, or need more. After every read the router calls it again on the whole prefix:

loop {
    if let Some(&kind) = hs.first() && kind != CLIENT_HELLO {
        return Hello::Invalid;
    }
    if let Some(len) = hs.get(1..4).map(u24) {
        if 4 + len > MAX_HELLO {
            return Hello::Invalid;
        }
        if let Some(body) = hs.get(4..4 + len) {
            return client_hello(body).unwrap_or(Hello::Invalid);
        }
    }
    // Header first: a non-TLS client must hear "no" now, not after it
    // sends as many bytes as its first five happen to spell.
    let Some(h) = records.take(5) else { return Hello::NeedMore };
    let len = u16::from_be_bytes([h[3], h[4]]) as usize;
    // 1 <= len <= MAX_RECORD in one unsigned compare.
    if u16::from_be_bytes([h[0], h[1]]) != RECORD_MAGIC
        || len.wrapping_sub(1) >= MAX_RECORD
    {
        return Hello::Invalid;
    }
    let Some(payload) = records.take(len) else { return Hello::NeedMore };
    hs.extend_from_slice(payload);
}

Re-parsing from the start after each read is quadratic, but only in the number of reads, and a cap on the hello (32 KiB) plus a timeout keep that small. The comment about the header is about plain HTTP: if the parser waited for a whole record before checking the header, a client that sends GET / would have its first five bytes read as a record length, and the router would politely wait for kilobytes that never come.

The first post has this parser compiled to WebAssembly, where you can split a hello into records and send it byte by byte.

The decision costs nothing compared to what follows. Every byte read so far goes to the backend first, so it sees the stream exactly as the client sent it.

Splicing without buffers

After the decision the router copies bytes both ways until both sides close. Tokio's copy_bidirectional does this with two 8 KiB buffers per connection, held for the whole life of the connection, and most connections spend most of their life idle. Add the hello buffer, which the first version kept until the end too, and an idle connection held about 21 KiB of user-space memory. Four thousand of them, about 84 MiB, for nothing.

The replacement waits for readability holding nothing. When the socket is readable it reads into a 16 KiB scratch buffer that belongs to the worker thread, and writes straight on. There is no .await between the read and the write, so the scratch is free again before the task yields and another connection on the same thread can use it:

from.readable().await?;
let rest = SCRATCH.with_borrow_mut(|buf| {
    let n = match from.try_read(buf) { /* 0 = EOF, WouldBlock = spurious */ };
    let m = match to.try_write(&buf[..n]) { /* WouldBlock = 0 */ };
    // Empty when everything went out: no allocation.
    Ok(Some(buf[m..n].to_vec()))
})?;

Only what the other side does not take right away is copied into a buffer of its own, and that buffer is dropped, not just emptied, once it drains. An idle connection now holds its task, around a kilobyte. That number is computed from the sizes, not measured on the server.

One detail I did measure. The first version tracked a partly written remainder with an offset, and a mutation (sent = m instead of sent += m) survived every test. To catch it I shrank the socket send buffer to 4 KiB and counted: 29 MiB through the relay, over 1800 remainders, and every one went out in a single write. On loopback the kernel lets a write overshoot the send buffer by a segment, so two short writes in a row never happen. Instead of a test that cannot fail, the offset went away: left.drain(..m) has nothing to get wrong, and the memmove of at most 16 KiB on a rare short write costs nothing.

Four lanes, one CAS

The first limit is on concurrent connections per client, where a client is an IPv4 address or an IPv6 /64 (what one subscriber usually gets). Without it, one machine opens four thousand connections, never sends a hello, and the router is full.

The obvious version is a mutex around a HashMap from client to count, and that is what I wrote first. It works, but every connection takes a lock, and the map grows with whatever addresses the internet throws at it. The current version is a count-min sketch in a fixed table: each AtomicU64 holds four 16-bit counters (lanes), and a client counts against two lanes of one word, picked by a keyed hash. Six ways to choose two lanes of four, so the pair comes from the hash too.

Admitting a connection means checking that not both of the client's lanes are at the cap, and adding one to both. All four lanes compare against the cap at once:

const ONES: u64 = 0x0001_0001_0001_0001;
const HIGH: u64 = ONES << 15;

/// Lanes of `word` that are at least `limit`, as their top bit.
const fn lanes_at_least(word: u64, limit: u64) -> u64 {
    ((word | HIGH) - limit * ONES) & HIGH
}

Every lane stays below 2^15. Setting its top bit and subtracting the limit therefore cannot borrow from the next lane, and the top bit survives exactly when the lane was at least the limit. A lane holding 5 with limit 8: 0x8005 - 8 = 0x7ffd, top bit gone, below. A lane holding 9: 0x8009 - 8 = 0x8001, still set. The whole admission is one compare-and-swap:

let mine = lanes << 15; // the client's two top bits
loop {
    if lanes_at_least(cur, per_client) & mine == mine {
        return Err(Refusal::MaxConnsPerClient);
    }
    match cell.compare_exchange_weak(cur, cur + lanes, AcqRel, Relaxed) {
        Ok(_) => break,
        Err(now) => cur = now,
    }
}

Releasing is fetch_sub(lanes). No lock, no allocation, 128 KiB of table for four thousand connections.

Why two lanes of one word and not a real count-min with lanes in two different words, which would collide less? Because two fetch_adds are not one atomic step. Let a client sit one below its cap, and two of its connections arrive at once. In word 1 the first increments before the second, in word 2 the other way round. Each sees one of its counters at cap - 1, the minimum is below the cap for both, and both get in. One word and one CAS keep the bound exact.

A lane is never below the number of live connections of any client on it, so nobody gets more than the cap. The reverse can fail: an honest client is refused when both of its lanes are full of other people. That mostly takes a twin, a client in the same word with the same pair, about clients / (6 * words) of them. A test with 600 clients all pushing toward a cap of 8 refuses 0.42 % of honest attempts. With the real cap of 64 an honest twin does not get there, so in practice being refused means sharing a cell with whoever is abusing it, and the keyed hash keeps anyone from choosing their cell.

The trick that fails, and then works

Count-min has a well-known improvement, conservative update: add one only to the lanes that hold the client's minimum. It cuts overestimates a lot, which here means fewer refused twins. It is also wrong here, even though each connection remembers which lanes it raised. Two clients, A on lanes (L1, L3), B on (L1, L2):

A opens 5 connections: L1 = L3 = 5.
B opens 3. L1 is above B's minimum, so only L2 grows: L2 = 3.
A closes all 5: L1 = 0.

B now has three live connections and a minimum of zero. The bound is gone, and B can open its full cap on top of them.

Plain count-min survives this because every connection does the same thing to the table: opening adds one to both lanes, closing takes one from both, so a closing undoes its opening exactly, whatever happened in between. In algebra this is called a homomorphism of groups. Conservative update makes what an opening does depend on the lanes at that moment, and the closing, later, has no way to undo just that.

Below is one word of the table with the router's own comparison, three clients and a cap of 3. Open and close connections by hand and watch the top bits decide, or press replay for the same story as above with smaller numbers. With conservative update off, B stops at the cap. Switch it on and replay, and B ends up with five.

One word of the per-client table, checked by the router's own lanes_at_least.

The second limit is on the rate of new connections per client, which the first does not cover: a client that opens and closes connections in a loop never holds many at once. The classic algorithm is GCRA. It keeps one number per client, the theoretical arrival time (TAT): a connection at time now is let in when max(TAT, now) - now <= tau, and then TAT = max(TAT, now) + T, with T the interval at the sustained rate and tau = (burst - 1) * T. Nothing but max and +: arithmetic in the max-plus semiring.

The TATs go into the same shape of table: 16-bit lanes, two per client, one CAS. A lane holds the largest TAT of everybody on it, so it never says a client's TAT is earlier than it is, and a collision can only make the limit stricter. And here conservative update works: the new TAT estimate comes from the smaller lane, and each lane is raised only to that, lane = max(lane, tat + T).

Here nothing is ever undone. A TAT is not taken back when a connection closes, it just falls behind as time passes, and raising a lane with max twice is the same as raising it once. So a lane never drops below anyone's true TAT. Counting needs undoing and breaks; taking a maximum never does and holds. Algebra has names for the two, a group and a semilattice.

Lanes count ticks since an epoch, and 15 bits run out. Once the epoch is a window old, the next admission moves it to now and subtracts the difference from the whole table, four lanes per word at once, saturating at zero:

/// `max(lane - d, 0)` in every lane, for lanes and `d` below 2^15.
const fn saturating_sub(word: u64, d: u64) -> u64 {
    // Surviving lanes as 0xffff each: the comparison's top bits,
    // moved to the bottom of the lane and multiplied out.
    let keep = (lanes_at_least(word, d) >> 15) * 0xffff;
    ((word | HIGH) - d * ONES) & !HIGH & keep
}

The rebase moves the epoch first and the words after it. Admissions read the epoch before and after their word and start over if it moved, in the style of a seqlock. One that read a word before the shift and writes it after fails its CAS, unless the word was all zeros, in which case its write is late by the shift, which again only makes the limit stricter. The defaults are 10 new connections per second with bursts of 40, enough for a reload spree, far below a scanner.

Evicting the silent

A slowloris connects and says nothing until the hello timeout. The per-client caps limit how many slots one address can take, but a botnet has many addresses, and once the router is full the classic response, turn newcomers away, works for the attacker. An honest client sends its hello one round trip after connecting. So when the router is full, the connection that has been silent the longest goes, and the newcomer gets its slot.

That needs the oldest of the connections still waiting for a hello, and it needs it under a flood. Waiting connections go into a ring of 64 time slots, each a slice of the hello timeout over 32 (156 ms at the default 5 s), so everyone waiting fits in half the ring. A u64 says which slots are occupied. The oldest slot is the first set bit counting from the slot after the current one, which is the one 64 slices ago:

let from = (self.slot_at(now) + 1) % SLOTS;
let s = (from + ring.bits.rotate_right(from as u32).trailing_zeros() as usize) % SLOTS;

Rotate so the oldest possible slot is bit 0, count trailing zeros, done. The same structure sits in the allocator as its size-class bitmap; there it finds the smallest class with a free block, here the oldest slice with a silent connection. Since nobody waits longer than the timeout, one word is enough and the second level of the allocator's bitmap is not needed.

The oldest connection must also have been silent for at least a second before it is evicted. Under a flood of fresh connections that never waited, everyone is young, and evicting would only make honest clients evict each other; the newcomer is turned away as before.

PROXY, and a web server that believes it

The web server behind the router used to see every visitor as 127.0.0.1. The PROXY protocol fixes that: before the first byte of the connection the router sends a small binary header with the client's address and the address it connected to. Sixteen bytes of signature and lengths, then the addresses. The header and the hello go out in one write, so in one segment.

Then the plaintext port, so that http:// redirects to https://. The web server's PROXY support is global, so it now expects the header on port 80 too, and the router forwards port 80 without parsing anything, header first. In the local test the redirect returned 404 whatever I did. The reason: with PROXY on, the web server takes the server port from the header, which is the public port 80, while it listened on 8080, so its "this is my HTTP port, redirect" rule never fired. The fix was to give it the same ports on loopback that the world sees, 80 and 443. It is now consistent with the outside, and the redirect comes with a clean https://panacan.xyz/... instead of an internal port.

The access log now has real addresses. A pipeline adds country, city and network operator from the free DB-IP databases, and a dashboard shows where the visitors come from, which pages, which referrers, and which bots. City is approximate at best: my phone, lying next to the laptop, shows up in a village 350 km east, because mobile operators hand out addresses from large pools.

Testing against exact models

Most of the tests compare the router with something slower that is obviously right:

the lane comparison and the saturating subtraction against loops, 100 000 random words each;
the connection sketch against an exact count per client, 200 000 random admits and releases: nobody over the cap, the table back to zero at the end, honest refusals under 1 %;
the rate sketch against exact GCRA per client on the same tick grid, 300 000 steps with hour-long pauses that force epoch moves: whenever the sketch lets a connection in, exact GCRA would have too;
end to end on loopback: routing, fragmented hellos, timeouts, caps, the PROXY header byte for byte, eviction under a full router.

Then mutations: break one line, check that a test fails. Rebase without moving the epoch, raise only one lane, min turned into max, no rotation in the ring, no minimum age for eviction, eviction off: each fails a test, the first by hanging it. One survives by design: the epoch check against concurrent admissions. It is reasoned through in a comment, and the tests are single-threaded.

What broke this time

A timeout around nothing. The first version of the plaintext mode read let read = read_hello(&mut client).await; and then wrapped read in a timeout. The timeout guarded a value that was already there. Found on reading it back, before any test, which is the kind of luck that does not repeat.

Resets that were refusals. The new end-to-end tests failed with "connection reset by peer". A connection refused right after accept is closed with the client's bytes unread, and the kernel answers that with RST, not FIN. The tests now count a reset as the router hanging up, which is what it is.

Quotes. The dashboard's log queries worked against the log database and failed in Grafana: its plugin mangles quotes in queries on the way through. The fix was to compute the field in the log pipeline, so the queries need no regular expression at all.

The dashboard that took Grafana down. I gave the data sources fixed identifiers so the dashboard could refer to them. Grafana looks up existing sources by that identifier, did not find the old ones, and refused to start. The provisioning now deletes them by name and creates them again on every start.

The neighbourhood

Rust has a few relatives of this router. rpxy-l4 is the closest: TCP and UDP, routing by SNI and ALPN including QUIC, and PROXY in both directions, still marked as work in progress. Rama is a framework for programmable proxies and lists SNI proxies explicitly. ByteDance's g3proxy has an SNI proxy mode inside an enterprise forward proxy. What none of them does in this form is the sketches, as far as I could see: fixed memory under attack, one CAS per decision, and a bound that holds by algebra rather than by a lock.

QUIC carries the same server name in its first packet, encrypted with keys anyone can derive from a public salt. Routing UDP on port 443 means decrypting that, reading the hello, and then following connection IDs instead of sockets. Others already do this well: rpxy-l4 routes QUIC by SNI today, and the big proxies have known QUIC inside out for years. So naturally I will try to implement it my own way, break it a few times on the way, and maybe learn something.

Further out is Encrypted Client Hello in split mode: the router would hold the key, decrypt the inner hello and route on the real name, while anything watching the wire sees only panacan.xyz. For a router whose reason to exist is a network that allows nothing but 443, that is the logical end.

Size as an 8-bit float

2026-10-06T00:00:00+00:00

The GPU hands you a large chunk of memory, say 256 MiB, and you are on your own from there. Hundreds of buffers and textures live in that chunk, created and destroyed every frame. So I need an allocator, but an odd one: I cannot see into that memory. I cannot write a block header into it like malloc does, and I cannot walk it. The whole allocator has to live on the side and hand out nothing but numbers, offsets into the chunk. Hence: an offset allocator.

It has to be fast and, above all, predictable. An allocation in the middle of a frame that takes ten times longer once in a while is worse than one that is always a little slow.

The result is holeinone, a hole in one. This post is how it got there.

Naive: a list of holes

The simplest allocator keeps a list of free ranges, holes. To allocate, it walks the list and takes the first hole the request fits (first fit), or the smallest one it fits (best fit), and puts the rest of the hole back. To free, it returns the range and glues it to neighbouring holes, if there are any.

It works, and it is linear. With a thousand holes that is a thousand comparisons per allocation, and holes multiply exactly when memory is most fragmented, that is, when it already hurts. Sorting the holes by size into a tree gives a logarithm, which is decent, but still a search.

Buddy: powers of two and XOR

The Linux kernel allocates pages differently. Every block is a power of two in size and sits at an offset that is a multiple of it. A block of size 2^k at offset o then has exactly one possible partner, its buddy, at o ^ (1 << k). Freeing does not search, it computes: is the buddy free? Merge the two into a block one order larger and try again. Free blocks are kept in lists per order, so an allocation takes the first non-empty list from the right order up.

Beautiful, but it costs memory. A request for 65 KiB gets 128 KiB. In the worst case almost half is thrown away.

Size as a float

Buddy has size classes by powers of two, which is too coarse. Linear classes would be too fine: from bytes to gigabytes there would be billions of them. I want something in between, and we all know that something: a float. A float has an exponent, so it covers a huge range, and a mantissa, so it tells a few steps apart within one power of two.

So I take the size and encode it in eight bits as a small float: three bits of mantissa, five of exponent. The exponent is the position of the highest set bit, one lzcnt instruction, and the mantissa is the three bits below it:

const fn uint_to_float_round_down(size: u32) -> u32 {
    if size < 8 {
        return size;
    }
    let highest_set_bit = 31 - size.leading_zeros();
    let mantissa_start_bit = highest_set_bit - 3;
    let exp = mantissa_start_bit + 1;
    let mantissa = (size >> mantissa_start_bit) & 7;
    (exp << 3) | mantissa
}

That makes 256 codes, of which sizes in u32 reach 240, and the top class starts at 3.75 GiB. Each power of two holds eight classes, so the sizes of blocks in one class differ by at most an eighth. A few examples:

size	class	class range
100	36	96 to 103
1,000	63	960 to 1,023
4,096	80	4,096 to 4,607
1,000,000	143	983,040 to 1,048,575

And this is not a float only figuratively. Convert the size to f64 and look at the bits. The exponent of an f64 is 1023 + h, where h is the position of the highest bit, and the top three bits of the mantissa sit at bits 49 to 51. My exponent is h − 2, so a shift and a subtraction do it:

fn round_down_f64(size: u32) -> u32 {   // for size >= 8
    ((size as f64).to_bits() >> 49) as u32 - (1025 << 3)
}

fn round_up_f64(size: u32) -> u32 {     // for size >= 8
    (((size as f64).to_bits() + (1 << 49) - 1) >> 49) as u32 - (1025 << 3)
}

Rounding up is adding "almost one" just below the three mantissa bits. When the mantissa overflows from 0b111, the carry walks into the exponent by itself, exactly as with any other float. I checked this against the original functions for every size from 8 to 2³² − 1, and they never differ. It is not faster, the conversion to f64 has a longer latency than lzcnt. But nobody can claim any more that it is only like a float.

A free block goes into the class of its size rounded down. A request is rounded up instead: whoever wants 100 bytes searches from class 37, that is, from 104. That guarantees any block in class 37 or above fits, and nothing inside a class needs comparing. I take the first block of the list and I am done. Not exactly best fit, but at worst an eighth off.

Where is the nearest non-empty class

What remains is finding the smallest non-empty class from 37 up. Naively, walk the lists one by one, all 256 in the worst case. But "is this class non-empty" is one bit, and 256 bits fit in four words. And "find the first one bit from position k on" is a mask and a tzcnt:

fn find_lowest_bit_set_after(mask: u32, start_bit: u32) -> Option<u32> {
    (mask & !u32::low_ones(start_bit)).first_set()
}

(low_ones and first_set come from hakmem; otherwise it is a plain mask & !((1 << k) - 1) and trailing_zeros.)

To make it one word instead of four, the bitmap has two levels, shaped exactly like the float: the top five bits of the class (the exponent) pick one of the 32 bits of the top mask, the bottom three (the mantissa) one of the eight bits of the byte below it. The search then goes: try the rest of the byte in your own power of two, and if it is empty, jump in the top mask to the next non-empty power and take its lowest class. Two bit searches, no loop, no walking lists. Allocation is O(1), and the constant really is small.

Free: neighbours by offset

To free, I need to glue the block to its free neighbours. Buddy computed them with an XOR; here blocks have arbitrary sizes, so XOR is no help. What helps is that every block, free or used, remembers its left and right neighbour in memory. Freeing looks at two pointers: is the neighbour free? Take it out of its class and attach it. Then put the result into the class of its new size. O(1) again, no search again.

Well, it should be. My first port had singly linked class lists. That saves 4 bytes per node and is enough for allocation, which always takes the head of a list. But the neighbour I merge with has to come out of the middle of its list, and that needs its predecessor. A singly linked list finds it only by walking from the head. I defended this in a comment: classes tend to be short, and fine classes "naturally bound" their length. They do not. Nothing bounds how many blocks one class can hold.

Allocate n equal blocks, free every other one (n/2 blocks in one class), then free the rest. Every free merges with a neighbour deep in the list:

n	singly linked	offset-allocator
10,000	3,361 ns per free	23 ns
20,000	6,481 ns	23 ns
40,000	12,764 ns	23 ns

The time per free grows linearly, the whole thing is quadratic. Upstream has doubly linked lists and holds a constant. A regression I had brought into my own fork, for 4 bytes.

The fix is a doubly linked list, and the 4 bytes can be found elsewhere. A node that sits in no list and no allocation has its list link unused, so the stack of free nodes can be threaded through those nodes instead of a separate array of indices. A node stays at 24 bytes, as before (20 plus 4 in the array), and there is a bonus: nodes are taken from the reserved array only when first needed. Creating an allocator with 128 Ki nodes went from 101 µs to 81 ns and reset from 80 µs to 30 ns, because neither touches the node array any more. And under churn at 90 % occupancy, where merging never stops, allocate and free together got twice as fast (80 against 39 ns per operation), without any constructed worst case.

The asymmetry where the bug hid

I insert rounding down and search rounding up. What if exactly one free block of size 100 exists and someone wants exactly 100? The block sits in class 36, the search starts at class 37 and skips it. The allocator reports out of memory with a perfectly fitting block right there.

This is not an academic exercise. The cleanest case is a fresh allocator holding its whole capacity in one block, asked for its whole capacity. Unless the capacity lies on a class boundary, the answer is None. The Rust port offset-allocator has it as a known bug (pcwalton #9, sebbbi #11 in the original). The fix is a slow path: when the fast search finds nothing, walk the list of the round-down class as well and check the actual block sizes. That class usually holds few blocks, and the fast path stays untouched.

It can all be said algebraically, and then it is clear why the bug happened. Rounding up is rounding down the number one smaller, plus one:

It holds because sizes are integers: a class's lower bound is at least r exactly when it is greater than r − 1. With floats it would not hold.

So the fast path searches for the strict successor of the class that r − 1 falls in. But the blocks that would fit exactly sit in class round_down(r), and unless r lies on a class boundary, that is the same class as round_down(r − 1): precisely the one the strict successor skips. The right query is "successor or equal" for round_down(r), with sizes to check in that one class only. The slow path does exactly this, it just has a name now. And the second rounding function is not needed any more.

The tip of the iceberg

But the slow path only gets its turn when everything above round_down(r) is empty. Until then the class is skipped every time. Whoever wants 100 bytes while class 36 holds a block of 101 does not get it: they get a piece of the first block a class or more higher, and that one gets split. On an empty allocator it does not matter. On a full one it turns into rubble.

I measured it on a long workload: log-uniform sizes in a range, allocate until occupancy reaches 90 %, then free random allocations, round and round a million times. How many allocations failed:

	64 MiB, blocks 64 B to 64 KiB	256 MiB, blocks 64 B to 4 MiB
search from `round_up` (original)	11.65 %	30.95 %
own `round_down` class first	0.049 %	21.1 %
and 16 classes per power of two	0.010 %	19.2 %

The fix is that identity again, this time in the fast path. The bitmap is asked not for the strict successor of class round_down(r) but for "successor or equal". When the round_down class is empty, which on a lightly used allocator it nearly always is, the query returns exactly what it did before: the same instructions, a different start bit. When it is not empty, I check at most four of its blocks, and if none fits, continue one class up.

The third row is the second lever. The asymmetry never goes away, the width of a class only sets how much of its own class a request skips. With four mantissa bits a class is a sixteenth wide instead of an eighth, and failures on the smaller blocks drop another five times, at the same speed, for 1 KiB more of class heads. With the large blocks, 90 % is past what any placement survives: even an exact best fit over a tree of free ranges fails 18.7 % of the same requests, so 19.2 % is at the ceiling.

Try it: two 4 MiB heaps get exactly the same stream of requests (blocks of 64 B to 8 KiB, occupancy held at the target) and differ only in where the allocator starts searching. Both hold the same occupancy, so both have the same amount of free memory. The difference is where that free memory is: the original has it broken into holes an ordinary request does not fit, and so it fails. The lower chart shows it directly, free memory by hole size.

The same workload, two search policies. The allocator is Rust compiled to WebAssembly, a faithful miniature of the one measured here.

Cutting in place

After all the fixes, holeinone was faster than upstream wherever blocks are freed and reused. Only allocation alone, carving up one large block in sequence, was 1.23 times slower. So I looked at where the time went.

perf annotate showed allocation bound by the latency of a chain of dependent loads: the bitmap, the class head (20 % of samples), the node, the store of the new head (9 %), the remainder's class and its head (18 %). The block comes out of its list, the allocation is cut off its front, and the rest goes back in. But when one large block is being carved up, the rest almost always lands in the same class the block came from. The list empties, the bitmap bits are cleared, and right after, the same list fills up again and the bits are set.

Turn it around: the free node keeps the rest. It stays where it is, in the same list, with its offset moved up and its size reduced, and the allocation gets the new node. Neither the lists nor the bitmap are touched. Placement does not change, the allocation still takes the front of the block, so the failure counts on the long workload came out identical to the last block. Only the cost changed: sequential allocation went from 44.9 to 32.4 µs, 0.89 of upstream.

One trap on the way: I generalised it to a cut at any offset, to cover aligned allocations with front padding as well, and the unaligned path slowed back down to 40 µs. The general function, one branch richer, was expensive on the hot path. The unaligned path now has its own specialised version.

Beyond 4 GiB, and freeing when the GPU is done

Offsets in u32 mean a heap of up to 4 GiB, and since the top bit of a size marks a node used, in fact up to 2 GiB. That is enough for blocks of the size VMA and friends allocate, but not for one huge buffer holding all geometry. So holeinone counts in quanta inside, units of a power-of-two number of bytes, and speaks bytes in u64 outside. A node stays 24 bytes for any capacity, the quantum is chosen as the smallest that covers the capacity (one byte up to 2 GiB), and GPU resources need alignment of tens of bytes and more anyway.

The second thing I only realised when I imagined actually using the crate: a renderer cannot free a frame's buffers when the CPU is done with them. The GPU reads them one to three frames later. Hence free_after(allocation, value), with a fence or timeline semaphore value, and retire(completed), which frees everything up to that value. Transients freed three frames later cost 33.2 µs against 28.3 µs for freeing right away: about 7 % is the queue, the rest more merging, since blocks leave while more of their neighbours are alive.

What it buys

The core is not mine: it is Sebastian Aaltonen's OffsetAllocator (2023), by way of Patrick Walton's Rust port. Against offset-allocator 0.2 on a laptop, median of three criterion runs, the ratio being holeinone's time over upstream's:

workload	holeinone	offset-allocator	ratio
allocate and free 2,000, random order	35.5 µs	44.3 µs	0.80
allocate and free 2,000, LIFO	29.5 µs	44.0 µs	0.67
transients, 10 frames × 200	28.3 µs	45.4 µs	0.62
8 assets plus transients	11.6 µs	18.9 µs	0.61
allocate only, 5,000 × 64 B to 64 KiB	32.4 µs	36.4 µs	0.89
the same into a full heap (90 % fail)	14.9 µs	11.3 µs	1.32
aligned to 4096, 500 × 64 B to 1 KiB	12.0 µs	4.0 µs	3.0

The last two rows are not the same work. A failing request in mine still checks the blocks in its own class; upstream gives up at once, at the price of bug #9. And with alignment, upstream users allocate size + alignment − 1 and the padding stays inside the allocation, so at an alignment of 4096 the heap holds several times what was asked for. holeinone gives the padding back.

Benchmarks that lied

This part was not planned, and it taught me the most.

The probe that cost 18 %. The first measurement on a light workload said the probe into the round_down class cost 5 to 18 %. But the counter said that on the light workload it never once touched a non-empty class. So I measured A/A: two variants with the same fast path, differing only in dead code in the slow path. One of them "got faster" by 3 to 10 %. On microsecond benchmarks, a shift in code alignment is as large as what you are trying to measure. The instruction count from perf stat settled it: the probe adds 2 to 5 % instructions and is indistinguishable from noise in cycles.

A fresh allocator every iteration. The benches built a new allocator in the iteration setup and consumed it in the timed part. Once the node array became lazy, the page faults moved from the constructor into the timed part and holeinone looked slower. What was measured was when the pages got paid for, not how fast allocation is. Now each implementation keeps one allocator, reset between iterations outside the timing, the way a real program uses it.

The bench that measured failure. "Allocate into 16 MiB" allocated 5,000 blocks of 32 KiB on average. A tenth of them fit. The bench pretended to measure allocation and measured the failure path.

Same code, another binary, a third off. Adding two branches to main of a measuring harness moved the times of an entirely different function by a third. I trust only comparisons within one binary, or instruction counts from perf stat.

Cross-crate inlining. After the API became a thin wrapper over an inner function, the body of allocation started being instantiated in the caller's crate, and small helpers without #[inline] could not be inlined across the crate boundary. A forced #[inline(always)], on the other hand, made sequential allocation 3 % faster and the full-heap path almost twice as slow.

An estimate from one harness. I estimated the cost of the new API at 7 % from a harness that dropped the returned handle almost at once. Criterion, which passes it on, showed over 20 %. Then perf showed it was about neither: 96 % of the time was in the allocation function, and that is where the road to cutting in place started.

1994

I started seeing the pattern everywhere: "round the key to a small float, get few classes, put a bitmap with lzcnt over them". Timers: since 2016 the Linux kernel sorts timers into buckets with a granularity of about an eighth, and admits a timer may expire that much later. The commit says it outright, "worst case inaccuracy of 12.5%" (500462a9de65). A priority queue for Dijkstra: the radix heap. Sets of integers: the van Emde Boas tree. I wrote a timer wheel on this pattern and measured it against BinaryHeap from std: seven times faster on dense data. That is to be expected: a binary heap knows nothing of keys being small integers. But then I wrote a priority queue for Dijkstra on the same pattern (Dial's algorithm: buckets by distance, a two-level bitmap over them), about a hundred lines, and measured it against the radix-heap crate. The radix heap is a structure capable people have been refining since Ahuja, Mehlhorn, Orlin and Tarjan described it in 1990. On dense graphs my queue was 1 to 22 % faster, on sparse ones 31 to 74 %. To be fair: a radix heap takes any monotone keys, mine needs a bounded edge weight and a bucket for every distance in the window, and the gain is exactly that with monotone keys nothing gets redistributed. Still. A general pattern! A universal primitive!

Then I found Matias, Vitter and Young, Approximate Data Structures with Applications, SODA 1994. Keys rounded to a truncated float, error at most 1 + ε in value, operations in constant time, and as the application exactly Dijkstra, with a proof of how the error adds up along the path. Their constant time comes with hashing, so only with high probability; two bitmaps over the classes do it deterministically. Three mantissa bits are their ε = 1/8, four are ε = 1/16. Thirty-year-old theory that authors of allocators, kernels and network packet schedulers have since reinvented more or less independently. TLSF from 2004 has a first- and second-level index, an exponent and a mantissa, and the Vulkan Memory Allocator uses it today with five mantissa bits, 32 classes per power of two.

I noted two things. That I was not thinking wrong, only thirty years late. And that it pays to search before naming the thing.

So

An offset allocator is a float and two bitmaps. The float squeezes a huge range of sizes into a few hundred classes with an error of at most a sixteenth, the bitmap turns "find the nearest non-empty class" into two instructions, neighbour links turn merging into two comparisons, and doubly linked lists keep it O(1) in a full class. Nothing is searched, everything is computed. A single character in the bitmap query, "successor or equal" instead of "successor", decides whether memory crumbles under load. And the fastest list operation is the one that does not happen.

Method

Times are from a laptop with criterion, median of three runs that agree within 1 %; the ratios matter, not the microseconds. Each implementation keeps one allocator, reset between iterations outside the timing. Allocation sizes come from a xorshift with a fixed seed, the same for both implementations. The cost of individual changes: perf stat (instructions, cycles, mispredictions) and perf annotate, A/B only within one binary. The timer wheel against BinaryHeap: 100,000 timers, dense distribution, 6 ns against 46 ns per operation. Dijkstra: an N × N grid with four neighbours, edge weights uniform in 1 to W, shortest paths from a corner, all queues must produce the same distances; "dense" means W ≤ 200, "sparse" W ≥ 500.

Long workload (examples/rd_first_churn.rs in holeinone): a million operations, log-uniform sizes, target occupancy 90 %, a failure frees a random allocation; percentages are the mean over three seeds. The best fit for comparison is an exact allocator over a tree of free ranges in a simplified simulation. Worst case of singly linked lists: n blocks of 64 B, free the odd ones, time freeing the even ones.

First post, delivered late by a router

2026-10-04T00:00:00+00:00

This is the first post. Before it existed, the site already had a static generator, a search index, a WebAssembly build, a Nix flake, a web server and an SNI router in front of it. I would read that ordering less as an accident and more as a fairly accurate portrait of the author.

Who writes here

By day I tune SQL in old ERP systems, the kind full of well-meant integrations: queries that ran fine for ten years, until a consultant and a developer had an idea. I was hired to do analytics on a data warehouse; the performance work found me anyway.

By night it is algebra, discrete math and bit tricks, mostly in Rust, and further down than is reasonable, which lately means compilers. The handle is might_sleep(), the kernel's annotation for a function that may sleep: call it from the wrong context and you get a warning. Given the hours, it fit.

These days most of the night shift ends up in the Rust data stack, arrow-rs, DataFusion and Vortex, because that is where the two shifts meet: the day job is databases, the night is bit tricks, and a columnar engine is mostly bit tricks pretending to be a database. A few small patches of mine got merged there. The rest went into hakmem, a crate that tries to treat bit tricks as an algebra with laws instead of a bag of one-line spells, named after a 1972 MIT memo it will never live up to, and into a Vulkan renderer whose geometry pass has been "almost working" for long enough to count as a research topic.

The pattern, if you squint: a weakness for problems that look serial but hide an algebra. A scan that is secretly a monoid. A predicate that is secretly a truth table. A curve that is secretly a Gray code.

What you will find

Short posts about such things, each with code and, where it helps, a demo that runs right in the page. Rust, C, occasional trace amounts of functional programming and good database design (usually by accident).

Measurements will always come with the method, because designing it is sometimes more interesting than the speedup itself. Some posts will admit that an idea turned out slower. Those are the ones you learn the most from, and let's be honest, they are not exactly a thrilling read for most normal people.

Posting schedule: yes.

Tech stack

Zola renders Markdown, Pagefind builds a search index that runs entirely in your browser, and each demo is a small Rust crate built with wasm-bindgen. One Nix derivation turns all of it into a directory, and a deploy is a new directory plus a flipped pointer.

It is served by Ferron, a web server written in Rust, from a small container at vpsFree. In front of it sits a router I wrote over this weekend: it reads the TLS ClientHello, looks at the server name and forwards the connection without decrypting anything. About four hundred lines, plus two hundred and fifty lines of tests, which took most of the time.

Below is roughly the first thing your browser sent to get this page, built in Rust and checked against the router's own parser, compiled to WebAssembly. Type another name, split the hello across more records, or press send and watch the bytes arrive the way TCP delivers them. The router never decrypts anything; it follows a few lengths, skips everything else, and only answers once the whole message is in.

A TLS 1.3 ClientHello as the router sees it. The parser is the router's, byte for byte.

What broke on the way

The ordering in the first paragraph has a price, and it is only fair to list it.

The certificate that renewed itself into the void. The domain has two DNS views: Cloudflare answers the public internet, and a resolver of my own answers inside a private network, with records nobody outside should see. Let's Encrypt checks ownership through a TXT record that only Cloudflare holds. The ACME client then waited for that record to propagate by asking the system resolver, which was my own one, which is authoritative for the domain and has never heard of the record. Renewal became a lottery, and on 21 September the lottery lost. The fix is one line telling it to ask 1.1.1.1. Split-horizon DNS is a fine idea right until something needs to see both horizons.

The resolver. The same box runs Hickory as a recursive resolver, validating DNSSEC from the root down. Version 0.26.3 had three bugs that all showed up in one evening: a validation loop on unsigned delegations, answers with the address before the CNAME that leads to it, and a validator that shuffled the answer again through a HashSet even after the second bug was patched. glibc reads the chain in order and gives up, so about a quarter of lookups for cache.nixos.org ended in "Could not resolve host". I spent half a day patching 0.26 before checking that main had already fixed all three. It now runs a pinned commit from main, which is not a sentence I expected to write about a DNS server.

The ghost on port 443. Before the web server, tailscale serve exposed a dashboard. I deleted the systemd unit that set it up, and the web server refused to start: address already in use. The serve config lives in tailscaled's own state, not in the unit, and outlives it. tailscale serve reset. Declarative configuration, meet the one imperative command that remembers.

The router I almost misconfigured. The plan was simple: the router reads SNI, sends one name to another service and everything else to the existing web server. But that web server picks a virtual host by the HTTP Host header, not by SNI. A client could say "public site" in the handshake and "internal dashboard" in the request, and get the dashboard from the internet. Domain fronting, self-hosted. The public site therefore gets its own web server instance that knows only public hosts.

Each of these ended as a one-line fix and a comment three times longer than the fix. That ratio is probably the most honest metric on this site.

Feedback

If something here is wrong, slow, or already in a 1972 memo, I want to know. GitHub: mightsleep, email: acannix@proton.me.

mightsleep

Two dimensions, one disk

Rows

A curve

Z-order

Hilbert

What a square costs

Try it

Where this lives

Next

Method

Four lanes, one CAS

Why a router at all

Where is layer 4, anyway

Reading one message

Splicing without buffers

Four lanes, one CAS

The trick that fails, and then works

Evicting the silent

PROXY, and a web server that believes it

Testing against exact models

What broke this time

The neighbourhood

Next

Size as an 8-bit float

Naive: a list of holes

Buddy: powers of two and XOR

Size as a float

Where is the nearest non-empty class

Free: neighbours by offset

The asymmetry where the bug hid

The tip of the iceberg

Cutting in place

Beyond 4 GiB, and freeing when the GPU is done

What it buys

Benchmarks that lied

1994

So

Method

First post, delivered late by a router

Who writes here

What you will find

Tech stack

What broke on the way

Feedback