← All posts

The relay that cannot read its own traffic

Group Ride lets several riders watch each other move on one live map. The server that carries those positions holds no key to any of them. Everything below — the Redis layout, the Lua scripts, the two-step join, the one deliberately unauthenticated endpoint — follows from that single constraint.

A key derived on a device seals four envelopes; across a dashed trust boundary the same shapes appear as unreadable blocks.

Most "live location sharing" works the obvious way: phones send coordinates to a server, the server keeps them, and everyone reads them back. It is simple, it scales, and it means the operator of that server can watch every user move in real time. For an app whose entire premise is that your routes stay yours, that was not an option.

So we built the other thing. The relay stores what it needs to route and authorise — and nothing else it could read even if it wanted to. What follows is the whole design, in enough detail to argue with.

1 · Topology

Four parties, one of which is untrusted by design

There is no socket, no push channel, and no server-side fan-out. Every client polls the same endpoint on an interval the server itself dictates, and the server keeps no connection state between calls — it is a stateless function in front of Redis.

CLIENTS Android app holds the group key iOS app holds the group key /g#token page landing only — no map Relay function one endpoint, 8 actions stateless no key material Redis 6 key shapes every key carries a TTL the group dies on its own Auth provider verifies ID token → uid POST sealed envelopes resolve only atomic scripts opaque blobs 7 of 8 actions
holds or carries sealed data plaintext / no key material browser, not an app client
The web page behind an invite link is a landing page, not a viewer. It confirms the invite is live and then points you at the app. Live group maps exist only in the apps, because only the apps hold the key that makes the positions readable.

2 · Trust boundary

What the relay can see, and what it provably cannot

The invite token is the root secret. It is generated on the creator's device, never transmitted, and every readable field in the system is encrypted under a key derived from it. What the relay receives is a hash of that token — enough to look a group up, useless for reading anything inside it.

TRUST BOUNDARY — nothing left of this line is transmitted ON THE DEVICE IN REDIS invite token 16 random bytes HKDF-SHA256 group key 32 bytes, AES-256-GCM seals 4 envelope purposes meta — name, destination, start roster — display name, avatar pos — lat, lng, speed, bearing code — the wrapped token AAD binds purpose + member id only sha256(token) crosses ciphertext only plaintext the relay uses group id · member id · timestamps revision · member count · state sha256(token) · join code enough to route and authorise, no more opaque to the relay every coordinate ever synced every name, avatar, destination a full database dump reveals none of it
The invite link puts the token in the URL fragment — after the #, not in the path. Fragments are never sent in an HTTP request, so the token cannot appear in a server access log, a proxy, or a referrer header. Putting it in the path would have leaked the root secret of every group into logs, permanently.

3 · Data model

Six key shapes, two of which are lookup indirections

A joiner arrives holding either a token or a six-character code, and neither of those is the group's identifier. Both resolve through a pointer key first. That indirection is what lets the code be short and disposable while the token stays the real secret.

POINTERS group:tok:<sha256> → group id · TTL = group lifetime group:code:<6 chars> → group id + token, wrapped TTL 30 min — expires early, on purpose group id GROUP STATE group:<id> record + sealed meta envelope group:<id>:members hash: member → sealed roster group:<id>:rev roster revision counter group:<id>:pos hash: member → sealed position
The code key holds the token wrapped under a key derived from the code itself, using a different derivation label so the same input could never produce both keys. Someone who typed six characters can therefore unwrap the real token locally. The relay cannot — it has neither the code-derived key nor the token.

Every key carries a TTL, and that is the deletion story. There is no cleanup job and no scheduled sweep. A group expires because Redis drops it — which is also why the leave path re-applies the remaining lifetime after any write: setting a value clears the expiry, and a group that quietly lost its TTL would outlive its own end date in the least visible way possible.

4 · API surface

One function, eight actions

All eight live behind a single endpoint and dispatch on the path segment. Seven verify a signed ID token and derive the caller's identity from it. One does not, deliberately.

ActionAuthDoesGuards
createsigned inWrites all five keys in one atomic call, returns the group id and codeFails if the token hash or code already exists
resolveopenToken hash → group id; code → group id plus the wrapped tokenRate limited per hashed IP
joinsigned inAdds the member and their sealed roster envelopeCapacity checked atomically; per-account hourly cap
syncsigned inWrites your position, returns everyone else's, plus the roster if yours is staleMembership re-checked every call; write floor
statesigned inMoves the group between preparing, live and endedLeader only; re-applies remaining lifetime
leavesigned inA member exits — or, if the caller is the leader, the group endsTwo different paths; see below
removesigned inLeader removes a memberLeader only; cannot remove the leader
metasigned inReplaces the sealed meta envelope: destination, start timeLeader only; envelope size capped

leave is two operations behind one name. If the caller owns the group, the handler never runs the leave script at all — it routes to the same end-group path that state uses, and everyone is told the leader left. Any other member runs the script, which removes them and bumps the revision. Only when that leaves zero members does the script delete all six keys outright: an empty group is not a group, and waiting for the TTL would leave one sitting there for hours.

Why resolve is open. Someone tapping an invite link may not have signed in yet, and putting an account wall in front of "is this invite even real?" would be hostile. The cost is that it cannot be limited per account — so it is limited per IP instead, and the IP is stored as a hash rather than in the clear, because a rate-limit counter is not a reason to keep a log of who looked at what.

5 · Join

Two paths in, and only one of them knows a code

Joiner Relay Redis LINK PATH reads token from the URL fragment resolve by token hash read pointer key group id (+ code, for re-sharing) CODE PATH user types six characters resolve by code rate limited per IP group id + token wrapped under the code unwrap locally code → key → token BOTH PATHS — now holding the token join — sealed roster envelope signed ID token → member identity atomic: capacity check, add, bump revision member count, revision, sync interval
The relay never sees a token on either path. The link path sends its hash; the code path gets the token back already wrapped, and unwraps it on the device. By the time join is called, a single flag is all that distinguishes the two — which is exactly why that flag has to be threaded honestly. Both paths converge on one handler, and hardcoding it there quietly recorded every link-join as a code-join for a couple of releases.

6 · The hot path

Sync is the only endpoint that runs in a loop

Every other action fires once. sync runs for the life of the group on every member's device, so its cost is the system's cost — and the server, not the client, decides how often it happens.

CLIENT REPORTS SERVER RETURNS moving · app in foreground moving · app backgrounded stationary group not started yet 10 s 20 s 60 s 30 s the live case screen off, still recording a parked rider costs little nobody is moving yet on failure retryable → back off exponentially to a ceiling, with jitter not retryable → stop; the group is over for this client
The interval is served, not compiled in. Every sync response carries the next interval, clamped server-side, so load can be shed globally by changing one setting instead of shipping a client release. A client that ignores it and writes too fast is told to slow down.

Roster transfer is revision-gated. The client sends the revision it last saw; the roster comes back only when the server's counter has moved. Positions are small and change constantly, rosters are larger and change rarely — sending both every ten seconds would spend most of the bandwidth re-sending names nobody changed. A member whose position has gone stale is dropped from the response entirely rather than drawn as a ghost.

7 · Limits

What stops it being abused

BoundEnforced whereWhy it exists
Members per groupInside the atomic join scriptTwo simultaneous joins cannot both pass a capacity check that runs outside the write
Group lifetimeKey TTLThe group's whole existence; expiry is the deletion
Join-code lifetimeTTL on the code keyMuch shorter than the group — a six-character code is guessable, a token is not
Code lookupsPer hashed IPMakes guessing codes at volume uneconomic
Join attemptsPer accountCaps how far a guessed code can actually get
Position writesTimestamp compare in the sync scriptA floor on the hot path regardless of what a client believes
Envelope sizesRequest validationBounds a blob nobody on the server can inspect

A removed member still holds the key. They can decrypt anything they already have, and nothing can take that back — so authorisation, not cryptography, is the enforcement point. Every sync re-checks membership and refuses the moment the member is gone, which is why that check sits inside the atomic script rather than in a cached session.

8 · Why every mutation is a script

Two problems, one answer

Atomicity. Creating a group writes five keys. If the token pointer landed and the record did not, the group would be permanently unreachable while still occupying its code — and half-written state in a store with no schema is very hard to detect later. The create script writes all five or none.

Connection safety. The Redis client multiplexes a single connection, and a serverless function can serve several requests concurrently on one instance — so a transaction block from one request can interleave with commands from another. Scripting sidesteps that entire class of bug, at the cost of one round trip.

There is a smaller lesson in there too. Script arguments are asserted before every call, because the failure mode is silent: an out-of-range argument reads as nil rather than raising, so a miscount does not crash — it writes the wrong thing, quietly, and you find out much later.

The part worth stealing

None of this is exotic. It is a hash table, some expiry, and a key that never leaves the phone. The only real decision was refusing the easy version at the start — because retrofitting "the server cannot read this" onto a system that already could would have meant rebuilding all of it.

Questions, or think we got something wrong? Tell us.

Get TrackMe