How we made the Bokdle leaderboard load 10× faster
A player reported on our ideas board that the leaderboard was slow. It was: up to 2.2 seconds before anything appeared. It now starts arriving in 0.1–0.18 seconds, about the same as a plain page. Here's why it was slow, and how we fixed it without spending a cent.
The setup
Bokdle runs on two servers in Johannesburg, close to most players. Your account, games and groups are stored in a database in Ireland. Every question a server asks that database is a round trip of about 170 milliseconds.
That's fine when saving a guess, which happens in the background after the page has already answered you. It's not fine for a page that has to read a lot before it can show anything, and the leaderboard is exactly that.
Why it was slow
1. Questions asked one after another. To build the page, the server asked the database who you are, then what this week's board looks like, then what's in the hall of fame. Each question waited for the one before, so the trips to Ireland added up to half a second or more before anything appeared.
2. A cache that ran out on someone. Each server kept a copy of the board for 30 seconds to 2 minutes. When it ran out, the next visitor had to wait while the whole month's games were fetched from Ireland and ranked again. That was the 2.2-second worst case.
3. Some things were never cached. The hall of fame and private group pages asked the database afresh on every view.
4. Nothing to look at while waiting. Tapping Leaderboard seemed to do nothing until the whole page was ready.
Options we weighed
| Option | Speed | Cost | Why it was or wasn't the answer |
|---|---|---|---|
| Quick wins: show the old copy while refreshing it in the background, plus a loading placeholder | good | free | Some visitors would still wait for the database |
| A copy of the database on each server | best, for every page | about $5 a month as we grow | The right long-term move for every account page; it's on our list |
| Rebuild every board on a timer | instant | expensive: re-reading the whole month every 15–30 seconds adds up to billions of database reads a month | Burns money re-reading data that hasn't changed |
| Keep the boards in memory and fetch only what changed | instant | about 1–2 million reads a month, within the free plan | ✅ chosen |
How it works now
The boards live in memory. Each server keeps everything it needs to build every board (this week's and month's games, nicknames, groups and medals) in its own memory. Ranking a few hundred players there takes a tiny fraction of a millisecond, so opening the leaderboard no longer asks the database anything.
It only fetches what changed. When a server starts, it loads everything once, in about half a second. After that, every 10 seconds it asks the database one small question: what's changed since I last looked? On a quiet minute, that's about one row. The database itself keeps track of what changed, so nothing slips through, even a fix made by hand.
Your own result shows straight away. When you finish a game, change your nickname or join a group, the server that handled it updates within half a second, so you see yourself on the board at once. The other server catches up within 10 seconds.
Ready before the first visitor. After an update to the site, a new server only starts taking visitors once its boards are loaded, so nobody gets a slow first load. And if the boards ever aren't ready, the page falls back to the old way: slower, but never broken.
Something to look at instantly. A placeholder outline of the board now appears the moment you tap, while the page loads.
Results
| Before | After | |
|---|---|---|
| Leaderboard, at its slowest | 2.21 s | 0.10–0.18 s: there's no slow case any more |
| Leaderboard, typical | 0.31–0.36 s | 0.10–0.18 s |
| A plain page, for comparison | 0.10 s | 0.10 s |
| Database reads for boards | grew with every view | about 1–2 million a month, however many people visit |
| How up to date | 30 seconds to 2 minutes old | your own result at once; everyone else's within 10 seconds |
How long the page took to start arriving, measured before and after the update on 15 October.
What's next
The same distance slows down other pages that read your account, like My stats. The long-term fix is a copy of the database on each server, so every read is local. That's next on our list.
Want the technical details?
This post skips the nuts and bolts on purpose. If you're curious how it's built (the database triggers, how the two servers stay in step, the code itself), we're happy to talk shop. Email [email protected], or drop a note on our ideas board.