Somewhere in the middle of all this, we built a density render of the highway index — a map of where the supercharging stars live — mostly because it’s cool. I looked at it for about one second and said, and the ledger records this verbatim, “what the fuck are these suspicious lines!?”
The immediate hypothesis practically writes itself, because when data looks
wrong you always suspect the pipeline before the universe: a bug in the
community data processing — some spatial join doing trunc()
where it meant floor(), truncation rounding toward zero instead
of down, so that grid cells −1 and 0 collide at the origin and systems
sitting on the seam lose their body records. We confirmed the dump really
does ship those systems with empty body arrays — systems listed, stars
missing. Case closed, surely. We drafted the upstream bug report. We built
the heal: backfill the missing star classes from live data, republish the
index, ride off into the sunset.
Then the verification came back mixed, and there followed the special misery I want to dwell on for a paragraph, because if you work with data you have lived it: for about three hours I genuinely did not know whose data was broken — ours, theirs, or the universe’s — and I want to be honest that it was three hours and not the epic it felt like, because the whole point of this post is that the numbers are real. Half the “missing” systems turned out to have full body data after all, with ordinary M and K dwarfs as primaries — no neutrons to restore, which meant the heal we’d built was resurrecting stars that had never existed. So we started cross-examining sources, and the sources started disagreeing with themselves. Spansh’s search results asserted star classes that Spansh’s own system pages, for the same systems, declined to confirm. For at least one system in the lane, EDSM — the database of what human players have actually scanned in the game — confirms that no commander has ever visited… and the search result confidently reported its star class anyway. Whatever those search classes are, they are not observations; somewhere upstream, someone’s model was quietly filling blanks and presenting them at the same counter as measurements. For a routing engine whose whole job is “send a human to a star that will refuel them,” this distinction is not academic.
The tiebreaker was EDSM’s scan statistics, which model nothing: across 6,059 player-visited systems in the lane, zero neutron primaries, where baseline rates predict dozens. Players have been flying through this desert for a decade, scanning as they go, and the scans agree with our render. The deserts are real. They are in the game.
And here the community’s own archaeologists take over the story, because — as we found once we knew what to search for — they had excavated this artifact years ago. A wonderful r/EliteDangerous thread on the density quirks of StellarForge — the procedural engine that generates the game’s galaxy — records the diagnosis: Frontier, the game’s developer, deliberately guarded the neighbourhood of Sol against generating neutron stars and other superheavy classes. Sensible world-building; you don’t want a magnetar next door to humanity. But the guard appears to have shipped with an or where an and belonged, so a condition meant to carve out a small box around home — suppress where both coordinates are small — instead suppressed every system where either coordinate is small: two full-length slabs, crossing at Sol, running the entire galaxy. The thread’s author, surveying the result on Spansh’s neutron map, described it as “a Kylo Ren movie poster.” Our trunc-vs-floor guess had the right shape of wrongness — an origin-straddling coordinate bug — and the wrong culprit line; theirs is better archaeology, and either way the conclusion stands: the bug is in the galaxy, a decade deep, and every community database has faithfully recorded it ever since. The bug report was retired unposted, sparing us the experience of explaining to a data provider that their data correctly describes a bug in reality.
|| where && belonged — the green box is
what was intended, the red cross is what shipped, suppressed to zero and
galaxy-long. The sky in the previous figure is empty, not thin; the
operator theory fits the crime.And the heal? The heal was killed at the finish line, because completing it would have injected phantom neutron stars into our index — stars the game does not contain — and the router would have confidently sent commanders to refuel at neutron stars that, on arrival, are red dwarfs. In a game where an empty tank is a tow truck, our “fix” was the poison. The silver lining rewrote the roadmap: if the voids are permanent terrain, then gap edges aren’t a workaround for bad data. They are the truth-preserving representation of a scarred galaxy.
The lesson, in four words: triangulate before reporting upstream. The postscript, for connoisseurs: an eternity — by which the record shows I mean later that same day — of Sol → Sagittarius A* profiling later, we noticed the geometric straight line from Sol to the core runs almost exactly inside the x≈0 desert the whole way. Humanity’s most-flown corridor has a 500-ly void lying under its centerline, and every route ever plotted along it has been quietly flying beside its own straight line. Nobody noticed, because nobody’s heuristic ever got what it asked for. (We eventually measured what that costs: plotting from Sol, dead center of the cross, burns four times the search effort of plotting the same distance from a start two thousand light-years off-slab — the frontier keeps getting yanked down the empty corridor before committing to a side. The route comes out fine, a jump shorter even, and the wall clock barely notices; the tax is all in the searching. The “sidestep seed” that fixes it is the subject of act 15, which happened, in real time, while this post was being written.)
The next idea was mine, and I produced it on my way out the door to work, which I mention because there is a genre of engineering idea that can only be produced by someone who is thinking about the problem while physically leaving it:
The problem isn’t lack of neutrons — it’s too much density. As density grows we can be lazy. Narrow the search to a ten-degree cone toward the target, grab the first star near the end of jump range, and just take it. I bet we can work out exactly when to get greedier, because we can measure the density curve. — me, departing
Read that against everything above and it’s an inversion. The whole campaign had armed the search against scarcity — walls, gap edges, landmarks, desert widening. Nobody had asked what abundance costs. In the galactic core, every expansion — every single step of the coarse search — scans a shell of thousands of candidate stars, sorts them, and weighs them against each other, for a decision that barely matters, because when the shelf is full, nearly any highway star near max boosted reach, pointed roughly at the goal, is fine. Choice is a luxury good. In the core we were buying it wholesale and using none of it.
The cone is only half of the idea, because density cuts both ways, and the other half points at the strangest address in this post: the galactic core is the densest star field in existence and it hosts a neutron desert — the Dark Cross’s x≈0 slab runs straight through downtown (Sagittarius A* sits at x = 25, comfortably inside the desORt, and yes, I’m keeping that spelling). On the highway map, that slab is a wall. On the full map it’s the easiest water crossing in the galaxy, because ordinary stars are everywhere — a plain 60-ly hop in the core always has somewhere to land. Out on the rim, the same-looking desert is lethal: the star field underneath a highway gap is as thin as the highway itself, and there is nothing to swim on. One measured curve, both verdicts — where the highway is rich enough to be lazy, and where a desert is safe to cross on plain jumps.
I should be honest about how much of that second verdict shipped, because this post does not get to preach benchmarking and then hand-wave its own roadmap. As shipped, every density dial in the planner reads highway density: the threshold that decides when the cone engages counts highway candidates, the oracle summarizes highway stars, and the floor you’ll meet in the next act counts highway stars per cell. The crossing half of the idea lives in the machinery only implicitly — a desert crossing is priced at plain-jump rates without asking what the field under it looks like, which is accurate over the core and optimistic over the rim, and the refine phase only discovers the difference after the pricing has already steered the search. Making crossings aware of the full field — a second oracle over all the stars, not just the chargers — is the open frontier, and it sits in the ledger under the usual rules: probe first, the benchmark matrix decides. (Late update, because this post kept being edited long enough to outlive its own roadmap: the probe ran. We built the second oracle, taught the benchmark to census every search expansion by the density of the space it landed in, and measured… no tax. Across four corridors, both ships, every phase of the search, expansions track the corridor’s own density profile at 0.66–1.03× — the planner was never overpaying in dense space to begin with. The frontier is closed, the dial has no phase to help, and the oracle stays on the shelf as a permanent instrument. The graveyard gets my second-favorite idea, and the negative is on the record where it belongs.)
The half that did ship began with the last line of the quote.
“Measure the density curve” turned out to be pre-instrumented:
the agg250 oracle from act 3 already stores per-cell star
counts, so local density was a constant-time lookup before the feature was
conceived. This is what a good instrument does — it answers questions that
hadn’t been asked when it was built.
Probes first, as always. A standalone greedy walker matched the full planner’s route quality in dense space while considering seventy times fewer candidates — then marched into the synthetic void wall and died at x = 11,347, where the wall starts at 11,000, precisely where the density curve said it must. Greed is free in the rich half, fatal in the poor half, and the boundary is measurable in advance. Then profiling bent the implementation the way it bent act 1: in dense space the coarse search turned out to be relax-bound, not scan-bound. “Relaxing” is A*-speak for the bookkeeping after the scan: for each neighbour you found, check whether the path through you beats the best known way to reach it, and update if so. Cheap per neighbour — but 44 of our 65 milliseconds went to relaxing nearly two thousand neighbours per expansion, because downtown, everyone’s a candidate. The cone’s real payoff was relaxing less: the coarse search fell from 65 milliseconds to 16.
Force-feeding the cone to every search made routes worse, so it shipped the way every risky idea here ships: as extra variants in the portfolio, where the tie-break cannot let a challenger win with a worse route. Every pinned benchmark route came back identical, byte for byte — and then the surprise: Wongi → Beagle Point came back eleven jumps shorter at the same wall-clock time. Threading the thin rim, the cone’s diet found a better chain than the full scan’s thicket. The sort was drowning the signal.
Around here, even the probes get the treatment. Chasing a one-jump discrepancy, we caught the walker cheating: it departed the start star — an ordinary G, no neutron jet to ride — at boosted reach, a free 300-ly opening hop the physics doesn’t sell. Run honestly, naked greed is roughly 25% worse than the planner, and refining its chains showed why: a scoop-blind pick burns near-max fuel every hop and threads few refuellers, so greedy chains buy their fuel in detours. The refuel-weighted candidate sort — the very thing the cone diets away — turns out to be load-bearing. The shipped result survived its instrument’s disgrace untouched, because the tie-break only ever accepted wins measured under full physics. The scaffolding was crooked; the building passes inspection anyway.
By now the scoreboard glowed. Every cell plotted, most in under a second, the hardest in twenty. Time to compare against the incumbent — Spansh, the community’s beloved router — on the hardest corridor, Colonia → Spase.
Spansh’s exact plotter, on my actual ship at ×4 boost: 137 jumps. Our router, with a strictly better drive at ×6: 284 jumps.
I would like to report that we took this well. We took it the way you take biting into what you believed was a chocolate chip cookie and finding raisins: technically nourished, spiritually betrayed. More than double the jumps, with a better engine. The diff told the story — Spansh’s route swings a 4,000-ly arc away from the straight line, adding 16% more distance flown, to stay on the neutron highway the entire way. Ours clung to the straight line, marching through the desert on dozens of tiny unboosted hops to save distance that distance was never the point of.
The root cause was philosophical: our coarse cost model priced light-years, when the thing a pilot spends is jumps. A 300-ly boosted hop and a 60-ly plain hop cost the same minute of a human life. Priced in distance, the desert shortcut looks thrifty; priced in jumps, it’s a catastrophe. The fix landed the same day: a goal field — flow outward from the destination across the cell graph, recording at every cell “how many jumps home from here, at your reach,” and let the coarse search steer by that instead of by distance, with a floor that refuses to route through cells too thin to chain boosts in. Result: Colonia → Spase went from 284 jumps to 90, and from twenty-four seconds of planning to one. Which is Spansh’s 137-jump ×4 optimum, rescaled to our ×6 drive, almost exactly. Two independent routers, converging on the same answer through different algorithms, is what being right looks like from the inside.
One more sanity check, run during final edits against the live Spansh plotter — and it turned into a full act-12 encore, provenance drama included. Round one: I plotted Wongi ↔ Colonia on Spansh and got 86 and 79 jumps against EDDA’s 58 and 54, drafted a tidy boost-ratio explanation… and then, auditing my own methodology, realized I couldn’t swear the ship I’d linked on the Spansh side was the build EDDA planned. Impounded. After eighteen acts of demanding provenance from everyone else’s numbers, the least I can do is impound my own. Round two, done properly: the machines pulled my actual Caspian’s loadout from the journal record, I imported it into Spansh verbatim, and we plotted the hardest corridor — Spase → Colonia, the very run this act is named for. Even reading the result needed receipts: the machines’ first automated read of Spansh’s route miscounted it as 103 jumps, I said “what? I see 85,” and the raw route data confirmed me. The true numbers, same ship on both sides: Spansh 85 jumps, 4 stops. EDDA, current build, best mode: 92 jumps, 5 stops. Run both through our own fitted-seconds judge and Spansh wins by about eight and a half pilot-minutes. Outright. On the corridor where they humbled us once already. And the anatomy, from their route data laid against our bench stats, is not the lesson I expected: the arcs are identical — both routers fly about 34,000 light-years on the same road. The difference is stride. Their average boosted throw is 398 light-years to our 370, their longest hop is 463 of a possible 467, and they cross the desert in 7 plain hops to our 10. They stretch every single jump closer to the drive’s maximum than we do, and seven jumps fall out of the route as a result. Same road, longer stride. So the act ends the way it began, and the title still stands: Spansh humbles us, and we say thank you. Their 85-jump route is archived in our repo, hop by hop, as ledger item 31 — because the best kind of open item is one where a concrete better answer already exists and is sitting in your benches folder, waiting to be understood.
That “price jumps, not distance” epiphany kept pulling. What does a route actually cost? Not search effort. Not milliseconds of planning. It costs minutes of a person’s evening. So we fit the model from the only honest source available: my own journal files — 65 sessions, 436 plain jumps, 745 scooping intervals, timestamped by the game itself. Verdict: a jump costs about 70 seconds of life (74 in the big ship, 64 in the small one). A fuel stop costs about 32 seconds of approach overhead plus the fill — tonnes scooped over the scoop’s measured rate. A five-tonne top-up is 36 seconds; a deep fill runs two minutes.
Then we made that fitted model the judge. The planner’s tie-break, its grace window, its escalation waves (when a thorough plot stalls, stubborn variants get re-run with bigger search allowances — “waves” — digging for a better answer) — all of them now score a route in estimated pilot-seconds. A route with two extra jumps but four fewer deep fills wins, because it should, because the pilot gets home three minutes earlier. (There’s a slider in the UI now — “jumps ↔ stops” — that scales the refuel term, and a “fewest jumps” mode that zeroes it and digs with the whole budget. The defaults are the measured coefficients, because the measured coefficients are what your evening actually costs.)
And here the Colonia clock, which has been falling all post, act after act, does the one thing I need you to see clearly, because it is the point of the whole campaign:
The wall-clock time went up. We spent two days making this planner fast, and then we taught it that its own speed was the cheapest thing it owned — and it immediately started spending milliseconds of planning to buy back minutes of flying. A plot that takes 600 milliseconds longer and saves four minutes of scooping is the best trade in this post. Fast was never the goal. Fast was the budget for good.
And a confession that arrived embarrassingly late — I was rereading this very post when it landed on me. Our fancy time model? The journal fits, the per-ship jump costs, the scoop-rate arithmetic? To first order, it’s a fuel gauge. Look at the pieces: the fill term is literally tonnes replenished. The stop count is how often the burn overruns the tank. And the jump count — because a neutron boost stretches the distance of a jump but not its burn — is close to total burn divided by one jump’s throw. Every term is fuel wearing a different hat. So I asked the machines to test it against the 25b matrix’s own numbers, and the answer came back yes, uncomfortably so: across all 108 runs, fitted pilot-seconds tracks tonnes scooped at R² 0.96–0.97 per ship — about 17 seconds per tonne in the big ships, 29 in the small one. The clock we built three instruments to fit is, to a couple of decimal places, a fuel meter with a per-ship exchange rate. The residual — where the judge actually earns its keep beyond the gauge — is a few percent on the long hauls and up to ±30% on short runs, where fixed costs and the stop quantization dominate. Naturally I then proposed the obvious simplification — just judge routes by fuel, fewer variables — and the machines, applying my own rules to my own idea, gated it on an overnight study instead of agreeing with me. Good thing, too. As a judge, the gauge is ruinous: it flips 48 of 54 benchmark cells and pays 22,604 fitted seconds — six pilot-hours — to save 1,417 tonnes, because ordering by fuel prices a tonne at ~16 seconds when scooping one costs 0.8. Cross-cell, fuel predicts time at R² 0.97; within a corridor, ranking by it inverts nearly every choice. A great model and a terrible judge are the same arithmetic pointed at different questions. But the lesson stands, and it’s the cheapest one in this post: we fit a model from 65 sessions of journal data, and its first principal component was sitting on the dashboard the whole time, labeled FUEL.
By this point the planner had accumulated tuning knobs the way any project does — the cone’s angles, the grace thresholds, the balance between the two bidirectional frontiers, a dozen more — each one set to whatever value won its own little benchmark the hour it shipped. My last big idea was to stop pretending anyone understood their interactions: treat the whole planner as a black box and let an optimizer (Optuna — a library that proposes knob settings, watches the scores, and concentrates its guesses where the scores were good) search the knob space against the fitted pilot-seconds objective. I attached one constitutional rule: the optimizer proposes; the full benchmark matrix disposes. No default changes hands unless the champion configuration survives the entire matrix of pinned benchmark routes.
Three studies and 183 trials in, that rule had already paid for itself.
Study 1: the champion wanted the greedy cone off and one of the planner’s floors — a threshold below which a feature declines to engage — lowered. The matrix split the bundle. The floor change was real, structural, and mechanistically explicable, and it shipped: it closed most of the return-trip asymmetry, taking Spase → Colonia from 108 jumps down to 94. The cone-off half was one lucky Beagle run that never reproduced; across the full matrix it cost 600 extra pilot-seconds. Rejected. The optimizer had been fooled by variance in a single cell of the matrix, and the matrix caught it.
Study 2: this champion passed the matrix as a bundle and then decomposed into nothing — when we cherry-picked its two “strong” knobs and measured them alone, they were a wash. That put a lesson on the record: an optimizer’s top-ten unanimity is partly the sampler herding around its own winners, not evidence of marginal effect. No defaults shipped — but the config, it turned out, was already on a plateau, which is itself a finding: six of the thirteen knobs measured flat across their entire ranges, and a dead knob is a knob nobody has to tune at two in the morning.
Study 3 ran with repeated measures and median scoring, because study 2 had taught us to distrust single runs. Under that discipline every leftover signal from study 2 dissolved, and exactly one genuine default emerged and shipped: a credibility ceiling of 1.25, a bound on how far above the theoretical best a candidate route may sit and still be allowed to start act 0’s grace countdown against everyone else. It exists so that a quick, mediocre route can no longer guillotine a reproducible 193-jump dig on the Beagle run just by finishing first.
Study 4 upgraded the objective to the journal-fit seconds model from the previous act. One more default survived the full gauntlet — a slightly tighter field-credibility ceiling that unlocked a reproducible improvement, identical across three runs, on the hardest return trip — while the rest of its champion bundle died in decomposition, the same sampler-herding autopsy as study 2. But this study’s real donation to the methodology chapter was a near-miss: the winning commit got stranded on a work branch by a stale working directory, and the “passing” verification was a benchmark that had silently failed and printed nothing — caught only because the final check printed the old route where the new one belonged. Even the verification steps get verified. Especially the quiet ones.
Final scoreboard for the gauntlet: four studies, eight champions proposed (counting the last bundle’s decomposed pieces separately, as the autopsy did), three defaults shipped — and every rejection was caught by a different stage of the pipeline: the matrix gate, the decomposition test, variance control, and one sharp-eyed read of a suspiciously quiet log. That is not a sad ratio; that is the pipeline doing its job. An optimizer with no gauntlet would have shipped all eight.