Location Intelligence Retail Site Selection Automation

Preparing OSM Network Extracts for Routing

Every drive-time catchment this site describes is computed on a graph, and that graph is built from an extract of OpenStreetMap that somebody prepared. This section covers that preparation: choosing and clipping the extract, refreshing it without breaking reproducibility, validating that it is genuinely routable, and recording the version so a contour can be traced back to the road data that produced it.

It is the stage most likely to be treated as setup and forgotten. A routing engine imported once and left alone gives a year-old view of the road network, which quietly misprices every catchment near a new junction or a closed bridge; an engine re-imported on a schedule with no version recorded gives catchments that move for reasons nobody can reconstruct. Both are avoidable with a modest amount of pipeline discipline around the same idempotency and versioning principles that govern the rest of the stack.

Concept: What a Routable Graph Actually Requires

An OpenStreetMap extract is a general-purpose geographic database. A routing graph is a specific, much smaller thing: a set of nodes and directed edges with traversal costs, plus the restrictions that say which turns are legal. The import process is a translation, and everything interesting about it is a decision.

The profile decides which ways become edges and how expensive each is. A car profile keeps motorways through residential streets and excludes footpaths; a walking profile does close to the opposite and treats a motorway as impassable. The same extract therefore produces entirely different graphs depending on the profile, which is why comparing routing engines without holding the profile constant measures nothing.

The topology decides whether the graph is connected. Ways that visually cross on a map do not connect unless they share a node, so a bridge over a road is correctly unconnected and a level crossing that should connect but does not is a defect. Extracts clipped at a boundary create defects of their own by cutting ways in half, which is the single most common cause of disconnected rural networks.

The restrictions — one-way tags, turn restrictions, access tags, barriers — decide whether a path that exists in the graph is one a vehicle may take. They matter far more in dense urban centres, where a catchment computed without turn restrictions can be ten to fifteen per cent too generous.

The import is a series of deliberate reductions A regional extract of 118 million objects is filtered by the routing profile to 4.1 million highway ways, converted into 9.4 million directed edges with costs, annotated with 62,000 turn restrictions, and finally contracted into the query structure the engine serves from. Each stage discards information that cannot be recovered later. Four reductions, each one a decision you cannot revisit later raw extract 118M objects roads, buildings, land use profile filter 4.1M highway ways car: no footways edges + costs 9.4M directed edges speed per class query structure contracted annotations applied during the edge stage 62,000 turn restrictions · one-way tags · access and barrier tags · surface and grade where the profile uses them omit these and the graph is faster to build, permanently too generous, and gives no sign of either The reductions are why the extract version has to be recorded rather than inferred: a contour built on a graph cannot tell you which ways the profile discarded, and re-deriving it later is guesswork. Counts are from one regional extract; the ratios are typical, the absolute figures are not.

Architecture: Extract, Build, Publish, Pin

The pattern that keeps this manageable has four stages and one rule.

Extract downloads a source file — a regional extract or a national one — and clips it to the markets you actually operate in, with a generous buffer. Build runs the engine’s import against a chosen profile, producing the graph files. Publish places the built graph at a versioned location and updates a pointer that the serving instances read. Pin records the version identifier into every artifact computed against it: contour, catchment, reachable-population figure.

The rule is that the pointer, not the graph, is what changes. Serving instances read the pointer at start-up or on a signal, so a new build is promoted by an atomic pointer update rather than by overwriting files underneath a running engine. That is the same atomic promotion used elsewhere in the pipeline, and it makes a rollback a one-line change rather than a re-import.

Configuration Parameters

Parameter Typical value Notes
source regional extract National only when markets span it
clip_buffer_km 30 Generous — the cost of too small is a broken graph
profile car, foot, bike One graph per profile; never one graph, many profiles
speed_source profile defaults or a speed file Recorded either way
turn_restrictions on Off is a 10–15% error in dense centres
build_version source date + profile + config hash The pin
retain_builds 3 Enough to roll back and compare
promote pointer update Never an in-place overwrite

clip_buffer_km is the parameter with the worst failure mode relative to its cost. A tight clip saves a little disk and cuts every road that crosses the boundary, producing a graph whose edge markets are quietly unroutable; a generous one costs storage that is measured in gigabytes at a time when gigabytes are the cheapest thing in the stack.

What a tight clip costs at the boundary Clipping exactly to the market boundary leaves 214 disconnected components and 31 stores whose contours are truncated. A five-kilometre buffer reduces that to 46 components and 6 stores; thirty kilometres to 12 components and none. The extract grows from 1.1 to 1.9 gigabytes across the same range. Buffer generously — the storage is cheaper than the debugging clip buffer components stores with truncated contours size exact boundary 214 31 1.1 GB 5 km 46 6 1.3 GB 30 km 12 0 1.9 GB The twelve components remaining at thirty kilometres are genuine islands and private networks, which is the floor for this geography rather than a defect to chase. Truncated contours are the expensive failure: they look like valid polygons, they under-count reachable population, and they systematically penalise sites near a market's edge — the ones expansion cares about.

Speeds and Costing: Where a Travel Time Actually Comes From

Nothing in an OpenStreetMap extract states how fast a car travels on a given road. A minority of ways carry a legal speed limit, which is not the same as an achievable speed, and the rest carry only a road classification. The profile fills the gap with a table mapping classification to assumed speed, and that table is the single largest determinant of every duration the engine returns.

This has three practical consequences worth internalising before tuning anything else.

The default table is a national average applied to your markets. An engine’s stock car profile assumes a residential street runs at some speed that is reasonable in the abstract and wrong in a dense city centre where the real average is half of it. The error is systematic by road class, so it does not cancel: markets with more of the mis-specified class are wrong in a consistent direction, and the ranking between markets shifts accordingly.

A legal limit is an upper bound, not a measurement. Applying posted limits directly produces optimistic durations everywhere there is congestion, signalisation or parking activity — which is everywhere retail catchments matter. Most production profiles apply a fraction of the limit by class, and the fractions are the honest place to encode local knowledge.

Observed speeds beat both, where you have them. Delivery telematics, loyalty-app trip timings and commercial speed datasets all provide measured travel times on real roads at real times. Feeding those in as a per-edge speed file — rather than adjusting the class table until a few known routes look right — replaces an assumption with a measurement and, importantly, makes the assumption’s remaining scope explicit: the classes with no observations are the ones still running on defaults.

The temptation at this point is to tune until a familiar route matches a familiar drive. Resist it. A profile fitted to a handful of routes an analyst knows well will be wrong in a hundred markets nobody checked, and it will be wrong in a way that is very hard to detect later because the tuning rationale lives in nobody’s head. Fit against a sample large enough to be representative, record the fit, and re-run it when the observation set grows.

Peak and off-peak deserve separate treatment where the format warrants it. A grocery catchment measured at Saturday lunchtime and one measured at Tuesday midnight describe different worlds, and the difference in reachable population between them is routinely larger than the difference between two candidate sites. Building two graphs — one at a representative trading-hour speed profile, one at free-flow — costs an extra import and lets a site be assessed at the time its customers actually shop. Where only one can be maintained, trading-hour speeds are the defensible choice for retail work, and the choice belongs in the build metadata rather than in an assumption.

Keeping Several Builds: Storage, Retention and Comparison

A versioned build strategy only works if the previous builds are still there, which raises a modest storage question and a more interesting analytical one.

The storage side is easily settled. A clipped regional graph is a few gigabytes; keeping three builds is a few tens of gigabytes, which is trivial next to the value of being able to roll back in seconds and compare in minutes. Retention beyond three has diminishing returns for operations, though there is a separate case for archiving one build per quarter indefinitely: a catchment computed eighteen months ago can then be reproduced exactly, which is precisely the question an auditor or a disappointed committee eventually asks.

The analytical side is where the value sits. With two builds available, a rebuild stops being an event to be trusted and becomes a measurable change:

  • Which routes moved, and by how much — from the fixed probe set, giving a distribution rather than an anecdote.
  • Which markets moved — by aggregating route changes geographically, which usually reveals that ninety per cent of the movement is in one or two markets with active roadworks.
  • Which stores’ catchments moved — the business-facing version of the same question, expressed in reachable population rather than in seconds.
  • Whether any ranking moved — the only question a real-estate committee actually cares about, and one that can be answered by re-scoring the shortlist on both builds.

That last comparison is worth institutionalising. Most graph rebuilds change no decisions at all, and demonstrating that quickly is what earns the pipeline the freedom to refresh regularly. The occasional rebuild that does move a ranking is then a genuine finding — a new road has changed a market — and it arrives with the evidence attached rather than as an unexplained movement in a number somebody trusted.

Automation and Refresh Cadence

Road networks change continuously and matter episodically. A new arterial or a closed bridge changes catchments materially; a thousand small tag corrections do not. That argues for a monthly rebuild in most markets, with an out-of-cycle rebuild when a known change lands, and it argues strongly against rebuilding nightly: a graph that changes every night makes every catchment non-comparable with yesterday’s for no analytical gain.

The refresh belongs in the orchestration layer as a normal, restartable task with its own validation gate. Its output is a build; the promotion of that build to serving is a separate, deliberate step that a person or a gate approves. That separation is what stops a bad import — a partial download, a profile file with a typo — from silently becoming the graph every catchment is computed on.

Validation and QA Gates

  • Edge and node counts within a band of the previous build. A twenty per cent drop is a partial download, not a quiet month in the map.
  • Component analysis — the share of the network in the largest connected component, and the count of components above a size threshold.
  • Store snapping — every store snaps to a node within a tolerance, and every store lands in the largest component.
  • Known-route regression — a fixed set of origin-destination pairs whose durations must stay within a tolerance of the previous build.
  • Contour area regression — a sample of stores whose fifteen-minute contour areas are compared build to build, with a threshold that triggers review rather than failure.

The last two are the ones that catch subtle profile changes. A speed table edited by hand moves every duration slightly, which no count-based check sees and a regression suite catches immediately.

The regression suite that makes a rebuild safe to promote Across 400 fixed origin-destination pairs, a healthy monthly rebuild moves 372 durations by under one per cent, 24 by one to five per cent, and 4 by more than five — each of the four traced to a specific road change. A build where 90 routes move more than five per cent is a profile change, not a map change. A healthy rebuild barely moves; a broken one moves everything healthy monthly rebuild 372 routes within 1% 24 routes 1–5% · 4 routes over 5%, each traced to a named road change the build that should not be promoted 220 routes within 1% 90 routes over 5% — an edited speed table, not a month of map edits No count-based check distinguishes these two builds: both have plausible node and edge totals, and both import without error. The route set is what tells them apart.

Integration Notes

Downstream, every artifact computed on the graph carries the build version. The contour cache uses it as part of the key, so promoting a new build starts a fresh namespace and the old entries expire on their own. The scoring stage records it alongside the weight version, so a committee question about a moved rank can be answered by comparing versions rather than by re-running anything. And the validation gates that watch for data drift treat a build promotion as an expected step change rather than an anomaly.

Frequently Asked Questions

National extract or regional extracts?

Regional, unless the analysis genuinely spans the nation in one query. Regional extracts import faster, fit in less memory, and can be rebuilt independently, so a change in one market does not force a national re-import. The cost is managing several graphs and routing each request to the right one, which is a small piece of routing logic and worth it — until catchments start crossing extract boundaries routinely, at which point a national graph is simpler than the special cases.

How much does turning off turn restrictions actually change a catchment?

In a dense grid with many one-way streets, ten to fifteen per cent of reachable area, concentrated in exactly the direction the restrictions prevent. In a suburban market with few restrictions, under two per cent. Because the error is systematic rather than random, it does not average out across a candidate pool: it favours urban sites uniformly, which is the kind of bias that survives every sanity check because the map still looks right.

Should the graph include private roads and service ways?

Service ways yes, private roads with care. Retail parks and shopping centres are full of service ways, and excluding them makes stores unreachable from their own car parks — a failure that shows up as an implausibly small contour. Private roads tagged as inaccessible should be excluded, since including them lets the engine route through gated communities and industrial sites, producing catchments no shopper can realise.

What is the right way to test a profile change?

On a fixed route set and a fixed store sample, against the previous build, before promotion. A profile edit is a change to every duration in the market, so the question is never whether it changed anything but whether it changed what was intended. Comparing durations on a route set answers that in seconds, and comparing contour areas on a store sample answers the version of it the business cares about.

How large a team does this need?

Less than the machinery suggests. The build itself is a scheduled job and the validation is a script; what needs a person is the occasional gate failure and the quarterly question of whether the clip and the profile still match the estate. In practice this lands as an hour or two a month of attention once it is running, and the alternative — an engine imported once by whoever set it up, drifting quietly out of date — costs far more in unexplained numbers.

What is the first thing to fix on an inherited setup?

Record the build version. Before tuning a profile, widening a clip or scheduling a refresh, make every contour say which graph produced it — because until that exists, no change can be evaluated and no discrepancy can be explained. It is an afternoon of work on the artifact schema and the pipeline metadata, and it converts every subsequent improvement from an act of faith into a measurement. The refresh schedule, the validation suite and the profile tuning all become straightforward once the pin is in place, and all of them are guesswork until it is.

Conclusion

An extract is not infrastructure to be set up once; it is a versioned input with its own refresh cycle, its own validation and its own failure modes. Clip generously, keep one graph per profile, promote by pointer, pin the build version into every artifact, and gate each rebuild on counts, connectivity and a fixed route regression. Do that and the road network stops being an invisible assumption underneath every catchment, and becomes a dated, reviewable input like any other.

← Back to Isochrone Generation & Network Analysis