Caching and Rate Limiting Geocoding Requests

This page solves one exact task: putting a persistent result store and a client-side rate limiter in front of a geocoding service, so a pipeline pays for each distinct address once and never overwhelms the provider it depends on.

Geocoding is the stage where a careless pipeline discovers its costs. The same store estate re-geocoded on every nightly run is thousands of wasted requests a week; a customer backfill issued without pacing is a rate-limit wall and, on a self-hosted service, a queue that turns a ten-minute job into a timeout cascade. Both problems have the same two-part answer, and both are cheaper to build than to retrofit.

Prerequisites #

• Python packages: httpx for the client, redis or a database of your choice for the shared store, and pandas for batch work. Install with pip install httpx redis pandas.
• Normalized addresses with a stable key from parsing and standardizing US store addresses. Caching on the raw string is the mistake this whole page exists to prevent.
• A known provider limit — requests per second, per minute or per day — and, for a self-hosted service, a measured saturation point rather than a guessed one.

Configuration and execution parameters #

Two of those settings are routinely conflated. Rate limits how many requests start per second; concurrency limits how many are outstanding at once. A pipeline with a 10 requests-per-second limit and no concurrency cap will, when the provider slows down, accumulate hundreds of in-flight requests and time them all out simultaneously. Both controls are needed, and they fail differently when missing.

Annotated implementation #

The client below composes the three pieces — store lookup, single-flight guard, token bucket — around whatever geocoding call sits underneath.

from __future__ import annotations

import asyncio
import json
import time

import httpx
import redis.asyncio as redis

REFERENCE_BUILD = "openaddresses-2026-07-31"
NEGATIVE_TTL = 60 * 60 * 24 * 30          # a no-match is a result worth keeping
RATE_QPS = 8.0
BURST = 16


class TokenBucket:
    """Paces request starts. Independent of how many are in flight."""

    def __init__(self, rate: float, burst: int) -> None:
        self._rate, self._capacity = rate, float(burst)
        self._tokens, self._updated = float(burst), time.monotonic()
        self._lock = asyncio.Lock()

    async def take(self) -> None:
        async with self._lock:
            now = time.monotonic()
            self._tokens = min(self._capacity,
                               self._tokens + (now - self._updated) * self._rate)
            self._updated = now
            if self._tokens < 1.0:
                wait = (1.0 - self._tokens) / self._rate
                await asyncio.sleep(wait)
                self._tokens = 0.0
            else:
                self._tokens -= 1.0


class GeocodeCache:
    def __init__(self, store: redis.Redis, bucket: TokenBucket) -> None:
        self.store, self.bucket = store, bucket
        self._inflight: dict[str, asyncio.Future] = {}

    def _key(self, address_key: str) -> str:
        # The reference build is part of the key: a new build is a new cache,
        # and the old entries expire on their own rather than being purged.
        return f"geo:{REFERENCE_BUILD}:{address_key}"

    async def resolve(self, address_key: str, components: dict,
                      client: httpx.AsyncClient) -> dict | None:
        key = self._key(address_key)
        cached = await self.store.get(key)
        if cached is not None:
            payload = json.loads(cached)
            return payload or None            # {} encodes a remembered no-match

        # Single flight: concurrent callers for one key share one request.
        if key in self._inflight:
            return await self._inflight[key]

        loop = asyncio.get_running_loop()
        fut: asyncio.Future = loop.create_future()
        self._inflight[key] = fut
        try:
            await self.bucket.take()
            resp = await client.get("/v1/search/structured", params=components,
                                    timeout=20.0)
            resp.raise_for_status()
            hits = resp.json().get("features") or []
            result = hits[0] if hits else None
            await self.store.set(key, json.dumps(result or {}),
                                 ex=None if result else NEGATIVE_TTL)
            fut.set_result(result)
            return result
        except Exception as exc:              # do not cache transport failures
            fut.set_exception(exc)
            raise
        finally:
            self._inflight.pop(key, None)

Two decisions in that class are the ones worth copying. A transport failure is never cached — caching it would turn a thirty-second network blip into thirty days of missing coordinates. And a positive result has no expiry, because its validity is tied to the reference build already in the key; giving it a time-based expiry would re-request unchanged data on a schedule that has nothing to do with when the answer could change.

Failure modes and debugging #

A hit rate that stays near zero. Almost always a key that varies when it should not — the raw string instead of components, a timestamp, a request identifier, or full float precision on a coordinate. Log a sample of keys and read them; the offending component is usually obvious within ten lines.

The thundering herd after a build change. Changing the reference build invalidates every entry at once, so the first run afterwards is a full cold pass. That is correct, and it should be expected rather than discovered: schedule the first post-build run when the provider can take it, or pre-warm the store from the previous build’s key set.

Retry storms. A provider returning 429 to a client that retries immediately produces more 429s. Honouring Retry-After, adding jitter so parallel workers do not resynchronise, and capping total attempts converts a storm into a slower but successful run. The retry policy discipline that applies to pipeline tasks applies here at request granularity.

An unbounded in-flight map. The single-flight dictionary grows with distinct keys in flight and shrinks as they complete — unless an exception path forgets to remove the entry. The finally block above is not decoration; without it, a run that hits errors leaks futures until it exhausts memory.

Verification #

• Assert the key is stable. Build the key twice from differently-formatted versions of one address and confirm they match. This is the single test that protects the whole mechanism.
• Confirm negative caching works. Geocode a deliberately unmatchable address twice and confirm the second call issues no request. Without negative caching, the worst addresses are retried forever.
• Measure the achieved rate under load. Run a batch and count request starts per second against the configured limit. A limiter that is never reached is either badly configured or the workload is smaller than assumed — both worth knowing.
• Test the single-flight guard. Issue fifty concurrent requests for one key and confirm exactly one reaches the provider. Concurrency bugs in this layer are invisible until a stampede.

Frequently Asked Questions #

Related #

• Batch Geocoding with a Self-Hosted Service — the batch client this layer sits in front of.
• Geocoding & Address Normalization — where the normalized key comes from.
• Caching Strategies for Repeated Network Queries — the same pattern applied to routing.

← Back to Geocoding & Address Normalization