Parsing and Standardizing US Store Addresses

This page solves one exact task: converting a column of free-text US addresses into typed, standardized components and a deterministic key, so that three spellings of one storefront collapse to one record before anything spatial touches them.

Parsing is the first half of geocoding and address normalization, and it is the half a pipeline can fully control. A geocoder’s match quality depends on reference data you do not own; a parser’s behaviour depends entirely on the rules you write. Getting it right means the geocoder sees clean, consistent input and the deduplication stage has something stable to compare.

Prerequisites #

• Python packages: usaddress for probabilistic component tagging and pandas for the frame. Install with pip install usaddress pandas. A fuzzy-matching library such as rapidfuzz is useful for the review queue but not required for parsing itself.
• A raw address column. One free-text string per record, ideally with a separate country field. Records that already carry structured components should skip parsing rather than being re-flattened and re-parsed.
• The parent context. This page implements the normalize function described in geocoding and address normalization; read that first for where the stage sits and what the precision tiers mean.

Configuration and execution parameters #

The one setting worth arguing about is abbreviation_direction. Both expansion and contraction produce a consistent result; mixing them does not, and mixing is what happens when one team writes the ingest and another writes the deduplication. Contracting to postal abbreviations is the pragmatic default because it matches how reference datasets store the components you will eventually match against.

Annotated implementation #

The parser below tags components probabilistically, standardizes each one, and refuses to produce a key when the tagging is ambiguous. Refusal is a feature: an ambiguous parse that yields a confident-looking key is how two different shops end up sharing a record.

from __future__ import annotations

import hashlib
import re
from dataclasses import asdict, dataclass

import pandas as pd
import usaddress

# Contract to postal abbreviations. Applied in ONE direction only.
ABBREV = {
    "STREET": "ST", "AVENUE": "AVE", "BOULEVARD": "BLVD", "ROAD": "RD",
    "DRIVE": "DR", "LANE": "LN", "COURT": "CT", "PLACE": "PL",
    "HIGHWAY": "HWY", "PARKWAY": "PKWY", "TERRACE": "TER",
    "NORTH": "N", "SOUTH": "S", "EAST": "E", "WEST": "W",
    "NORTHEAST": "NE", "NORTHWEST": "NW", "SOUTHEAST": "SE", "SOUTHWEST": "SW",
    "SUITE": "STE", "APARTMENT": "APT", "UNIT": "UNIT", "BUILDING": "BLDG",
}
PUNCT = re.compile(r"[.,#]")


@dataclass(frozen=True)
class Parsed:
    number: str
    street: str
    unit: str
    city: str
    region: str
    postcode: str
    confidence: float

    def key(self) -> str:
        parts = [self.number, self.street, self.unit,
                 self.city, self.region, self.postcode]
        return hashlib.sha256("|".join(parts).encode()).hexdigest()[:20]


def _clean(value: str) -> str:
    """Upper-case, strip punctuation, contract known words. Order matters:
    punctuation is removed AFTER tagging so the parser keeps its cues."""
    text = PUNCT.sub(" ", value).upper()
    return " ".join(ABBREV.get(tok, tok) for tok in text.split())


def parse_address(raw: str) -> Parsed | None:
    """Return typed components, or None when the parse is too ambiguous to trust."""
    try:
        tagged, _ = usaddress.tag(raw)
    except usaddress.RepeatedLabelError:
        return None                      # two street names in one string: review it

    def part(*keys: str) -> str:
        vals = [tagged.get(k, "") for k in keys if tagged.get(k)]
        return _clean(" ".join(vals))

    number = part("AddressNumber")
    street = part("StreetNamePreDirectional", "StreetName",
                  "StreetNamePostType", "StreetNamePostDirectional")
    region = part("StateName")
    postcode = part("ZipCode")[:5]

    # A usable US address needs at least a number, a street and one of
    # region or postcode. Anything less cannot be matched reliably.
    filled = sum(bool(x) for x in (number, street, region or postcode))
    if filled < 3:
        return None

    return Parsed(
        number=number,
        street=street,
        unit=part("OccupancyType", "OccupancyIdentifier"),
        city=part("PlaceName"),
        region=region,
        postcode=postcode,
        confidence=round(len(tagged) / (len(tagged) + 1), 3),
    )


def parse_frame(df: pd.DataFrame, column: str = "address_raw") -> pd.DataFrame:
    """Attach parsed components and a key; unparsed rows keep a null key."""
    rows = [parse_address(v) if isinstance(v, str) else None for v in df[column]]
    out = df.copy()
    out["parsed"] = [asdict(r) if r else None for r in rows]
    out["address_key"] = [r.key() if r else None for r in rows]
    out["needs_review"] = [r is None for r in rows]
    return out

The filled < 3 guard is doing real work. A string that tags as a city and a state alone — common when a source system stores a market name in an address column — parses without error and produces a key that will match every other record from that city. Refusing it here means the record reaches the review queue instead of silently merging a market’s worth of stores into one.

Failure modes and debugging #

A street name that looks like a directional. 100 North St is a house number on a street called North; 100 North Main St is a house number on North Main. The tagger usually gets this right and occasionally does not, and the symptom is a street component that lost or gained a leading direction. Where a source is known to produce these, comparing the parsed street against the reference street list for the postcode catches them cheaply.

Unit designators buried in the street line. 1200 Wells St Suite 4 parses cleanly; 1200 Wells St 4 does not, because the trailing token is indistinguishable from a street number suffix. Records from point-of-sale systems are especially prone to this. The tell is a street component ending in a bare number, and the fix is a source-specific pre-clean rather than a global rule.

Post office boxes and non-addressable locations. A box is a valid mailing address and not a place, so it must never reach the geocoder as if it were one. Detect it during parsing, mark the record explicitly, and let the pipeline decide — for a customer file a box is usable at postal-code resolution, and for a store record it is always an error.

Silent normalization drift. Someone adds AVENUE: AVE to the abbreviation map six months after the store table was keyed, and every address containing an avenue produces a new key. Nothing errors; the deduplication stage simply reports thousands of new records. Treat the abbreviation map as versioned configuration, and when it changes, re-key everything in one deliberate pass rather than letting the two conventions coexist.

Verification #

Confirm the parser is behaving before its output is keyed into anything permanent:

• Round-trip a sample by hand. Take fifty records spanning every source, parse them, and read the components. Automated metrics will not tell you that a whole source has its city and state swapped; twenty minutes of reading will.
• Check key stability. Parse the same input twice and confirm identical keys, then parse a deliberately reformatted version of each and confirm it lands on the same key. That second test is the one that proves normalization is doing its job.
• Count keys against records. A sudden fall in distinct keys means over-normalization — usually a component being dropped — and a rise means under-normalization. Both show up immediately in this ratio and nowhere else.
• Assert no key on an unparsed row. A null key must propagate; a row that quietly received a key built from empty components will match every other such row.

Frequently Asked Questions #

Related #

• Geocoding & Address Normalization — where this stage sits and what the precision tiers mean.
• Deduplicating Store Records with Fuzzy Address Matching — the stage this parsing feeds.
• Automating Coordinate Validation with Python and Shapely — the checks applied once coordinates exist.

← Back to Geocoding & Address Normalization