Location Intelligence Retail Site Selection Automation

Documenting Model Assumptions for Investment Committees

This page solves one exact task: producing a short, generated document that accompanies every site-selection recommendation and lets a non-technical committee interrogate the model behind it — what it measures, what it assumes, how accurate it has been, and the circumstances in which it should not be relied on.

The document is not a courtesy. A committee approving several million pounds of capital on a ranked list is entitled to know how that list was produced, and the absence of a plain-language description is what allows a score to be treated either as gospel or as noise, depending on whether it agrees with the room.

Prerequisites

  • A validation history with measured accuracy, from tracking model decay across refresh cycles.
  • The versioned configuration: weights, decay parameters, criterion definitions, and the input versions each run used.
  • A documented list of known limitations, maintained as the model is developed rather than reconstructed at the end.
  • A template and a generator — the document should be produced from the artifacts, not typed.

Configuration and execution parameters

Section Content Length
What it predicts The outcome, in the units a committee uses 2 sentences
What goes in The criteria and their weights, plainly named A table
Key assumptions The three or four that drive the answer 4 bullets
Measured accuracy Correlation, decile lift, signed bias, and the design A table
Where it is weak Segments and situations with poor accuracy 3 bullets
What it does not consider Explicit exclusions 4 bullets
Version and date Every input version, and when accuracy was last measured 1 line

The section that changes discussions most is what it does not consider. Committees routinely assume a site model has weighed things it has never seen — landlord quality, staffing availability, planning risk, the chief executive’s view of the town — and stating the exclusions moves those factors into the discussion where they belong instead of leaving them silently assumed.

One page, seven sections, all generated The model card places what it predicts and what goes in at the top, key assumptions and measured accuracy in the middle, and where it is weak plus what it does not consider at the bottom, with the version line last. Every figure is generated from the validation record rather than written by hand. Everything a committee needs, on one page What it predicts · year-two sales per square metre, ranked What goes in 5 criteria and their weights named in plain language Key assumptions decay, capture, competitive set, remote-working share Measured accuracy · rank correlation 0.52 prospective · top decile +41% signed bias −4% · measured on 214 stores, last updated this quarter Where it is weak city centres · new formats · rural What it does not consider lease terms · planning · staffing Version line: weights w-2026-03 · demographics acs-2024-5yr · graph 2026-08-01_car_4b1c · accuracy measured 2026-07-31

Annotated implementation

The card is generated, because a hand-written one drifts from the model within a quarter.

python
from __future__ import annotations

from dataclasses import dataclass
from datetime import date

import pandas as pd


@dataclass(frozen=True)
class ModelCard:
    predicts: str
    criteria: pd.DataFrame          # name, plain_language, weight
    assumptions: list[str]
    accuracy: pd.DataFrame          # design, metric, value, n
    weaknesses: list[str]
    exclusions: list[str]
    versions: dict[str, str]
    measured_on: date

    def to_markdown(self) -> str:
        parts = [
            f"## What this model predicts\n\n{self.predicts}\n",
            "## What goes into the score\n\n" + self.criteria.to_markdown(index=False),
            "## Key assumptions\n\n" + "\n".join(f"- {a}" for a in self.assumptions),
            "## Measured accuracy\n\n" + self.accuracy.to_markdown(index=False),
            "## Where this model is weak\n\n" + "\n".join(f"- {w}" for w in self.weaknesses),
            "## What this model does not consider\n\n"
            + "\n".join(f"- {e}" for e in self.exclusions),
            "## Versions\n\n"
            + " · ".join(f"{k}: {v}" for k, v in sorted(self.versions.items()))
            + f"\n\nAccuracy last measured {self.measured_on.isoformat()}.",
        ]
        return "\n\n".join(parts)


def build_card(config: dict, validation: pd.DataFrame,
               limitations: list[dict]) -> ModelCard:
    """Assemble from artifacts. Nothing here is typed by a human at build time."""
    latest = validation.sort_values("quarter").groupby("design").tail(1)

    return ModelCard(
        predicts=config["predicts_statement"],
        criteria=pd.DataFrame(config["criteria"]),
        assumptions=[a["statement"] for a in config["assumptions"]],
        accuracy=latest[["design", "metric", "value", "n"]],
        # Weaknesses come from the segmented validation, not from memory.
        weaknesses=[l["statement"] for l in limitations if l["kind"] == "weak"],
        exclusions=[l["statement"] for l in limitations if l["kind"] == "excluded"],
        versions=config["input_versions"],
        measured_on=latest["measured_on"].max(),
    )

Deriving the weaknesses from the segmented validation rather than from a maintained list is the detail that keeps the card current. A segment whose accuracy falls below a threshold appears in the weaknesses automatically, which means the card tells the committee about a problem in the same quarter the modelling team learns of it.

Failure modes and debugging

A card that describes the intended model. Written during development and never regenerated, it describes weights that have since been refitted and an accuracy that was aspirational. Generating from the configuration and the validation record removes the possibility.

Accuracy quoted without its design. “Correlation 0.78” means one thing in sample and another prospectively, and quoting the flattering figure without the design is the most common way a card misleads without containing a false statement. Every accuracy row should carry its design and its sample size.

Exclusions written defensively. A list of exclusions that reads as a disclaimer invites the committee to discount the model entirely. Written as a division of labour — the model handles catchment and competition, the committee handles lease, covenant and operational risk — the same list strengthens the recommendation.

No named owner. A document with no author is a document nobody can be asked about. The card should name the person accountable for the model, which is also what makes the annual review happen.

The same fact, framed two ways Written defensively, a limitation reads as a warning that the model may be unreliable in city centres. Written as a division of labour, the same fact says the model measures drive-time catchments and city-centre sites need the pedestrian and transit assessment the property team already produces. A limitation is either a disclaimer or an instruction defensive "The model may be less accurate for city-centre locations and its results should be treated with caution." → the committee discounts everything a division of labour "The score measures drive-time catchments. City-centre sites need the pedestrian and transit review the property team produces." → the committee knows what to ask for Both sentences describe the same measured weakness. Only the second tells anyone what to do about it, and only the second survives contact with a committee that has a decision to make.

Verification

  • Regenerate the card and diff it against the published version before every committee cycle. A diff of zero when the model was refitted means the generator is reading a stale artifact.
  • Check every number appears in the validation record. Any figure in the card that cannot be traced to a measurement is an assertion.
  • Ask someone outside the team to read it and say what the model does. If they cannot, the plain-language column is not plain enough.
  • Confirm the weaknesses list changes when a segment’s accuracy falls, by injecting a synthetic decline.
The questions change once the card exists Before the card, committee questions were mostly about whether the model could be trusted at all. After it, the questions moved to the assumptions — the remote-working share, the competitive set, the decay parameter — which are questions the modelling team can answer and act on. From "can we trust this?" to "is this assumption right?" before the card "How does the model know that?" "Has this ever been right?" "Did it look at the lease?" after the card "Is 22% the right remote-working share for this market?" "Does the competitor set include the new discounter?" "The card says city centres are weak — what should we do about site 4?"

Frequently Asked Questions

How long should the card be?

One page, and the constraint is the point. A ten-page methodology document is read by nobody and therefore protects nobody; a single page with seven short sections is read in the meeting. Everything that does not fit belongs in a technical appendix that the card links to, which is where the modelling team’s own detail should live anyway.

Should the card include the weights?

Yes, with plain-language names. Committees are perfectly capable of engaging with the statement that reachable population carries thirty per cent and rent ten, and that engagement is exactly what should happen — the weights encode a business judgement, and the committee is who owns that judgement. Hiding them behind a composite score removes the one part of the model non-technical stakeholders are best placed to challenge.

What if the measured accuracy is poor?

Publish it. A model with a rank correlation of 0.45 that is honestly reported is more useful than one claiming 0.8 on an in-sample fit, because the committee can weight it appropriately against the other evidence. The temptation to withhold a disappointing number is also the mechanism by which a model’s credibility collapses the first time somebody computes it independently.

Who should own the card?

The person accountable for the model, with the validation figures produced by someone else — the same separation that keeps the validation honest. The card is generated, so ownership is about maintaining the assumptions and limitations lists rather than about writing prose, which is a small and genuinely useful responsibility.

How often should the card be reissued?

With every committee cycle, regenerated rather than reviewed. The cost of regeneration is nil once the generator exists, and reissuing it each cycle means the accuracy figures, the input versions and the weakness list are always the current ones. A card reissued only when something changes invites the question of whether nothing changed or nobody checked.

What belongs in the technical appendix rather than the card?

The derivations, the full criterion definitions, the fitting method, the validation designs in detail, and the history of changes. Everything the modelling team needs to reproduce the work, none of which a committee needs to make a decision. Keeping the split clean is what allows the card to stay at one page — and the appendix, being written for a technical reader, can be as long as the subject deserves.

Does the card change how the model is used?

In practice, yes, and mostly by narrowing its use to where it works. Committees given a segmented accuracy table start asking for the model’s view where it is strong and for other evidence where it is weak, which is exactly the behaviour a modelling team wants and rarely gets by asserting it. The card also tends to shorten meetings, because the questions it pre-empts are the ones that previously consumed the first twenty minutes.

What if the committee never reads it?

Then attach it to the decision rather than to the pack. A card referenced in the minutes — the model version, its accuracy, its stated weaknesses — becomes part of the record of why an approval was given, which is where its value ultimately sits. Committees that skip the document during the meeting still benefit from its existence the first time a decision is revisited.

← Back to Validating & Backtesting Site Selection Models