Geocoder · historical documents · calibrated uncertainty

HistGeocoder

Historical records name places that no longer exist, in language that isn't a place name at all. This turns that language into a coordinate — and tells you how far off it might be.

In development · design frozen, implementation underway

"3 mi. E of San Jose, Cal." · 1906 Plan view · WGS84 geodesic

Stated distance 3 mi

Our radius: —
Point-pin error: —
σ bearing: —
σ distance: —
σ anchor: —

The two diagrams on this page are drawn with JavaScript. With it disabled you can still read everything else — the pipeline, the error model, the API, and the current state of the build.

Drag the distance. The wedge is the real geometry of the word east — a 45° arc, not a ray — so bearing error grows as arc length while the anchor's own extent stays fixed. A commercial geocoder returns the red pin at San Jose and reports nothing about the gap.

01 The problem

Two assumptions that historical text breaks

Every commercial geocoder assumes the place exists today, and that the text names it directly. One line from a 1906 earthquake report violates both.

Severe at Agnew's Asylum, 3 mi. E of San Jose, Cal. USGS earthquake intensity report · 1906

Agnew's Asylum was destroyed in that earthquake and appears in no modern gazetteer. And "3 mi. E of San Jose" is not a name — it is a relative offset, a direction and a distance measured from somewhere else.

What a commercial API returns

37.3382, −121.8863

Snapped to San Jose city hall
The offset silently discarded
Four decimals of false precision
No signal that it may be miles wrong

What HistGeocoder returns

37.3960, −121.7856 ± 4.2 km

Anchor resolved, offset applied on the ellipsoid
A radius decomposed into four error sources
An abstained flag when it shouldn't be trusted
A trace a scientist can audit line by line

02 Pipeline

Seven stages, in order

Stages one through six run per request. The seventh needs a whole document at once, because its evidence is the other mentions.

01rules

Normalizedeterministic

Expand period abbreviations, fold Unicode, repair the OCR confusions that actually occur — rn read as m, Sta. for Santa. Rules derived from 200 real archive strings, not an imagined table of 400.

02llm

Parse under a schemaconstrained decoding

The JSON schema compiles to a finite state machine; at every generation step the logits of illegal tokens are set to −∞. Invalid output isn't retried — it carries zero probability. Token log-probabilities are kept as per-field confidence for stage six.

03postgres

Retrieve candidatesrecall first

Three searches unioned: trigram similarity over a GIN index, double-metaphone for names written by ear, and exact hits on historical variant names. Fifty candidates come back — missing the right one here is fatal, carrying forty-nine wrong ones costs almost nothing.

04scored

Ranklinear, then logistic

Jaro-Winkler, trigram score, phonetic hit, feature-class prior, log population, admin agreement, temporal fit, name-source weight. Hand-weighted first so there is a baseline; logistic regression second so the gain is measurable. The runner-up margin feeds uncertainty.

05pyproj

Project the offsetgeodesic direct

Anchor, azimuth, distance → a point on the WGS84 ellipsoid. Not degrees added to a longitude. Sanity checks follow: same state, on land, plausible range. Failing one drops confidence hard.

06conformal

Quantify and abstainthe point of the project

Four error terms combined in quadrature, then a calibrated threshold that decides whether to answer at all. Abstention is a 200 response carrying abstained: true, not an error.

07batch

Document coherencetwo passes

Mentions in one report cluster geographically. A Springfield that landed in Illinois inside a Bay Area document is almost certainly wrong, so low-confidence results are rescored against the high-confidence centroid — and the pass is skipped entirely when that centroid rests on too few or too scattered points.

03 Uncertainty

An honest radius, and the right to stay silent

Four independent error sources, added the way independent errors add — in quadrature.

src/histgeocoder/uncertainty/radius.py

# a city is a big target; a survey marker is not
sigma_anchor   = anchor_feature_extent_m
# "about 3 miles" is looser than "3 mi."
sigma_distance = distance_m * distance_rel_err
# arc length of the compass wedge — grows with distance
sigma_bearing  = distance_m * radians(wedge_deg / 2)
# how close the runner-up candidate came
sigma_match    = (1 - rank_margin) * MATCH_SCALE

radius = sqrt(sigma_anchor**2 + sigma_distance**2
              + sigma_bearing**2 + sigma_match**2)

The bearing term is the interesting one. Three miles east is a narrow arc; twenty miles east is an enormous one. That falls out of the geometry for free — it is the wedge in the diagram above, widening as you drag.

Calibrated abstention

A confidence score of 0.83 means nothing until it is calibrated. Split conformal prediction fixes that with a held-out calibration set: choose a tolerance α, then find the threshold τ where the error rate among accepted answers sits at α — using the finite-sample quantile ⌈(n+1)(1−α)⌉ / n, which is the correction that makes the guarantee hold on data the system has never seen.

Calibration set · accept when confidence ≥ τ Drag τ, or pick a tolerance

Tolerance α

Threshold τ: —
Coverage: —
Risk if accepted: —
Sent to a human: —

Synthetic points, drawn to make the mechanism legible — these are not results. Green landed within tolerance of truth, red did not. Raising α accepts more and risks more; lowering it hands more records to a person.

What the guarantee actually promises. Among the records the system does not abstain on, no more than α exceed the error tolerance — assuming the calibration data and the live data are exchangeable. Distribution shift breaks it, and a gazetteer calibrated on California then pointed at Missouri is exactly that shift.

04 Gazetteer

A gazetteer that knows when places existed

Two tables, because one place carries many names — official, variant, historical, colloquial — and the historical ones are the whole point.

Source	What it contributes	License
GNIS	Official US names, feature classes, counties, and the variant & historical names file — the part commercial geocoders lack	Public domain
GeoNames	Alternate names and population figures the ranker leans on	CC BY 4.0
OpenStreetMap	Streets and buildings the gazetteers miss — an enrichment layer that can be switched off	ODbL, share-alike
Wikidata	Founding and dissolution dates for settlements	CC0

src/histgeocoder/store/schema.sql · excerpt

CREATE TABLE place (
    id            BIGSERIAL PRIMARY KEY,
    feature_class TEXT,
    extent_m      INT,     -- feeds sigma_anchor
    valid_from    INT,     -- NULL = unknown or always
    valid_to      INT,     -- NULL = still exists
    geom          GEOGRAPHY(POINT, 4326) NOT NULL
);

CREATE INDEX place_name_trgm ON place_name
    USING GIN (name_norm gin_trgm_ops);

Geography: Not geometry. GEOGRAPHY measures on the sphere and returns meters. GEOMETRY measures on a flat plane in degrees, which is wrong at continental scale and fails quietly.
Validity: The valid_from and valid_to columns are what let the system answer did this place exist in 1906. Most rows will be NULL; unknown is recorded as unknown rather than guessed.
Loading: One idempotent script per source, all writing the same two tables, all reachable from make data on a fresh clone.

05 Interface

GeoJSON in, GeoJSON out

Every mapping tool consumes GeoJSON without conversion. No coordinate leaves without a radius attached to it.

POST /v1/geocode

{
  "text": "Severe at Agnew's Asylum, 3 mi. E of San Jose, Cal.",
  "year": 1906,
  "alpha": 0.10,
  "explain": true
}

200 OK · GeoJSON Feature

{
  "type": "Feature",
  "geometry": { "type": "Point",
                "coordinates": [-121.7856, 37.3960] },
  "properties": {
    "confidence": 0.83,
    "uncertainty_radius_m": 4200,
    "abstained": false,
    "method": "offset_from_anchor",
    "anchor": { "name": "San Jose", "source": "gnis", "id": 277577 },
    "offset": { "distance_m": 4828, "bearing_deg": 90,
                 "bearing_wedge_deg": 45 },
    "alternatives": [
      { "name": "San Jose", "score": 0.61,
        "reason": "anchor_only_fallback" }
    ]
  }
}

Endpoint	Purpose
POST /v1/geocode	One string
POST /v1/geocode/batch	A document at once — coherence needs the whole thing, so this is one request, not N
GET /v1/places/{id}	Inspect a gazetteer entry
GET /v1/health	Liveness
GET /metrics	Prometheus scrape

With explain: true the response carries a trace: per-stage timings, the raw parser struct, and the top five candidates with their individual feature scores. It is both the debugging tool and the demo.

06 Scoreboard

The benchmark, built before the ranker

A gold set of 200–500 labeled historical strings, split 60 / 20 / 20 — development, conformal calibration, and a held-out test touched exactly once.

System	Median err	p90 err	Within 10 km	Coverage
HistGeocoder, full	—	—	—	—
HistGeocoder, LLM disabled	—	—	—	—
Google Geocoding API	—	—	—	—
Nominatim	—	—	—	—

No numbers yet. The harness and its metrics are specified; nothing has been run, so every cell is empty rather than estimated. They get filled from make eval output and from nowhere else.

The second row is the baseline most projects skip and the one worth having: the same pipeline with the LLM parser switched off, isolating exactly how much the model contributed. If that gap turns out to be three percent, three percent is what gets published.

07 Build

Stack and current state

Service: Python 3.12 · FastAPI · Pydantic v2 — one schema definition drives both the API contract and the LLM output grammar
Store: PostgreSQL 16 · PostGIS · pg_trgm · fuzzystrmatch — spatial, fuzzy, and phonetic search in one engine
Model: llama.cpp with GBNF grammars, or vLLM with Outlines · Gemini responseSchema kept as a comparison backend
Math: pyproj for geodesy · RapidFuzz for string similarity · scikit-learn for the ranker
Cache: Redis 7 — parse results, full results, and candidate sets, each keyed by a hash of its input
Testing: pytest · Hypothesis for geodesy round-trips · testcontainers against real PostGIS, because the bugs live in the SQL

Where it stands

doneDesign document — pipeline, schema, error model, and evaluation protocol specified end to end
nextCompose stack, schema migrations, GNIS and GeoNames loaded for California
nextGold set assembled and split; evaluation harness printing a table before any ranker exists
thenConstrained parser, retrieval, ranker, geodesic projection, uncertainty model
thenService, caching, CI, container, deployment, load test
laterConformal calibration, document coherence, temporal filtering, coverage beyond California

The deployment problem, stated up front. A 4-bit 7B model is roughly 5 GB, which is a slow cold start on a scale-to-zero platform. Version one ships with the LLM stage behind a flag: the public demo runs the classical path, and the model-enabled benchmark numbers come from local runs. Honest, cheap, and it works.