Rebuild familiar forms from reviewed ORCID pairs, add opt-in alternate-spelling, and fold keyboard spellings - #1379

Rebuilds the familiar forms #1372 dropped from pairs found in CC0 data and reviewed, adds an opt-in outcome for a name spelled two ways by choice, and folds the spellings a keyboard forces into the default.

What changes for a caller

A chosen spelling is not default: people sometimes spell a name differently on purpose, to be told apart. Only a spelling no one chooses, which a keyboard or an encoding forces, is folded by default.

Keys change for names holding ae, oe, ue, aa or dj (decided, as its cost is modest; matching is new in 3.1, so no caller has stored a key). A key now writes those digraphs as their first letter (Doe keys do), so every spelling of Müller cuts one key and a name's keys don't grow 2^k with its forms. Measured under the default setting, master against this PR:

• Largest growth: the folds' jorgensen;m, 5 → 38 (Jørgensen / Joergensen / Jorgensen, which the folds now match). The collapse's own merges are about 2× at most: sad;* (Saad / Saeed) and xu;y (Xue / Xu).
• Doe and Do: do;* grows from 363 to 412 names in orcid (largest 112 → 118) and from 43 to 101 in psc.
• Keys per name: unchanged from master (at most 18 on one name). One key per spelling form would give up to 76.

How the pairs were found and reviewed

• tools/vocab/familiar_candidates.py finds a pair where one ORCID holder (public data file, CC0) wrote a one-word given-name box and, before the same family name, a different word in a published name (given = Katarzyna, credit = Kasia Nowak). It reads no compiled list of familiar forms except the nicknames list and its refusals, to skip what was already reviewed (decisions/shipped-data-licences, FACTS WE MEASURE). The nicknames list's 165 pairs it had refused as spellings were reviewed again.
• Review: two independent review agents read each of the 1,633 candidates under one written rubric, the second on another model (Sonnet 5.5). They agreed on 1,495 (91.5%). A pair ships where both agree, or where the maintainer decided a split. Every verdict is in tools/vocab/familiar_reviewed.tsv, with no family names.
• Routing:
  • folded by default: 66 spellings a keyboard forces, which ship no reading;
  • alternate-spelling: 429 other spellings;
  • familiar-form: 668 pairs;
  • recorded only: translations (235), abbreviations (7) and different names (116);
  • refused: 30.

Measured (tools/differential/match_diff.py, 31,247 pairs × 22 settings)

No fold makes any verdict inconsistent. Two verdicts go from consistent to inconsistent, both from a fix to how a middle name's words combine: an accepted weaker outcome no longer hides a stronger one the caller refuses. They are John Liz Jean Smith / John Elizabeth Jean-Paul Smith under +hyphenated-part alone, and an accented middle name under -folded. The parse snapshot is byte-identical: no parse moves.

Design

• A new matching-only kind, spelling = { spelling_of = [...] }, compiling into ALTERNATE_SPELLINGS alone, not a given reading's same_as, which means "written short" and moves parses (measured in the decisions entry).
• Pairs are read either way round and never chained, as familiar forms are.
• A given name needing both an alternate spelling and a familiar form is accepted only where the caller accepts both.

The records are in decisions/name-matching: "2026-11-12 A KEYBOARD'S SPELLING IS FOLDED" and "2026-11-12 ALTERNATE SPELLINGS ARE OPT-IN".

Open, not built: a translation outcome, cross-script matching, phonetic matching (Gerry / Jerry), routing abbreviations (Md, which moves parses), a caller's same_as reached from other forms of the word, and Wikidata for the languages ORCID covers thinly (German, Russian, Ukrainian, Belarusian, Hungarian, Romanian, Bulgarian, Croatian, Serbian, Albanian; figures in the entry).

Tests and reviews

• Checks: the full suite passes (46,922), as do ruff, mypy, the Sphinx doctests and the README doctest.
• Controls: every new guard has a negative control that fails with its code removed.
• Reviews:
  • a design-docs review and a code review of the change;
  • a review of each of the three fix commits;
  • every finding fixed, or recorded as Open where noted.

🤖 Generated with Claude Code