Add Policy.initials_as_family, reading letters ending a name as its family (#1254) - #1347

The second of three PRs implementing the #1254 decision (its decision comment is on the issue). It follows #1345 (the suggester's trailing-initial gate and crash fix) and branches from master on its own.

What it does

A new opt-in switch, Policy.initials_as_family, off by default (rules/O8). It reads two or more letters ending a name as its family name:

• Lakshmi R S → given Lakshmi, family R S
• Ram Bahadur B K → middle Bahadur, family B K

The default is unchanged: middle R, family S, as in 1.4.0, 2.0.0, 2.8.0 and 3.0.1 (run against each wheel).

What it reads: two or more letters in a row, each in the shape rules/T5 reads as an initial (bare, dotted or mixed), ending the name words, after a name word.

What it leaves alone:

• one letter (Kavitha S)
• letters before or inside the name (R S Lakshmi, John A B Smith)
• a name of nothing but letters (J R S)
• the given part after a family comma (Lakshmi, R S)
• an empty head (, R S)
• a name read family-first: under both family-first orders a byline written surname first (Huntley R P) already reads right, and the rule stands down

Also covered by the rule file:

• titles, suffixes, credentials, nicknames and maiden clauses after the run
• names written wholly in one case (rules/T4)
• a connective between the letters (P3/P8), which already joins them
• a one-letter particle (P7) or a nickname between the letters, either of which breaks the run
• precedence against O6, O7 and O3

It runs in _post_rules before O6. With the switch off it is one attribute test and no call.

locales.NE turns it on, through a [rules] table in ne.toml. The pack now declares a DEVIATES for its patch: _policy.initials_as_family_may_move, which the suggester's cheap pre-test shares, dividing words at every character that is not a letter or a combining mark.

Parser.suggest turns it on where one run of letters ends at least 2 of the group's names and at least half of the names ending in a run. A group decided so casts no trailing-initial vote on its order, and the order evidence's note says so. The evidence is the initials_as_family entry of Suggestion.evidence: no per-name vote, a note naming the letters. A group never turns the switch off, and a base reading names family first (FAMILY_FIRST, FAMILY_FIRST_GIVEN_LAST, locales.VI) is suggested nothing and its names are not read for runs, the rule standing down there.

Two kinds of run count:

• A spaced run, counted where rules/O8 moves the name. This is asked of the parse itself, with the switch on and off, so the signal reads exactly where the rule does.
• A run joined with full stops alone (B.K.), counted only where the parse makes it the whole family; a hyphen (J.-P., J-H) joins one given name and does not count.

Measurements

The pack

config_diff.py diff --local nepal-candidates "plain:NE@initials_as_family=0" "plain:NE": 552 of 237,408 spellings move, every one from middle to family. Of the corpus's 623 spellings ending in a spaced run of Latin letters (B K 494, K C 85), 543 move; the other 80 hold a lower-case letter (Arjun B k), no initial among capitals (T4). The other 9 moved carry a bracket or comma beside the letters, or are the one Devanagari name with dotted letters. All 551 romanized names were read against the Devanagari beside each:

• 508 write the same letters
• 23 write them joined, with joiners or with commas
• 20 write the surname in full (विश्वकर्मा), or add a clan name in brackets
• of those 20, 2 write another surname of the same group

cldr_agreement.py: NE 19/24 → 19/24, ne-np 10/10 → 10/10. Every record matches.

Switch off

The committed snapshot changes only by the rule's 27 example names, new to the rules corpus. snapshot.py --local <source> --base <checkout of 62bbdeb1> matches the base, at the final head, for:

• nepal-candidates, ctgov-officials, tse-donors, jrc-names
• psc-individuals, orcid-researchers, patentsview-inventors, fec-individuals

opensanctions-parliaments was not read: its cache predates six of the datasets the tool reads, and a fetch would rewrite the shared cache.

The suggester

suggest_measure.py --base <checkout of 62bbdeb1> (re-taken after the review) moved one cell: PatentsView's Dutch rows written family-first under a given-first base, 38.7% → 38.3% at 2 names and 92.0% → 91.7% at 5. No same-order row, no parliament list and no TURKIC row moved. The tool now counts the groups deciding the switch, at most 11 of 9,600 per population (Dutch), every one under the given-first base. With --set _MIN_RUN_NAMES=1000000, none.

On the study's populations (half B of each, 200 groups a size, sizes 2–200; scripts in the session notes):

• Nepal, all initialed: gets the switch in 77 / 94 / 99 / 99 / 100% of groups at 2–5 names and 100% from 10. Names read right rise 77% → 97%.
• Whole real lists: at most 1% of groups.
• Bylines: at most 3%. Brazil: at most 2%.
• Stacked on Gate the suggester's trailing-initial vote on the group's share; fix a title-only name's crash #1345 (e9dfab2): Nepal is made worse in at most 2% of groups at every size.

Sweeps

All run on this branch:

• the four comma shapes and the three orders
• bare, dotted, mixed, lower-case and all-capitals letters; 2, 3 and 4 letters
• a title in front; a suffix, credential, numeral, nickname or maiden clause after the run
• NFD, lone_name_word, every pack with the switch on
• the views: initials() is unchanged (L. R. S.), capitalized() is unchanged, surname2 is empty, and each letter's Reading is initial as family
• Parser.revise
• Parser.matching().check: under the switch Lakshmi R S and Lakshmi RS are consistent (joined); with it off they conflict

Tests

• cases.py: a default row (parity, measured on the four wheels) and nine feat(#1254) rows, one per comma shape and order, plus the pack.
• The rule file's examples.
• Pickles: a 3.0.1 pickle of Policy and of PolicyPatch (written by the wheel) loads with the default, and the refused-skew rows are added.
• The CLI flag, and the field rosters in test_parser, test_policy, rules_doc and test_vocab.
• The suggester: the run is read exactly where the rule moves a name; which spellings count; the decision's edges; the order vote set aside; evidence counts; a family-first base; and recorded controls for _MIN_RUN_NAMES and _MIN_RUN_SHARE.

Decisions for the maintainer

• Name and type: initials_as_family: bool, beside two_family_names and two_given_names. An enum (positional / letters) would leave room for a byline value, which the decision declined. Other names considered: trailing_initials_family, family_initials.
• Rule id O8, in the O section with the other positional switches (O3, O6, O7). A T-section id was also possible, since the letters are read by T5's shape.
• The suggester's evidence is a group property with no per-name vote. SettingEvidence.value may now be a bool, and setting may name initials_as_family.
• The shipped suggester test differs from the prototype in two ways. It counts joined runs only where the parse makes them the family, and never bare capitals (BK), which are written the way a short surname is (Zhi LI).
• Counting joined runs is wider than where the rule acts, against the Suggest settings for a group of names (Parser.suggest) #1310 lesson as written. It is deliberate: three quarters of Nepal's runs are joined, and spaced-only evidence left groups of ten undecided about half the time. The setting still moves only spaced runs.
• A decided switch is inert under a family-first base. The rule stands down there; a test pins it.
• Other packs: none turned on.
  • The hi pack's nearest population, India's Lok Sabha: 14 of its 8,805 spellings with no comma end in a spaced run (10 bare, 4 dotted), and 19 more follow a family comma, every one a South Indian name.
  • ORCID's credit names from India hold 40: 22 letters-the-family, 7 bylines, 2 middle-and-family.
  • Tamil, Kannada, Malayalam and Telugu packs, which do not exist, are where the switch belongs.

Docs

• docs/policy.rst: a table row, and a new section Names ending in initials that are the family name.
• docs/locales.rst: the ne row; the "choice and nothing else" list; the DEVIATES example now names hi.
• docs/suggest.rst: a paragraph and doctest on the new setting.
• docs/usage.rst: the CLI flag list.
• The release log, one bullet.
• decisions/O8 (new), decisions/group-suggestion (a LETTER RUNS entry and its Standing line), rules/O8 (new), and both indexes.
• Root AGENTS.md: the lesson that ORCID's name is composed from the boxes and cannot show which word a writer put first, plus the ADDED_IN, [rules] and pack rosters.
• nameparser/vocab/README.md's rule-key table.

Review round 1 (fix commit 29e86c7 on top of f20211f)

1. DEVIATES and the suggester's pre-test under-declared names whose letters the parse divides where str.split() does not (Anjana K C(Maharjan), Bimala Kumari Singh (B. K.), Ram B K,Jr, U+200B, U+FEFF, U+2063, ・, ፡): all reproduced. One shared predicate now divides at every non-letter; a test holds DEVIATES true wherever the pack moves a name, and one holds a vowel sign inside its letter's run.
2. The suggester:
   • Hyphenated runs no longer count.
   • A family-first or middle_as_family base is suggested nothing.
   • The letter readers keep the base's other switches. Only middle_as_family's reset changed anything, and it made the inert suggestion.
   • Root AGENTS.md states the joined-run widening and the extended harness proof.
   • Chance byline flips reproduced (Smith J A, Jones J A, Lee J A). Three stricter rules were measured stacked on Gate the suggester's trailing-initial vote on the group's share; fix a title-only name's crash #1345 and declined, the figures in decisions/group-suggestion: three names sharing a run; a run held by half of all names; deciding only where Gate the suggester's trailing-initial vote on the group's share; fix a title-only name's crash #1345's gate does not read bylines. Each costs Nepal far more than the ≤3% of byline draws it saves.
3. Untested guards:
   • len(family) != 1: tested by Maria R A y S.
   • O8 before O7: tested by Ram R S with both switches; the old example read alike either way.
   • The family-comma and order guards: removed. No input reaches them over 78,914 texts, because the family stands first in text order there.
   • The joined branch's comma test: removed. _name_words gives no words for a comma name.
   • The first-word and hyphen arms: rows added.
   • test_vocab: the initials_as_family = false refusal row added.
4. rules/O8 Accepted: van R S, Lakshmi R S V, Ram Bahadur B.K.T. (S3), and Devanagari letters, 598 spellings, left to What does an initial look like in Devanagari, Bengali and the scripts like them? (एम. एस. वर्मा gives initials() 'ए. ए. व.') #1273.
5. decisions:
   • One Nepal population, reconciled with the 552 moved.
   • The ORCID figures, each named by its cut.
   • The Lok Sabha recipe.
   • The report credited to Should Armitage JO read family 'Armitage'? A surname followed by initials, with no comma, reads the initials as the family name #1137's own change.
   • The 125,960 lesson dated, with an input.
   • LETTER RUNS' gate figures re-taken on e9dfab2.
6. Rosters and text:
   • docs/locales.rst and the ne.py docstring corrected.
   • mechanisms.md#DECLARED-SPAN amended to cover O8, and the vocab README says a new rule kind may amend an entry it shares.
   • O8's Signal: line is left as it is. It names a caller switch with the letters' position and shape. O6 and O7 read a word's shape as well, but name it in prose under position; O8 names shape as its own tag. Either way it stays one rule: the switch acts only through the shape, so a split would leave one rule with nothing to read.

Review round 2 (da8ef58)

1. The inert gate regressed a middle_as_family base (reproduced: the group fell back to a family-first order, families B K Ram). Round 1 had dropped the readers' middle_as_family reset, so no run moved under that base. Now:
   • The gate fires only under a family-first base, and there no name is parsed for runs.
   • The readers reset middle_as_family alone.
   • The note under a family-first base claims no count.
   • A sweep runs the Nepali group with each Policy field's non-default value as the base (tied to dataclasses.fields(Policy)) and with every pack as the base. The middle_as_family row fails on 29e86c7.
2. The dead name_order=GIVEN_FIRST reader setting is removed, and so are the wasted parses under a family-first base.
3. The Devanagari limit now says what T5 reads: a bare letter with a full stop is an initial (झलक बहादुर आ. म. moves), a letter without one or with a vowel sign is not. A row covers each, and the count is 597 kept, 1 moved.
4. With the switch on, Smith, John costs 137 frames against 136, since the removed family-comma test skipped the position walk. That is one frame on a family-comma name, only with the switch on; the switch off is unchanged.

suggest_measure.py --base <checkout of 62bbdeb1>, re-taken: the same single cell as round 1 (PatentsView's Dutch FF/GF, 38.7% → 38.3% at 2 names, 92.0% → 91.7% at 5), and the deciding groups Dutch 11, Taiwanese 7, Brazilian 5, Spanish, French and Indian 3, Russian 1.

On master (after #1345 merged)

Master (bc229fe) is merged in. The resolved hunks, each keeping both PRs:

• _suggest.py:
  • A group whose letters are its family sets its trailing-initial votes aside through Gate the suggester's trailing-initial vote on the group's share; fix a title-only name's crash #1345's gate path, with this PR's note, replacing this PR's own pop.
  • _decide_order's note parameter is Gate the suggester's trailing-initial vote on the group's share; fix a title-only name's crash #1345's trailing_note.
  • The NameEvidence docstring names both reasons a trailing-initial vote is left out.
• The rest: test_suggest.py's recorded controls, suggest_measure.py's SETTABLE, AGENTS.md and decisions/group-suggestion keep both PRs' rows and text.

Every other library file is byte-identical to da8ef58, save master's own changes.

suggest_measure.py --base <checkout of bc229fef>: one row moved, the ORCID bylines written family-first under a given-first base (84.0% → 83.3% at 10 names). Six of its groups repeat a run by chance and set their trailing-initial votes aside. No same-order row, parliament list or TURKIC row moved. The Dutch cell that moved against 62bbdeb no longer moves: the gate already sets those votes aside. The study populations (bylines, Nepal, South Indian, TSE, every whole list) read exactly as the stacked tree did. The LETTER RUNS entry states these figures.

Conflicts

With #1345: nameparser/_suggest.py (thresholds, suggest(), _decide_order's note parameter), tests/v2/test_suggest.py (the recorded controls), tools/differential/suggest_measure.py (SETTABLE), decisions/group-suggestion (Standing and Log). I resolved the stack once in a scratch tree: keep both threshold blocks, and when the letters are decided, set the trailing-initial votes aside, with this PR's note.

With PR 3 (the report), which is stacked on this branch: the release log only.

Checks

• uv run pytest: 45,834 passed (on master).
• uv run pytest -n0 -m serial: 62 passed, so the parse-cost band holds.
• uv run mypy, uv run ruff check, uv run sphinx-build -b doctest docs, uv run sphinx-build -W -b html docs and uv run python tools/vocab/build.py --check: clean.

Closes #1163
Refs #1254

🤖 Generated with Claude Code