Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -152,7 +152,7 @@ More shapes the parser reads:
>>> ivan.given, ivan.patronymic, ivan.family
('Иван', 'Петрович', 'Сидоров')
>>> locales.available()
('bn', 'cs', 'es', 'es_ar', 'ga', 'gd', 'hi', 'hr', 'ja', 'mn', 'ne', 'pt', 'pt_br', 'ru', 'sk', 'sl', 'tr_az', 'vi', 'zh')
('ar', 'bn', 'cs', 'es', 'es_ar', 'ga', 'gd', 'he', 'hi', 'hr', 'ja', 'mn', 'ne', 'pt', 'pt_br', 'ru', 'sk', 'sl', 'tr_az', 'vi', 'zh')

When a reading is wrong for your data
-------------------------------------
Expand Down
4 changes: 2 additions & 2 deletions docs/design/decisions/family-without-an-inherited-name.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
- The patronymic view keeps rules/R9's basis, filled only by a word or a rule that says patronymic, never by a position; it may overlap the family or be a part of it, and a father's name nothing in the text marks (Ethiopia) fills no view — 2026-11-12 #1253.
- An Ethiopian or Eritrean name stays positional, its family under the default given-first order the grandfather's name where it is written and the father's where it is not; a caller matching such a list builds the matcher from a parser with `middle_as_family`, keeping the default parser for the fields (docs/matching.rst) — 2026-11-12 #1253.

Open: #1188 (Kazakh), #1155 (`بن` chain), #1231 (Burma), each a value question under the two statements; #1210 (Hebrew `בן`), decided with #1257; #1165 (`S/O`, `W/O`), a decision of its own.
Open: #1188 (Kazakh), #1155 (`بن` chain), #1231 (Burma), each a value question under the two statements; #1165 (`S/O`, `W/O`), a decision of its own.

## Log

Expand All @@ -19,7 +19,7 @@ Open: #1188 (Kazakh), #1155 (`بن` chain), #1231 (Burma), each a value question
- #1187 Kyrgyz: with no surname, the father's name and its marker are the `family` and the patronymic view alike (the reading lands with #1187, in rules/O2).
- #1188 Kazakh: an ending rule fills the view, as the East Slavic one does; which choice turns it on is a value question.
- #1155 a family name after a `بن` chain: the view may be a part of the family, stopping before a family name a closed signal marks; which signal is a value question.
- #1210 Hebrew `בן`: decided with #1257.
- #1210 Hebrew `בן`: decided with #1257, a clan particle filling no view (decisions/vocabulary-collisions' 2026-11-12 #1257 entry).
- #1231 Burma: a pack, the last word the `family` by position, the honorifics titles.
- #1165 `S/O`, `W/O`: out of this decision; the words after the marker are another person's name, which the model has no place for. Malaysian `a/l` and `a/p`, where the father's name is part of the legal name, read as `bin` does and are a vocabulary question.
- **Measured for #1203** (dated snapshots):
Expand Down
1 change: 1 addition & 0 deletions docs/design/decisions/group-suggestion.md
Original file line number Diff line number Diff line change
Expand Up @@ -154,6 +154,7 @@ Open: bylines written with the initials first (`GH Yoon`), which the capitals si
| az_parliament (124), kz_mazhilis (96, Cyrillic), tr_parliament (592), the whole list | decided: 124 for / 0 against, 92 / 0, 565 / 0 | — |

The East Asian row reaches 95% at 10 names, the whole famcaps population at 3 on the threshold (97.0% given-first, 95.0% family-first) and clear of it at 4: the short families are no evidence, and the floor is kept for them. The user-facing claims say "from 4 names" and "from 10".
Re-taken 2026-11-12 after #1257 made `ait`, `nait`, `ould` and `ouled` never-given particles and `el` an article that may be given (`suggest_measure.py --base <a checkout of 161ebd28>`, 300 groups a size, seed 20261110): five cells moved, four of the capitals table and one of the order table's, its orcid-givcaps anti-goal row, a list writing its given names in capitals that the capitals signal reads the wrong way round by design. (1) orcid-famcaps, family-first written, given-first base, deciding `capitals_as_family` at 2 / 3 / 4 / 5 names: 77.3 / 95.0 / 99.0 / 99.7 → 77.0 / 95.0 / 99.0 / 99.7; (2) the same written order on a family-first base, the same figures; (3) orcid-famcaps' families read as the box where decided: 352,086 names scored, 351,382 right with the switch and 312,999 without (99.8%, 88.9%) → 352,082, 351,378 and 316,781 (99.8%, 90.0%), a family written in capitals after the given name that opens with one of the particles (`Youssef AIT HADDOU`, `Sara EL HADDOU`) now reading right without it; (4) that anti-goal row, given-first written on a given-first base, groups suggested at 2 / 3 / 4 names: 26.7 / 8.3 / 3.0 → 26.7 / 8.7 / 3.0; (5) orcid-givcaps' families read as the box: 351,802 names, 1,704 right with the switch and 4,256 without → 1,704 and 4,260. The user-facing claims stand.
- Variants declined, a dated snapshot of 2026-11-12 on master bda4a632, taken with `capvote.py` (the 3.1 cycle's session notes, `659-pr-b/`, not the repository: the vote as a function outside the library, before the title and lower-case clauses of #1369's review, run over suggest_measure's populations and ORCID's, 300 groups a size, seed 20261110):

| population (capvote.py) | as shipped (L4, n2, > 2/3) | L4, n3 | L2, n2 |
Expand Down
1 change: 1 addition & 0 deletions docs/design/decisions/name-matching.md
Original file line number Diff line number Diff line change
Expand Up @@ -158,3 +158,4 @@ Open: a family name of initials alone against the same initials in another order
- **2026-11-09 #1286 A BIRTH NAME OF PARTICLES ALONE IS KEYED AS THE CHECK COMPARES IT.** Under `across-roles`, `Jack de` / `Jack Smith (née de)` and `Jack B. al` / `Jack S. (née al)` read consistent, the family matched against the birth name whole, but shared no key: the keys added the birth name's base, its text less its particles, which is empty there (`de;j` against `smith;j`). Of #1286's two ways, the keys taking the text whole, particles kept, as the check compares it, or the check refusing a cross-role match of particles alone, #1261's keys already do the first (its keys entry above: under `across-roles`, each text the check compares the family with, whole), so the pair now shares `de;j`. The second way, refusing a cross-role match of particles alone, is declined below.
- Measured, 2026-11-09: on master at 0e4bd46c, before #1261, each of `test_a_birth_name_of_particles_alone_is_keyed_as_the_check_compares_it`'s six rows reads consistent and shares no key, and from b23f5fcc on each shares one. A grid under `across-roles` of 16 particles and particle runs (`de`, `al`, `van`, `von der`, `de la`, `da`, `el`, `bin`, `ibn`, `di`, `du`, `le`, `la`, `ter`, `van den`, `al-`), each written `P` as a family in four shapes (`Jack P`, `P, Jack`, `Jack B. P`, `Jack P Jr.`) against a birth name in four (`Jack Smith (née P)`, `Jack Smith née P`, `Smith, Jack (née P)`, `Jack S. (née P)`), both ways, 512 pairs, all consistent: 480 share no key at 0e4bd46c and none at e1ae8086.
- Declined (maintainer, 2026-11-09): refusing a cross-role match whose text is particles alone, #1286's second way, which it called the more conservative. The particle list cannot tell a clipped `de` from a surname that is also a listed particle, and in real names a family name of particles alone is mostly such a surname: in up to 300,000 distinct names per local corpus (each corpus's names deduplicated, then shuffled by `random.Random(1286)` and cut; jrc-names' Latin-script spellings only, tse-donors' filed and registry names both), each parsed by `parse()`, a text being particles alone where each of its folded words, hyphens stripped, is in `Lexicon._sets.particles`, the commonest are `das` (570 in nepal-candidates), `le` (190 in orcid-researchers, 61 to 75 in ctgov-officials, patentsview-inventors and psc-individuals), `du` (95 to 103 in ctgov-officials and patentsview-inventors), `do`, `graf`, `van`, `bar`, `di` and `prinz`, only tse-donors' `de`, `da` and `dos` reading as clipped names. Refusing would read `Mary Das` / `Mary Smith (née Das)` a conflict. And it would protect against no pair the data holds: birth names are rare there (385 in 2,437,885 names, each source's deduplicated: thirteen local corpora sampled as above, and every distinct name in `tools/differential/snapshot.jsonl`, 119 of its 6,982), one of them particles alone, `Smt. Bandana Dome nee Das`, the Bengali surname. The mutant is master at e1ae8086 with one test added to `Matching._across`: a match is skipped where either text is particles alone, as above, and the role is the family. `match_diff.py --base <master at e1ae8086> --tree <the mutant>`, run from master at fd5bed17 so its draw holds #1290's rows, moved 76 of 654,066 rows (19 pairs under each of the four settings with `across-roles`), every one a constructed `Jack` pair of the test above, family `across-roles` → `conflict`, and no corpus pair; the same mutant with the role test dropped, refusing a particles-alone text in every cross role (a given name against a middle name or a nickname too), moved the same 76.
- **2026-11-12 #1257 THE NEW PARTICLES MOVE MATCHING THROUGH THE PARSE ALONE.** #1257 reads `el`, `ait`, `nait`, `ould` and `ouled` as particles by default and `אבו` and `אל` in Hebrew script, and changes no matching code; the pairs move because the family now holds the particle. `NAMEPARSER_TOOLS_CACHE=<a folder of its own, the corpora linked in> uv run python tools/differential/match_diff.py --base <a checkout of the merge base 73718fd9>` (2026-11-12) moved 1,029 of 655,452 rows, 49 pairs under each of the 21 settings, every one holding `el` or `El`: 22 pairs become consistent, two spellings of one person writing the article as `El` and `al` or with and without its hyphen (`Abd el-Fatah` / `Abd el Fatah`), and one becomes inconsistent only in the setting that takes `particles-aside` out of the accept set (`Ahmed Nashar` / `Ahmed El Nashar`, read alike by default through that outcome); the rest move a role's outcome or the keys alone (`Sara El Haddou` keyed with its particle). No consistent pair shares no key.
2 changes: 1 addition & 1 deletion docs/design/decisions/v1-xfail-triage.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ The reconciled v1-style banks (`tests/test_*.py`) carried eight `@pytest.mark.xf

- A trailing `I` before a further comma stays a middle initial, the commoner reading: `Maier, Amy I, Jr.` reads middle 'I', suffix 'Jr.', where `Maier, Amy I` reads suffix 'I' (rules/C9) — 2026-09-01 (Derek's triage)
- `Dr King Jr` reads title 'Dr', family 'King', suffix 'Jr', and `King, Dr Jr` reads the same fields, with no report since `king` became a given name outside a title phrase (`Dr Pope Jr` keeps H3's floor and its report) — 2026-09-01 (Derek's triage); 2026-10-26 #752, phase 2j
- `ben` stays out of the particles, colliding with the given name Ben where the particle claim acts: `Ahmad ben Husain` reads middle 'ben', family 'Husain' — 2026-09-01 (Derek's triage)
- `ben` stays out of the default's particles, colliding with the given name Ben where the particle claim acts: `Ahmad ben Husain` reads middle 'ben', family 'Husain' (an Arabic or Hebrew choice reads it as a clan particle written `default = false`, `locales.AR` giving family 'ben Husain', decisions/vocabulary-collisions' 2026-11-12 #1257 entry) — 2026-09-01 (Derek's triage)
- An input that is all titles takes its last title word as the name, a convention accepted rather than defended: `The Right Hon. the President of the Queen's Bench Division` reads family 'Division' — 2026-09-01 (Derek's triage)
- The one surviving marker is `E.T. Smith`'s, cited by issue (#490 until 2026-11-09, #1296 since): dotted initials against dotted vocabulary; `U.S. District Judge Marc Thomas Treadwell` was fixed on 2026-11-09 (decisions/H10); #489's `Her Majesty Queen Elizabeth` (given 'Elizabeth') was fixed on 2026-09-08 — 2026-09-01
- `juan garcia III` repairs to `Juan Garcia III` and its test is an ordinary pin — 2026-09-23 #492.
Expand Down
Loading
Loading