Skip to content

Read the Maghrebi, Amazigh and Hebrew family-name particles; add locales.AR and locales.HE (#1257) - #1373

Merged
derek73 merged 14 commits into
masterfrom
feat/issue-1257-maghrebi-hebrew-particles
Nov 13, 2026
Merged

derek73 merged 14 commits into
masterfrom
feat/issue-1257-maghrebi-hebrew-particles

Conversation

@derek73

@derek73 derek73 commented Nov 13, 2026 •

Copy link
Copy Markdown
Owner

Implements the decision on #1257 (its decision comment, 2026-11-12): where a particle reading ships turns on its script, its bearers and the mid-name test, and its kind on what the phrase it opens is in the names its reading serves.

What changes

Word Reading File Default Example
ait, aït, nait clan, never given, cased new ber.toml on Youssef Ait Haddou → family Ait Haddou (was middle Ait)
ould patronymic, never given, cased ar.toml on Ahmed Ould Haddou → family and patronymic Ould Haddou
ouled clan, never given, cased ar.toml on Karim Ouled Haddou → family Ouled Haddou
el article + given (as al), cased ar.toml on, a decided exception to rule 2 (below) Sara El Haddou → family El Haddou; El Mehdi Haddou → given El, reported
ben (Latin) clan, cased + a given reading on (en.toml, he.toml) ar.toml, he.toml off, on under locales.AR / locales.HE default Karim Ben Haddad unchanged; AR → family Ben Haddad; Ben Haddad → given, reported
בן clan (was patronymic, never given) + given he.toml on בן כהן → given בן, reported (1.4.0's reading; 2.0–3.0.1 gave family בן כהן); יעל בן חיים → family בן חיים, patronymic now empty
אבו as abu: given, teknonymic, bound given he.toml on סאמי אבו חליל → family אבו חליל
אל as al: given, article he.toml on נאדר אל חלבי → family אל חלבי; אל כהן → given, reported
בת unchanged

clan is the kind the decision's rule 3 asks for. The README defines it as "a lineage prefix, where the word after it is a family name shared by the lineage rather than a parent's name", and no view reads it. Latin ben and Hebrew בן take it alike. One test decides the kind: a reading is patronymic where, among the bearers measured, the phrase it opens names the bearer's own father more often than it is a family name passed down unchanged, and clan otherwise. It binds words a change adds or reopens; bar, bin, binti, ibn, بن and בת are not re-judged. By it ben and בן are clan (ORCID's Tunisian family boxes 993, the Knesset's בן 15 of 15, against the 148 register chains #1248 counted). The test reads ould as clan too (family boxes DZ 29, MA 8, FR 9 against MR 20); it is patronymic by a choice decided on 2026-11-12 for the Mauritanian use, with the Algerian and Moroccan family names filling the view as its accepted cost. R9, the patronymic view, and decisions/vocabulary-collisions state it.

locales.AR and locales.HE are each a choice of their one language and nothing else. Each is registered, with rotators, a contents and pickling test, and CLDR's ar and he files in the locale view (20/24 and 18/24, unchanged by either pack). Three costs of the packs, each stated in the docstring and the docs:

  • Choosing ar drops en's credential ABD for ar's bound given name abd (Jane Smith, ABD → given ABD).
  • The packs are for surnames, not register-style chains (Karim ben Ali ben Salah Haddad → family ben Salah Haddad).
  • parser_for(locales.AR, strict=True) reads a leading Ben as the family, and drops ber.toml's ait/nait unless ber is chosen too; locales.HE keeps Ben given under strict=True.

Each Latin word carries a cased mask. Family boxes write them with a capital (Ben 1,294 to ben 1, El 2,849 to 13, Ait 203 to 1, Ould 36 to 0), and without a mask case repair lowered them as particles (Karim ben Salah).

el: on by default, a decided exception to rule 2

el fails rule 2 (Algerian compound given names) and is read by default anyway, on the measured trade decided after the gate. Rule 2 stands. Fixed/broken at this PR's head, the word removed as the comparator (config_diff.py score "expr:Parser(lexicon=Lexicon.default().remove('el', kind='article'))" default), against al's article measured the same way; each group is capped at 20,000 records unless marked:

group el al
ORCID MA 2001/26 26/0
ORCID EG 286/20 33/0
ORCID LB 432/6 196/6
ORCID SA 42/4 411/1
ORCID DZ 37/135 1/0
PSC Moroccan 389/13 5/0
PSC Algerian 3/10 0/0
PSC British, uncapped (680,166) 170/25 —

Algeria's breaks are the article inside a compound given name (Nour El Houda Haddou → family El Houda Haddou), and a bound given name before it is the same shape (Abd El Kader Haddou → given Abd, family El Kader Haddou, where 2.2.0 through 3.0.1 read given Abd El, middle Kader, and 1.4.0 through 2.1.0 given Abd, middle El Kader). Both are Accepted under rules/P2 (particles join forward) and in decisions/vocabulary-collisions. #1378 asks for the vocabulary that lifts them.

Canaries, run against master:

  • El Greco and El Mehdi Haddou keep given El, reported.
  • Haddou, Sara El → family El Haddou.
  • Sara El-Haddou, Juan de el Valle, Juan del Valle and Maria do Carmo Souza do not move.
  • Maria El Salvador → family El Salvador.
  • Lower-case Sara el Haddou → family el Haddou.

What moved

Measured at this PR's head against master's readings, rebuilt in the tree as an expr: lexicon. Its compiled sets equal those of master 161ebd28 and of the merge base 73718fd9 (36 sets compared). The shared tools cache was only read; match_diff.py wrote its kept base to a cache folder of its own.

  • Committed snapshot: eight rule examples added (git diff 73718fd9 HEAD -- tools/differential/corpus_rules.jsonl). Three names built on Zein El Dine moved with el (family Dine → El Dine; neither is the whole name, so unclear), and no other. The ben given reading and the off-by-default particle were each byte-identical in their own commits.
  • Local corpora (config_diff.py diff): JRC-Names 3,483 of 1,315,051 spellings and opensanctions-parliaments 16 of 24,192, nearly all the particle joining the family. Before el (commit 08629c8 and earlier) the figures were 287 and 15.
  • Scored (config_diff.py score; ORCID and PatentsView by country, PSC by nationality, each group capped at 20,000 records; ORCID's extract holds at most 20,000 a country), fixed/broken:
group default (head) locales.AR / locales.HE (head)
ORCID MA 2275/34 2403/34
ORCID DZ 132/142 184/145
ORCID EG 286/20 287/20
ORCID LB 432/6 434/6
ORCID TN 92/14 1056/17
ORCID MR (336) 38/6 38/6
ORCID IL 3/3 164/4
ORCID GB, US 12/2, 6/0 13/3, 7/0
PatentsView IL 0/2 146/61 (mostly filers who typed Ben in the given box)
PatentsView TN (154) 0/0 4/9
PSC Moroccan 452/16 477/16
PSC Algerian 5/11 7/11
PSC Tunisian 8/1 54/5
PSC Israeli 0/0 6/1

Agreement in every box, ORCID Tunisia: 87.6% by default, 96.1% under either pack. Uncapped, ben read as a particle breaks 148 PSC British, French and Tunisian filers against 120 it fixes (--cap 5000000), British middle names Ben, which is why it is off by default.

  • Name matching (match_diff.py --base <73718fd9>): 49 pairs move under each of the 21 settings, every one holding el. 22 become consistent: spellings of one person writing El and al, or the article with and without its hyphen. One becomes inconsistent only in the setting that removes particles-aside. No consistent pair shares no key. Recorded in decisions/name-matching.
  • Group suggester (suggest_measure.py --base): five cells moved. Four are in the capitals table (orcid-famcaps deciding capitals_as_family at 2 names 77.3% → 77.0% on either base; families read right without the switch 88.9% → 90.0%; orcid-givcaps' families 4,256 → 4,260). One is in the order table: orcid-givcaps' given-first anti-goal row at 3 names, 8.3% → 8.7%. All are listed in decisions/group-suggestion; the user-facing claims stand.
  • Other canaries unchanged: Ben Affleck, Van Johnson, Al Gore, Park In Hwan, Nguyen To Nga, John V. Smith.

Accepted

Recorded in rules/P1 (the lone particle fold), rules/P2 (particles join forward), rules/R9 (the patronymic view) and decisions/vocabulary-collisions:

  • A listing written family first without a comma reads whole as the family where a never-given ait or ould opens it (Ait Haddou Karim), the fold's existing cost.
  • A family name written with Hebrew בן first and no comma, under a family-first order, gives בן alone to the family (בן חיים יעל → family בן), where master read בן חיים. Should Hebrew בן opening a name be the given name Ben? (בן כספית gives family 'בן כספית' and no given name) #1209 counted 8 Knesset family names written so; a comma keeps them whole. Accepted (decided 2026-11-12) as a cost of the given reading.
  • ould fills the patronymic view in an Algerian fixed family name (Karim Ould Haddou).
  • An Arab name transcribed in Hebrew script with בן for the father leaves the patronymic view empty, as a Tunisian register's chain does with ben.
  • A family name written alone with בן reads given בן (Should Hebrew בן opening a name be the given name Ben? (בן כספית gives family 'בן כספית' and no given name) #1209).
  • אל inside a given name, and El inside an Algerian compound given name or after a bound given name, give the article and the rest to the family.
  • The packs' three costs above, and a strict Arabic choice reading ben as never given (with cheikh in the reviewed table of words strict ungates; strict Hebrew keeps it given through he.toml's own given reading).

Not changed: moved to #1378

Also left out:

Checks and reviews

  • Run here: uv run pytest, -n0 -m serial, mypy, ruff check, sphinx-build -b doctest and -b html, python -m doctest README.rst and build.py --check.
  • Python 3.15 in its own environment, without namedivider, plus the locale, CLDR, Wikidata-native and snapshot tests with the ja extra.
  • Reviewed in this session:
    • the measurement gate before any vocabulary edit;
    • sweeps per word over comma shapes, the three orders, case, capitals_as_family, initials(), capitalized() both ways, Parser.revise and Parser.matching().check both sides;
    • the 1.4.0, 2.0.0, 2.1.0, 2.2.0, 2.8.2 and 3.0.1 wheels for every release-log and case-row before/after claim, re-run after the second review found the Abd El Kader Haddou claim wrong for 1.4.0 through 2.1.0 (fixed).
  • The independent code, test and design-docs reviews were run on the PR, and this commit answers them.

#1210's parent issue is #1253 (decisions/family-without-an-inherited-name), which deferred it to #1257.

Closes #1209
Closes #1210
Closes #1211 (the Hebrew עבד with completed_by is in #1378)
Closes #1240
Closes #1248
Closes #1249 (N'Ait and the Arabic-script ايت/ولد are in #1378 or left out above)
Closes #1257

🤖 Generated with Claude Code

derek73 and others added 9 commits November 12, 2026 21:44
A given reading alone compiles into nothing: the default's sets and every
corpus reading are unchanged (snapshot byte-identical). It is the reading
a particle written default = false needs on its word (#1257).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#1248)

ar.toml and he.toml each carry ben as a clan particle written default =
false, with its case mask: the default's sets and every corpus reading are
unchanged (snapshot byte-identical), and a choice of ar or he reads
Karim Ben Salah with family 'Ben Salah'. ar.toml's patronymic exclusion
keeps refusing that kind, its reason now pointing at the clan reading.
he.toml records the Israeli given name Ben too, so a strict Hebrew choice
keeps Ben Affleck a given name; a strict Arabic one does not, added to
the reviewed table of words strict ungates (#1257).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…1248)

Each turns on ben's clan reading (Karim Ben Salah gives family 'Ben
Salah'); AR's choice also drops en's credential ABD for abd's bound
given name, which its docstring states with the example. Registered,
with rotators, a contents and pickling test, and the CLDR locale view's
ar and he files, where neither pack moves a rendering (20/24 and 18/24).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…1249)

A new ber.toml holds the Amazigh clan particles ait, aït and nait;
ar.toml holds ould, a patronymic (Mauritanian names still form the
father's name with it, so it fills the patronymic view) and ouled, a
clan particle. Each is never a given name and carries its case mask, so
Youssef Ait Taleb gives family 'Ait Taleb' and Ahmed Ould Salem family
'Ould Salem', patronymic 'Ould Salem'. config_diff's country table lists
ber under Morocco and Algeria; familiar.py's line order for ben (#1257).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…as Arabic particles (#1209, #1210, #1211)

בן gains a given reading and becomes a clan particle, as Latin ben is:
opening a name it is the given name and reports the fork (בן כספית),
inside one it starts the family name, which no longer fills the
patronymic view, every Knesset family name written with it being a
family name. אבו takes abu's readings and אל al's, so the Knesset's
Abu and al- family names read whole (סמי אבו שחאדה).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…costs (#1257)

decisions/vocabulary-collisions records the three-part rule with each
word's measurement and recompute; rules/P1 and R9 gain the costs as
examples; docs/languages.rst and docs/locales.rst describe locales.AR and
locales.HE, migrate.rst the readings moved from 1.4, and the release log
the four changes. Case rows, a capitalization, patronymic-view and
matching test for each word, and the rules corpus and snapshot gain the
three new examples; no existing corpus name moves.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
)

suggest_measure.py against master 161ebd2 moves five cells, none of the
order table's: orcid-famcaps decides capitals_as_family at 2 names in
77.0% of groups (77.3%), and reads 89.2% of its families right without
the switch (88.9%); the user-facing claims stand.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ar.toml writes el as al is read, an article that may be a given name,
with its case mask; the article reading is default = false beside a
given reading on by default, so the default leaves El to position and
locales.AR reads Sara El Haddou with family 'El Haddou'. Whether it is
read by default instead is held for review; that is this one line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@derek73 derek73 added this to the 3.1 milestone Nov 13, 2026
@derek73 derek73 added enhancement vocabulary Fixed by a word-list entry: per-word vocabulary and language packs non-latin-script Names written in a script other than Latin (rules/W) particles Particles and surname prefixes (rules/P) locale-packs Locale packs and language choices (nameparser.locales) patronymics Father's names and the patronymic view (rules/R9) labels Nov 13, 2026
@derek73 derek73 self-assigned this Nov 13, 2026
derek73 and others added 4 commits November 12, 2026 22:30
…hrebi-hebrew-particles

# Conflicts:
#	nameparser/vocab/en.toml
#	nameparser/vocab/he.toml
Decided for #1257 as an exception to its rule 2, on the measured trade:
the gains in Morocco, Egypt and Lebanon outweigh Algeria's cost, where
compound given names hold the article (Nour El Houda Haddou gives family
'El Houda Haddou'), recorded under rules/P2's Accepted and in
decisions/vocabulary-collisions beside the al comparison. El opening a
name stays the given name, reported. The committed snapshot moves three
names built on Zein El Dine; the suggester's order table is unmoved.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…osts (#1257)

The review's findings, records and docs only: no reading moves, the
snapshot gaining the four new rule examples alone.

- decisions/vocabulary-collisions: every figure re-taken at head with
  el, each recipe written in full with its comparator first; the
  expression rebuilding master checked against 161ebd2 and 73718fd;
  the kind criterion stated so it gives one answer for ould, ben and
  בן; rule 2's reach and its place beside C-i; the costs Accepted.
- rules/P1: בן under a family-first order; rules/P2: a bound given name
  before El; rules/R9: the kind criterion.
- The packs' docstrings and docs: register-style chains, strict=True,
  and choosing ber and en, with a doctest.
- decisions/group-suggestion names all five moved cells;
  decisions/name-matching records match_diff's moves; stale lines in the
  vocab README, he.toml, vocabulary-exclusions and the notes fixed.
- Private names from cache-only corpora replaced with made-up ones;
  follow-ups point at #1378.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Decided 2026-11-12: under a family-first order a family name written
with בן first and no comma gives בן alone to the family, reported, a
cost of its given reading. rules/P1 and decisions/vocabulary-collisions
say it is accepted; docs/languages.rst and the release log tell a
family-first caller to write the comma. No reading moves.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…imits (#1257)

The second review's findings, docs and records only; the snapshot and
the rules corpus are byte-identical to 7ec9945.

- The kind test is stated once, in rule 3, its Standing line, the entry
  head and rules/R9, scoped to words a change adds or reopens; by it
  ould reads clan (family boxes DZ 29, MA 8, FR 9 against MR 20), and it
  is a patronymic by a choice decided for the Mauritanian use.
- The release log's Abd El Kader Haddou claim corrected against the
  wheels (2.2.0 through 3.0.1 gave given 'Abd El'; 1.4.0 through 2.1.0
  given 'Abd'); the strict limit scoped to locales.AR; eight rule
  examples, not four; ait's 349 to 16 at head, the sixteenth read.
- rules/P1 gains the FAMILY_FIRST_GIVEN_LAST row; he.py and docs/locales.rst
  gain the register-chain limit; the gate, the בת and בר notes, Ouled's
  countries and #1378's comment named.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@derek73
derek73 merged commit 1db4fe3 into master Nov 13, 2026
14 checks passed
@derek73
derek73 deleted the feat/issue-1257-maghrebi-hebrew-particles branch November 13, 2026 08:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement locale-packs Locale packs and language choices (nameparser.locales) non-latin-script Names written in a script other than Latin (rules/W) particles Particles and surname prefixes (rules/P) patronymics Father's names and the patronymic view (rules/R9) vocabulary Fixed by a word-list entry: per-word vocabulary and language packs

Projects

None yet

1 participant