Skip to content

Divide 付, 代, 候, 芦 and 农 under locales.ZH (#952) - #975

Merged
derek73 merged 4 commits into
masterfrom
fix/issue-952-zh-mainland-surnames
Oct 31, 2026
Merged

derek73 merged 4 commits into
masterfrom
fix/issue-952-zh-mainland-surnames

Conversation

@derek73

@derek73 derek73 commented Oct 31, 2026

Copy link
Copy Markdown
Owner

Closes #952.

parser_for(locales.ZH).parse("付晓明") now gives family 付, given 晓明; through 2.8.0 it gave family 付晓明.

What changed

Five mainland surnames are added by hand at the tail of zh.py's census block, each annotated with its evidence as 萧 is: 付, 代, 候, 芦 and 农. With them come 蘆 and 農, the traditional forms the Taiwan ranking attests.

  • They rank 65th, 106th, 187th, 197th and 198th by people in China's national ID registry (NCIIC 2008). That source is read only as a cross-check, since it is GPL/NC.
  • They were the only unlisted surnames in that registry's top 200.
  • Neither of the list's rankings reaches them: Taiwan ranks them 435th, 545th, 351st, 642nd (蘆) and 317th (農).
  • The tool's generated block is unchanged. The hand rows sit before it, and tools/zh_surnames.py --write reproduces it byte for byte.

decisions/W1 (unspaced CJK division) gets the following:

  • a dated entry with the measurements and their recipes;
  • an amended Accepted bullet for the Japanese cost;
  • wainshine/Chinese-Names-Corpus under "Sources not used as data", because it ranks distinct names taken from text, not people;
  • an Open line for the third-source route.

Measured

All measured with parser_for(locales.ZH), comparing 4ebb51d with this branch, and asserting __file__ on each side.

population before after
wainshine mainland list (1,144,226 names), left whole 34,472 28,960
wainshine Japanese list (172,413 names), divided 40,668 40,805
local tw-donors (308,667 names), moved — 21
  • Mainland list: the 5,512 that moved are every name opening with one of the five: 付 3,488, 代 1,323, 芦 371, 农 213, 候 117.
  • Japanese cost per character: 付 0, 代 14, 候 0, 农 0, 農 0, 芦 102, 蘆 21. ZH already divided 23.6% of that list, and 芦 adds about 0.06 points.
  • tw-donors: 21 names moved, each from whole to divided: 農 7, 候 6, 蘆 5, 付 3. One of them is a committee named for the district 蘆洲, which now reads family 蘆.
  • Default parser: no reading moves. The snapshot only gains this PR's nine case rows.
  • Other records: the CLDR, Wikidata-native and per-pack agreement records all match unchanged. JA was checked too, with namedivider installed.

The list files hold one more name each than #952 counted (1,144,226 and 172,413 non-empty lines after the header). The counts above use those totals.

Tests

  • Case rows for each surname under zh, plus the traditional 農建華 and 蘆建國.
  • Japanese cost rows: 代田稔 → family 代, and 芦川誠 → family 芦.
  • corpus_cjk.jsonl and snapshot.jsonl regenerated for the new rows.
  • Checks run: uv run pytest, -n0 -m serial, mypy and ruff. The suite was also run with namedivider installed and without it.

Left open

The Open line for a mainland-weighted third source in tools/zh_surnames.py has no issue yet, and #952 closes with this PR. A draft for one is waiting for a decision on filing it.

🤖 Generated with Claude Code

derek73 and others added 3 commits October 30, 2026 22:49
Add four mainland surnames that neither ranking's top 300 reaches by
hand at the tail of zh.py's census block, annotated as 萧 is, with 農,
the traditional form the Taiwan ranking attests. They are the mainland's
65th, 106th, 187th and 198th by people (NCIIC 2008), the unlisted ones
in its top 200 save 芦, which opens 102 Japanese names and is left out.

Under parser_for(locales.ZH), wainshine/Chinese-Names-Corpus's mainland
list leaves 29,331 of 1,144,226 names whole, down from 34,472; its
Japanese list gains 14 wrong divisions, all 代 (代田稔). The default
parse does not move.

Closes #952

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
芦 is the mainland's 197th surname by people, the last unlisted one in
its top 200; 蘆 is its traditional form, Taiwan's 642nd. It opens 102
of the 172,413 names in wainshine's Japanese list (芦田, 芦川, 芦名),
and 蘆 21 more, now divided after it: the cost decisions/W1 records
beside 安's and 代's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eadline

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@derek73 derek73 added this to the 3.0 milestone Oct 31, 2026
@derek73 derek73 added enhancement vocabulary Fixed by a word-list entry: per-word vocabulary and language packs labels Oct 31, 2026
@derek73 derek73 self-assigned this Oct 31, 2026
Wikidata ranks 农 585th through its 農 item; only 芦 has no item. Date
the Taiwan and Wikidata figures in zh.py's comment, and call the rows
zh_surnames.py reads by hand "the hand-kept rows", since they now hold
seven entries that are not census rows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@derek73
derek73 merged commit 8d5b383 into master Oct 31, 2026
13 checks passed
@derek73
derek73 deleted the fix/issue-952-zh-mainland-surnames branch November 4, 2026 20:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement vocabulary Fixed by a word-list entry: per-word vocabulary and language packs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Should locales.ZH divide 付伟 and 代红梅? Two common mainland surnames fall below both rankings' top 300

1 participant