Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -261,7 +261,7 @@ The compiled sets, read by `Lexicon` (mostly as the `_sets` field of the same na
- the views' kinds — `GENERATION_WORDS`, `CREDENTIAL_WORDS` (a credential written as a word), `HONORIFIC_WORDS` and `PATRONYMIC_PARTICLES` (3.0's 4b and 4c), private: the kinds the other sets do not record, which classify reads to tag a word for the `generation`, `credentials`, `honorifics` and `patronymic` views (rules/R8, rules/R9)
- case masks — `CAPITALIZATION_EXCEPTIONS` mapping of case MASKS (e.g. `{'phd': 'PhD'}`), compiled from the `cased` property, at most one per word: each value is its key's own letters and digits recased, laid over the word as written, so `ph.d.` repairs to `Ph.D.` and repair never writes the value's own punctuation into the word (#459); `Lexicon` raises `ValueError` on a value that spells anything else; and `CAPITALIZED_PARTICLES` (the `saint` and `rank` kinds, private), the particles case repair writes with a capital rather than in lowercase, keyed on the kind so a word needs no mask of its own (rules/R4, #1069), and `CAPITAL_OPENING_PARTICLES` (the particle readings written `capital_opening`, `ter` and `der`, private), the particles case repair writes with a capital where the word before it in the family name is no particle, nor, where it opens the family name, the word written right before it in any role (`Levon Ter Petrosyan`, but `Jan van der Berg` and `Van der Walt`), keyed on a property because `der`'s kind, `article`, is `de`'s and `la`'s too (rules/R12)
- listed spellings — `SAME_AS`, private, read by `Lexicon` as `_same_as` and nowhere in a parse: every `same_as` keyed by the group of its reading's kind (`_reading_index.SAME_AS_GROUPS`: `given` for a given name or a bound given name, `family` for a surname or a particle, `title`, `suffix`), each word with every full form it carries there (`sr` among titles is both `señor` and `senhor`). The name match (`_match.py`) reads each role in its group through `Lexicon._headwords`, every word reachable through the group's links, so a given name's spelling never matches a surname (`Ma, Jack` / `María, Jack` conflict) and a loop is simply all its words (decisions/name-matching, 2026-11-08); and `SAME_AS_NAMED`, per group, the words of it whose name reading's `same_as` stands beside a name reading of the same kind with none, a name of its own too (`jo`), whose links `_headwords` follows only where `_match._spellings` read the word with its trailing period (the 2026-11-08 #1006 follow-up)
- familiar forms — `FAMILIAR`, private, read by `Lexicon` as `_familiar` (through `Lexicon._familiar_forms`) and nowhere in a parse: each `familiar` reading's `short_for`, the longer given names a short one stands for (`max` -> `maksim`, `maxim`, `maximilian`), every file's together, which `Lexicon` reads both ways (a pair matches whichever way its source wrote it, never chained) and the name match reads as the opt-in `familiar-form`. `familiar` is a category of its own, never a `given` reading, which would move parses where the word is also a title, particle or suffix; a choice neither prefers nor drops by it (`_reading_index.UNPREFERRED`), and `tools/vocab/familiar.py` writes the shipped ones from nam_dict and from the nicknames list, less the pairs `tools/vocab/nicknames_excluded.tsv` refuses (decisions/name-matching, 2026-11-08 FAMILIAR FORMS and 2026-11-09 THE NICKNAMES LIST)
- familiar forms — `FAMILIAR`, private, read by `Lexicon` as `_familiar` (through `Lexicon._familiar_forms`) and nowhere in a parse: each `familiar` reading's `short_for`, the longer given names a short one stands for (`max` -> `maximilian`, `maxwell`), every file's together, which `Lexicon` reads both ways (a pair matches whichever way its source wrote it, never chained) and the name match reads as the opt-in `familiar-form`. `familiar` is a category of its own, never a `given` reading, which would move parses where the word is also a title, particle or suffix; a choice neither prefers nor drops by it (`_reading_index.UNPREFERRED`), and `tools/vocab/familiar.py` writes the shipped ones, English alone, from the nicknames list, less the pairs `tools/vocab/nicknames_excluded.tsv` refuses (decisions/name-matching, 2026-11-08 FAMILIAR FORMS, 2026-11-09 THE NICKNAMES LIST and 2026-11-12 ENGLISH ONLY UNTIL REBUILT)

- the group suggester's tables — `config/_order_names.py`, private, written by `tools/vocab/order_names.py` (`--write`; its hand-kept spelling lists sit above the generated blocks) and read by `_suggest` only, never by a parse or a `Lexicon`: `ROMANIZED_SURNAMES` (zh, ko and vi surnames folded by `_suggest.fold`) and `COMMON_GIVEN_NAMES`. They are not `Lexicon` kinds, so a language choice, `strict=True` and a caller's edit do not reach them and no snapshot moves with them (decisions/group-suggestion, 2026-11-10 TABLES)

Expand Down Expand Up @@ -372,7 +372,7 @@ Don't use the bare `python3 -m doctest <file>.rst` CLI (no `optionflags`) to che

A parse-behavior test can still be a disguised constant-content test if it only exercises one or two hardcoded entries from a large list (e.g. guarding a specific missing-comma typo in `SUFFIX_ACRONYMS`) — it documents that one instance without catching the same bug class elsewhere in the list. Prefer a test on the structural property itself, or skip a dedicated test for narrow, unrepeatable typo fixes.

**A guard that decides its own scope can stop guarding without failing.** Two shapes hit in #329, both in live code. `_POLICY_SHAPES` in `test_benchmark.py` had one entry, and pytest turns an **empty parametrize into a skip, not a failure**, so deleting it would have dissolved the quadratic guard into the skip count. And the 2.x `test_facade_cases.py::_constants_for` decided which case rows it could express: its gate had been structurally unsatisfiable since the maiden-wins canonicalization landed, silently skipping seven rows, while an unenumerated `Policy` field would instead let a row run MISTRANSLATED. Neither could fail, because a skip is indistinguishable from "correctly declined". Assert the table is non-empty, give each shape a reachability probe, and tie any hand-written field enumeration to `dataclasses.fields(...)` the way `test_policy.py`'s `test_policy_patch_mirrors_policy_field_names` does. **A component that filters its own input needs a test on the filter, not only on what survives it.** The same holds for a record split by scope: a check that reads each scope's share of a record never sees an entry that falls in no scope, so test that every entry has one — #672's per-pack CLDR ratchet missed record entries for a file dropped from `LOCALE_FILES`, and its re-record carried them over as if their pack were merely unmeasured.
**A guard that decides its own scope can stop guarding without failing.** Two shapes hit in #329, both in live code. `_POLICY_SHAPES` in `test_benchmark.py` had one entry, and pytest turns an **empty parametrize into a skip, not a failure**, so deleting it would have dissolved the quadratic guard into the skip count. And the 2.x `test_facade_cases.py::_constants_for` decided which case rows it could express: its gate had been structurally unsatisfiable since the maiden-wins canonicalization landed, silently skipping seven rows, while an unenumerated `Policy` field would instead let a row run MISTRANSLATED. Neither could fail, because a skip is indistinguishable from "correctly declined". Assert the table is non-empty, give each shape a reachability probe, and tie any hand-written field enumeration to `dataclasses.fields(...)` the way `test_policy.py`'s `test_policy_patch_mirrors_policy_field_names` does. **A component that filters its own input needs a test on the filter, not only on what survives it.** A filter whose only witnesses are shipped files loses them when the data changes: `_reading_index.tags()` leaves out a file of familiar readings alone, and #1370's follow-up deleted `mt.toml` and `sq.toml` and emptied `hu.toml`'s familiar readings, the last such files, leaving the filter untested until review planted one (`test_a_file_holding_only_familiar_readings_is_chosen_only_strictly`); when a change removes data, ask which guard used it as its example. The same holds for a record split by scope: a check that reads each scope's share of a record never sees an entry that falls in no scope, so test that every entry has one — #672's per-pack CLDR ratchet missed record entries for a file dropped from `LOCALE_FILES`, and its re-record carried them over as if their pack were merely unmeasured.

**Deleting a test module: sort its tests by the code they exercise, not by their names.** A module named for a retired tool also holds tests of code that survives it. #664 deleted `test_ledger_guards.py` and rewrote `test_differential.py`, triaging both by test name, and review found five tests guarding live code in the discard pile: four of `build_issues_corpus.py`'s harvester, and the check that `compare.py` reports the entries it skips. For each test, grep its module-level handles (`load_tool("...")`, imports) and ask whether that code is also being deleted. The converse holds for rewriting tests: a tool that reads test SOURCE by its syntax breaks when the tests change form though no assertion moved. #357 ported the 1.x suite from `HumanName(...)` to `parse(...)` and helpers wrapping it, and `tools/differential/build_corpus.py`, which collected the first argument of `HumanName` calls, returned 183 of the 566 names it reads now, with no warning, until review ran it; grep `tools/` for the call names a port rewrites.

Expand Down
14 changes: 4 additions & 10 deletions LICENSE-nam_dict
Original file line number Diff line number Diff line change
@@ -1,13 +1,7 @@
Two parts of nameparser are derived from Jörg Michael's nam_dict.txt and are
distributed under the license below:

- the familiar forms seeded from it: the `familiar` readings in
nameparser/vocab/*.toml whose note opens "nam_dict (Jörg Michael, 2008;
GNU FDL 1.2)", and their compiled copies in nameparser/config/_compiled.py
and nameparser/config/_index.py, written by tools/vocab/familiar.py;
- the common given names of the group suggester, the generated
_COMMON_GIVEN block in nameparser/config/_order_names.py, written by
tools/vocab/order_names.py.
One part of nameparser is derived from Jörg Michael's nam_dict.txt and is
distributed under the license below: the common given names of the group
suggester, the generated _COMMON_GIVEN block in
nameparser/config/_order_names.py, written by tools/vocab/order_names.py.

The file's copyright and license notices, as it states them:

Expand Down
6 changes: 3 additions & 3 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -302,9 +302,9 @@ under other licences, which apply to those parts:

- the familiar forms from Old Dominion University's nicknames list:
Apache License 2.0, in ``LICENSE-nicknames``;
- the familiar forms and the group suggester's common given names from
Jörg Michael's nam_dict: GNU Free Documentation License 1.2 or later,
in ``LICENSE-nam_dict``;
- the group suggester's common given names from Jörg Michael's
nam_dict: GNU Free Documentation License 1.2 or later, in
``LICENSE-nam_dict``;
- ``locales.ZH``'s simplified surname forms and the group suggester's
pinyin surnames from the Unicode Consortium's Unihan database: Unicode
License v3, in ``LICENSE-unicode``;
Expand Down
8 changes: 4 additions & 4 deletions docs/customize.rst
Original file line number Diff line number Diff line change
Expand Up @@ -169,14 +169,14 @@ and property means. A ``familiar`` reading is a short given name
standing for the longer ones its ``short_for`` lists (``max`` for
Maximilian): no parse reads it, and :ref:`name matching
<matching-names>` reads it as the opt-in ``familiar-form``.
``add("max", kind="familiar", short_for=["maxwell"])`` *extends* the
longer names the shipped vocabulary lists for ``max`` (Maksim, Maxim,
Maximilian), in a reading of your own that names no file, unlike an
``add("max", kind="familiar", short_for=["maxim"])`` *extends* the
longer names the shipped vocabulary lists for ``max`` (Maximilian,
Maxwell and others), in a reading of your own that names no file, unlike an
add of any other kind the word holds, which updates that reading. A
pair lives on the word whose reading lists the other, as
``readings_of("max")`` shows: to cut ``Max`` / ``Maximilian``,
``remove("max", kind="familiar")`` and add back the names to keep,
``add("max", kind="familiar", short_for=["maksim", "maxim"])``. A
``add("max", kind="familiar", short_for=["maxwell"])``. A
familiar reading needs at least one name, and none may be the word
itself.
``same_as="..."`` is accepted on a reading of every kind but
Expand Down
2 changes: 1 addition & 1 deletion docs/design/decisions/group-suggestion.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

- Proposed, for review: the parser never detects language; the group suggester (`Parser.suggest`) may, over a group the caller declares, says what it saw, and its answer is configuration the caller applies; the parse itself is untouched by it — 2026-11-10 SCOPE.
- The two word tables the suggester reads (romanized zh, ko and vi surnames; common given names) are private compiled tables for the first version, not `Lexicon` kinds: no parse reads them, a language choice or `strict=True` does not filter them and a caller cannot edit them — 2026-11-10 TABLES.
- The derived given-name table from nam_dict is approved (the table is derived from nam_dict, GNU FDL 1.2, already the source of the shipped familiar forms) — 2026-11-10 TABLES.
- The derived given-name table from nam_dict is approved (the table is derived from nam_dict, GNU FDL 1.2, which seeded the shipped familiar forms until decisions/name-matching's 2026-11-12 ENGLISH ONLY UNTIL REBUILT) — 2026-11-10 TABLES.
- Proposed, for review: the given-name table's size and filter (the top 500 per country, a family share of at most 5% over at least 20 uses) and the hand-kept surname lists — 2026-11-10 TABLES.
- Proposed, for review: the evidence declined (the vocabulary's name readings, `Reading.languages`, the familiar forms) — 2026-11-10 DECLINED.
- Proposed, for review: a comma name states its own order and does not vote on order — 2026-11-10 SIGNALS.
Expand Down
Loading
Loading