Repository navigation
Read script properties from one private table: Georgian has no case, Burmese and Thai write no initials (#1256) - #1377
Merged
Conversation
_policy._SCRIPT_PROPERTIES holds one row per script, keyed by a private name rather than by Script: whether the name-case contrast reads case in it, whether it writes initials, whether it writes period abbreviations, and whether a bare run of its letters with a period is a spelled Latin initial, each row with its reason and source. The readers that asked _NO_INITIALS, _INDIC_BRAHMIC and Python's case properties at the three contrast sites now read spans selected from it, built once at import. "Writes initials" (T5, R7, the name match) and "writes period abbreviations" (H2's shape, S3's joined Ph. D.) are separate columns, which the four East Asian rows answer alike. No reading moves: snapshot.jsonl byte-identical, the frame band unmoved. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
) Python counts every Mkhedruli letter as lower case, so a Georgian name beside a Latin capital was mixed case and the capitals leaned to the credential: 'ჯეკ MA' read suffix 'MA' and no family from 2.4.0 through 3.0.1. The script-property table's Georgian row (Mkhedruli and Mtavruli) has no case for the contrast, so the one-case fact, the credential lean and the case-repair gate drop its letters, and the name reads as 'محمد MA' does: family 'MA', with a suffix-or-name report. rules/R4's Mtavruli repair reads the same row, changing the same 46 letters over every codepoint. capitalized() now repairs a Mtavruli word or first letter beside Mkhedruli on the default path, the snapshot's one move ('Უჩა გოგოლაძე' -> 'უჩა გოგოლაძე'). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Burmese abbreviates no name to a letter and a period, and the first character of a syllable with a medial is not even its onset as written: 'ကြည် ကြည် ဌေး'.initials() gave 'က. က. ဌ.', ကြည် (Kyi) and က (Kyaw) alike. The script-property table's Myanmar row writes no initials, so initials() gives '' for a Burmese word, as for Han, Hangul and kana, the name match reads a Burmese letter as a whole name, and T5 tags none as an initial. No parse field moves; 39 snapshot entries move initials() only, 35 of them CLDR's fictional renderings. The row rests on the convention, decided on #1234, not on CLDR: my's personNames block is en's copied (36 of 42 patterns identical once the comma is written ၊, the rest differing by a space), so the recipe does not answer. Whether the script writes period abbreviations is left as it was. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…column (#1256) period_joined_vocab's shape verdict (rules/S3) refuses a script with no period abbreviations, as its docstring says, but asked the initials column. The two columns agree for every row so far, so nothing moves; they part for Thai, whose degrees are written as dotted consonant runs (พ.บ.). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ns (#1256) CLDR's th authors 20 name patterns and uses an initial in none, its own block rather than en's, so Thai joins the scripts that write no initials: 'สมชาย ใจดี'.initials() gives '' where 1.4.0 through 3.0.1 gave 'ส. ใ.', and the name match reads a Thai letter as a whole name. Thai still writes titles and degrees short with periods (ดร., ร.ต.อ., พ.บ.), so the row keeps period abbreviations and those read as before. 73 snapshot entries move, 72 in initials() only; 'สมชาย พ.ต.ส.' also drops its order report, which reads a word in a script that writes initials, as a byline's, its reading unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#1256) mechanisms.md gains SCRIPT-PROPERTY-TABLE; rules/H2's Accepted clause states its period-abbreviation veto as its own question rather than the initials veto's; nameparser/AGENTS.md records the sweep a split column asks for, with the S3 reader the Thai row found on the wrong column. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
decisions/R4 records the default-path move as decided, naming both shapes with their readings in 1.4.0 through 3.0.1 (kept as written) and how it differs from a Latin surname in capitals; rules/R5 gains it as an Accepted bullet; the release-log bullet names the capitalized() move. #1223's test pinning the Latin-habit capital as kept moves with it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From the comment review: the FULL_STOPS bundle's fifth site and the period-abbreviation veto name the column they read and all four of its readers, the ASCII shortcut's bound is U+007F (the conjoining jamo start at U+1100), the table header claims only its four questions, the caseless-letter table names R4's reader, and Cyrillic is said to have no row rather than one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From the code review: the lower-case run of initials (rules/T7 in _assemble, R3 in initials()) and S8's name word behind a credential or generation word asked Python's case, so a Georgian word counted as lower case there. They drop the caseless rows' letters as the other contrast sites do: 'j.r. ბერიძე' reads given 'j.r.' and 'ჯეკ ბერიძე Jr. ნინო' middle 'ბერიძე Jr.', each as its Arabic twin. A twin test holds fifteen shapes, ten of which master failed. The design-docs review's fixes: stale Standing lines on H2's veto and the scripts with no initials, the test's reach stated as it is, snapshot counts by distinct name. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The source scan now refuses any character range overlapping a row (escapes, hex pairs, chained comparisons, literal characters in a class), with a planted control of what it sees and lets pass and a named allowlist for the Thai marks other rules read. R4's repair joins the per-row case test; Thai rows guard is_leading_title and split_pair on the abbreviation column; the Georgian comma shape joins the twin test, whose recorded negative control (11 rows apart with the caseless table emptied) runs in the suite. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
codecov/patch found _ranges_pattern's refusal of an empty selection unrun; a test now selects a value no row holds. _capitalize's guard is one condition, its longer-lower-case arm having no codepoint to reach. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ds corrected (#1256) A1: rules/T7's and R3's lower-case run of initials go back to Unicode's case. They ask whether a dotted run of single letters is initials, which Georgian answers yes in its one case, so 'ნ.ბ. ბერიძე' reads given 'ნ.', middle 'ბ.' and 'j.r. ბერიძე' given 'j.', middle 'r.', as on master; the twin test, an oracle only where the scripts differ in the column under test, now holds sixteen shapes (eleven apart on master, a recorded control) and the dotted runs have tests of their own. S8's two sites keep reading the table: a gain, by S8's own caseless clause. A2/A3: Thai and Burmese letters being no initials moves O8 under initials_as_family and O6 under two_family_names, and drops the order report before a Burmese letter or a Latin one behind Thai: recorded in rules/O8 and O6 (Accepted, with examples) and the decisions entry. B1-B13: one copy test for CLDR, stated and run (my 42 of 42); the initialPattern step restored to the recipe; the Burmese row's source is the 2026-11-12 amendment to #1256's decision; #1214 and #1224 are not rows; what a script with no row reads as, and what was not examined; rows' reasons stated as conventions with dated sources; R5's Except narrowed to its boundary; the snapshot's one moving entry named; Standing bullets moved into their lists; release-log bullets split and reworded; migrate.rst's 3.1 changes; nameparser/AGENTS.md's lesson completed; stale _NO_INITIALS mentions in tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ows pinned (#1256) C2/C3: every locale pack and Policy switch swept over the Thai, Burmese and Georgian grids; the two-family-names packs, locales.NE, a lone letter's report and the order report before a Latin letter recorded in decisions/script-properties, and Parser.suggest's trailing-initial vote in decisions/group-suggestion (suggest_measure --base master: 0 cells moved). C4: S3's paired initials in a script with capitals recorded as a site left on Unicode by decision. C5: e8e200b restored two tags 72c6766 had dropped, recorded. The Burmese and Thai rows point at #1256's 2026-11-12 amendment, with Thai's pen-name cost. C6: each row's values pinned by a reading, its spans by the Unicode names of its script's letters (U+A8FE, Devanagari Extended, named outside the brahmic row), all 46 Mtavruli letters through R4. C7: the table and the caseless map read-only, has_case typed Literal[False] | None, an empty column failing the import by name, the guard-off control patching each reader's binding. C8: every Script member's spans in a row or a stated default (Cyrillic), with its control. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ipt-properties # Conflicts: # docs/design/decisions/group-suggestion.md # tests/v2/test_cases.py
…1256) Records and tests only; no reading moves. rules/T7 and S8 gain Accepted bullets, with examples, for the two unwritten shapes the independent review found: a Georgian dotted run left whole once S8 reads a Latin credential behind it as a name word (as beside a Hebrew word on master), and a listed Georgian salutation behind a credential read as the family (as its Hebrew and Latin Title-case twins read). Tests assert both script-table mappings are read-only, give R3's Georgian run its own row, and say why Myanmar Extended-C, digits only, pins no letter. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements the decision on #1256: one private table of script properties, read by every rule that asks, with the two leaves it settles.
Closes #1222
Closes #1234
Part of #1256
"Part of" rather than "Closes": three of #1256's sub-issues stay open after this PR.
एम. एस. वर्माgivesinitials()'ए. ए. व.') #1273: what an initial looks like in the Brahmic scripts. This is the brahmic row's own open question.O'tkirando'g'lilowercase after the apostrophe, as Uzbek writes them? (O'TKIR SOBIROVgivesO'Tkir) #1274: the Uzbek apostrophe in case repair, a language question.capitalized()write the DutchIJdigraph under a Dutch choice? (PIETER IJZERMANgivesPieter Ijzerman) #1242: the DutchIJ, a language question that the decision keeps out of the table.decisions/script-properties lists all three under
Open:. #1214 (the Cyrillic apostrophe) and #1224 (a particle opening a hyphenated word) are not rows: they are questions of a language's spelling, which no script column could hold, so they stay in decisions/R4.What changed
The table (
_policy._SCRIPT_PROPERTIES). Each row is keyed by a private name, never byScript, and has spans,has_case,writes_initials,period_abbreviations,spells_initials, a reason and a source. It replaces_policy._NO_INITIALS,_vocab._INDIC_BRAHMICand Python's own case properties at every name-level case contrast. Readers build matchers once at import from the spans a column selects, so no reader adds a frame. The first commit moves no reading:snapshot.jsonlis byte-identical, and the frame band and serial benchmarks pass.The split.
_NO_INITIALSanswered two questions: "writes initials" (T5, R7, the name match) and "writes period abbreviations" (H2's shape, S3's split and joined shapes). Thai answers them differently, so each is now its own column. One reader,period_joined_vocab(S3's dotted credential), was asking the wrong one. It moved in its own commit, with no reading moved.Georgian (Should Georgian script count as caseless? (
ჯეკ MAgives family '' and suffix 'MA', whereمحمد MAgives family 'MA') #1222). Georgian, Mkhedruli and Mtavruli, has no case for the name-case contrast. Six readers drop its letters withstr.translate:T7's and R3's lower-case run of initials read the row in one commit and went back to Unicode after the design-docs review. That run asks whether a dotted run of single letters is initials, which Georgian answers yes in its one case, so
ნ.ბ. ბერიძეreads givenნ., middleბ.as on master.Both of the following leaves rest on #1256's amendment of 2026-11-12. Each is its own commit.
Burmese (Should
initials()skip Burmese script, as it skips Han and Hangul? (ကြည် ကြည် ဌေးgives 'က. က. ဌ.') #1234). The Myanmar script writes no initials, decided from the writing convention. CLDR'smyblock isen's: all 42 patterns are equal once spaces and commas are removed, and it holds{surname-prefix}though Burmese names have no surname. (72c6766)Thai. Thai writes no initials, decided by the CLDR count:
thuses an initial in 0 of 20 locale-authored patterns. It does write period abbreviations, soดร.,ร.ต.อ.andพ.บ.read as before. The accepted cost is a pen name likeส. ศรีสุข: it keeps its fields, butinitials()gives''. (28f73e9)What moved (snapshot, master 161ebd2 → head)
initials()/capitalized()ჯეკ MAfamilyMA;ჯეკ, MAgivenMA, familyჯეკ, asمحمد, MAreads (both the 1.4.0–2.3.0 readings, checked against the wheels);ჯეკ ბერიძე Jr. ნინოfamilyნინოᲜინო Ბურჯანაძე→ნინო ბურჯანაძე(the one pre-existing snapshot entry that moves; the rule examples add more)suffix-or-nameadded where the count now decidesဌေး က. ခ.)orderdropped before a Burmese letter (ကြည် က.)initials_as_familyใจดี ส. ก.keeps middleส., and undertwo_family_namesสมชาย ส. ใจดีreads familyส. ใจดี(rules O8 and O6, Accepted)orderdropped before a Thai or Latin letter (สมชาย พ.ต.ส.,สมชาย V)Local corpora,
snapshot.py --local … --basemaster (2026-11-12):jrc-names: 565 spellings moved, 561 of them ininitials()only (548 Thai, 13 Burmese). The other four are Georgian and match their Arabic twins on master:ალექსი II მოსკოვის პატრიარქიandკარლ XVI შვედეთიnow capitalize the numeral asIiandXvi, andპეტრე I რუსეთიandფრანსუა I საფრანგეთიnow reportconjunction-or-initial(see Accepted)nepal-candidates: matches the base.opensanctions-parliaments: matches the base.A test from #1223 moved, deliberately.
test_georgian_in_mtavruli_repairs_to_mkhedrulipinnedᲜინო Ბურჯანაძე(a Mtavruli first letter on Mkhedruli) as kept on the defaultcapitalized()path. With R5's gate reading the row, Georgian is one case to the gate, so the name is written in Mkhedruli, asforce=Truealready wrote it. A Mtavruli surname beside a Mkhedruli given name (ნინო ᲑᲣᲠᲯᲐᲜᲐᲫᲔ) moves the same way. That differs from Latin, whereJean DUPONTis kept: Latin's capitals are case, while Mtavruli is a style of a caseless script. This is recorded as DECIDED in decisions/R4 and as an Accepted bullet in rules/R5. The test keeps a row for each shape with the new reading.Recorded as Accepted
Word-level tests stay on Unicode: a Mtavruli word is a word in capitals. O9 still reads
ᲑᲣᲠᲯᲐᲜᲐᲫᲔ ნინოundercapitals_as_familyas familyᲑᲣᲠᲯᲐᲜᲐᲫᲔ, so it loses no reading. Under that switchჯეკ ბერიძე MAkeeps familyMA, unchanged from master.A mid-name roman numeral in a Georgian name now repairs as it already did in an Arabic one (
ალექსი II …→Ii, asمحمد II علي حسنgives on master). That is a defect of caseless names in general, not of this row.The Myanmar row's
period_abbreviationsstays true, there being no evidence either way:ကက. ဌေးstill reads titleကက..A Georgian dotted run followed by a credential that S8 reads as a name word stays whole. The credential's Latin capitals then count among the name words, which turns off T7's all-lower-case reading. So
ნ.ბ. ბერიძე Jr. ნინოgives givenნ.ბ., middleბერიძე Jr., familyნინო, where master gave givenნ., middleბ., suffixJr. ნინო.ნ.ბ. MAkeeps its run whole. Master already reads a run whole beside a Hebrew or Arabic word (ნ.ბ. בן ბერიძე). Recorded in rules/T7's Accepted bullet.A listed Georgian salutation behind a credential is the family name:
ჯეკ ბერიძე MA ბატონიgives familyბატონი(master: titleბატონი, suffixMA);ქალბატონიandქ-ნიread the same way. Its Hebrew and Latin Title-case twins already read this way on master (משה כהן MA מר,Jack Smith MA Mr), and behindJr.it stays the title. Recorded in rules/S8's Accepted bullet.Docs
capitalized()move), Burmese and Thai.Swept against master: every comma shape, the three orders, every policy switch, every locale pack, NFC and NFD,
Parser.revise, both sides ofmatching().check, andParser.suggest. 11 mutations of the changed conditions are each caught by a test, and so are the code review's row-value mutations.Moves under the packs and the suggester, all recorded in decisions/script-properties:
locales.CS,SK,ES,ES_AR), a Thai or Burmese letter with a period between two names is now the first family word rather than a middle initial:Nino ก. Beridzegives familyก. Beridze, asJohn 王. Smithalready did on master. This goes through rules/O6.locales.NE(initials_as_family), such letters are no run, through rules/O8.က.orก.now reportsgiven-or-family, as any lone word does.orderreport (က V,สมชาย V).Parser.suggestno longer suggestsFAMILY_FIRSTfor a group of Thai or Burmese names, each a word followed by a letter. That is recorded in decisions/group-suggestion.suggest_measure.py --basemaster: 0 cells moved.Commit e8e200b moved tags. It is "no reading moved" against master, not against its parent: 72c6766 had dropped
shape:acronymandvocab:suffix-ambiguousfromက.က.inက.က. က, and e8e200b gives them back. No field moved.Mutations, on the final head (bb606fd)
Each changed condition was mutated alone, and the test-file set below was run against it (
test_script_properties,test_render,test_match,pipeline/,test_cases,test_rules_doc). All 22 are killed:test_script_properties::test_a_georgian_run_of_initials_reads_as_initialstest_script_properties::test_a_georgian_run_ending_the_name_gives_an_initial_per_lettertest_script_properties::test_georgian_reads_as_its_arabic_twintest_script_properties::test_georgian_reads_as_its_arabic_twintest_script_properties::test_has_case_reaches_every_contrast[georgian]test_script_properties::test_has_case_reaches_every_contrast[georgian]test_script_properties::test_has_case_reaches_every_contrast[georgian]test_script_properties::test_has_case_reaches_every_contrast[georgian]test_script_properties::test_the_title_shape_reads_its_two_columns[myanmar]pipeline/test_vocab::test_two_dotted_letters_merge_only_outside_a_cased_scripttest_script_properties::test_a_script_with_no_initials_may_still_write_abbreviations(กข.)_read_spansdoes not raisetest_script_properties::test_a_column_with_no_row_fails_the_import_by_namedictfor the table'sMappingProxyTypetest_script_properties::test_the_table_and_the_caseless_map_are_read_onlydictfor the caseless map'sMappingProxyTypetest_script_properties::test_the_table_and_the_caseless_map_are_read_onlytest_script_properties::test_every_mtavruli_letter_repairs_to_mkhedruliperiod_abbreviationsFalsetest_script_properties::test_each_row_is_pinned_by_a_reading(राम शर्मा क.ख.)period_abbreviationsFalsetest_script_properties::test_each_row_is_pinned_by_a_reading(ကက.)test_script_properties::test_every_letter_of_a_row_script_is_in_the_row[myanmar]test_script_properties::test_every_letter_of_a_row_script_is_in_the_row[myanmar]test_script_properties::test_every_letter_of_a_row_script_is_in_the_row[myanmar]test_script_properties::test_a_script_with_no_initials_may_still_write_abbreviations(ดร.)test_script_properties(the case-column tests)Myanmar Extended-C (U+116D0–116FF) is the one span no test pins: it holds only digits as of Unicode 17. It stays in the row so that a letter assigned there later reads like the script's other letters, and it moves no reading today.
Checks
uv run pytest,-n0 -m serial,mypy,ruff,sphinx-build -b doctest(874, 0 failures),python -m doctest README.rst,tools/vocab/build.py --checknamedivider-python, and with itReview aspects run
has_caseis typedLiteral[False] | NoneScriptenum is bound to the table by spanis_leading_titleandsplit_pair, the Georgian comma shape, and an in-suite negative control (11 twin rows apart with the caseless table emptied)🤖 Generated with Claude Code