Fix manifest generator dropping handbook hierarchy by sonali195 · Pull Request #442 · WordPress/hosting-handbook

sonali195 · 2026-09-28T16:47:13Z

What

Fixes wp hosting-handbook gen-all so regenerating the manifest no longer drops the handbook hierarchy.

Why

The generator rebuilt bin/handbook-manifest.json from scratch using only top-level .md files, so running it (as CONTRIBUTING.md instructs) removed the version/ pages and every parent/order value. On Windows checkouts it also added a trailing \r to every title.

Changes

Include pages in sub-directories (e.g. version/)
Keep existing slug, parent and order values; titles are still refreshed from each page's # Heading, as before
Keep existing page order; append new pages with the next free order and remove deleted ones
Strip CRLF/BOM when reading titles
Error on invalid manifest JSON, JSON encoding failures (e.g. invalid UTF-8 in a heading), unreadable/unwritable files, unreadable directories, and duplicate keys. Previously, an encoding failure overwrote the manifest with an empty file.
Warn when:
- a page has no # Heading (empty title)
- pages are added, listing their slug, parent and order for review
- pages are removed because their files no longer exist
- a page's parent is not in the manifest (e.g. a sub-directory without an index.md), which would break the import
Remove unused Mustache_Engine and WP_CLI\Utils imports
Use canonical URLs in get-involved.md (the old /handbook/handbook/ links worked only via redirect)
Document generator behaviour in CONTRIBUTING.md

Testing

Running the generator on this branch keeps all pages, slug, parent and order values and markdown_source links unchanged; titles match the original generator's output (without the \r)
Note: 6 manifest titles differ from their page headings and will update on the next regenerate, as they would with the original generator (e.g. "Upgrading" → "Upgrading WordPress"). The live handbook already shows the heading titles.
Tested with LF and CRLF checkouts, new/removed pages, a new version/ page, invalid JSON and duplicate keys
Tested JSON encoding error handling with non-UTF-8 characters
php -l passes

AI Assistance Disclosure

AI assistance was used in the preparation of this pull request. All changes, code, and documentation were manually reviewed, verified, and tested.

- Include pages in sub-directories (e.g. version/) when generating the manifest - Preserve existing title, slug, parent and order values instead of overwriting them - Keep existing page order; append new pages and remove deleted ones - Strip CRLF/BOM when reading page titles so Windows checkouts don't add "\r" - Add error handling for invalid JSON and unreadable/unwritable files - Remove unused Mustache_Engine and Utils imports - Fix doubled /handbook/handbook/ links in get-involved.md - Document generator behaviour in CONTRIBUTING.md

added change

…headings ## What Fixes `wp hosting-handbook gen-all` so regenerating the manifest no longer drops the handbook hierarchy. ## Why The generator rebuilt `bin/handbook-manifest.json` from scratch using only top-level `.md` files, so running it (as CONTRIBUTING.md instructs) removed the `version/` pages and every `parent`/`order` value. On Windows checkouts it also added a trailing `\r` to every title. ## Changes - Include pages in sub-directories (e.g. `version/`) - Keep existing `slug`, `parent` and `order` values; titles are still refreshed from each page's `# Heading`, as before - Keep existing page order; append new pages and remove deleted ones - Strip CRLF/BOM when reading titles - Error on invalid manifest JSON, unreadable/unwritable files and duplicate keys - Remove unused `Mustache_Engine` and `WP_CLI\Utils` imports - Use canonical URLs in get-involved.md (the old `/handbook/handbook/` links worked only via redirect) - Document generator behaviour in CONTRIBUTING.md ## Testing - Running the generator on this branch reproduces the committed manifest; titles, slugs and `markdown_source` match the original generator's output - Tested with LF and CRLF checkouts, new/removed pages, a new `version/` page, invalid JSON and duplicate keys - `php -l` passes --- AI Assisted

vairafiq · 2026-10-02T05:09:45Z

Tested this branch locally (PHP 8.3, generator called through a small harness that stubs WP_CLI).

Test	Result
`main`, clean checkout	Bug confirmed: 19 entries become 16, every `parent` and `order` is lost
This branch, clean checkout	19 entries kept, all `parent` and `order` values kept, only the 6 titles change (as described) ✅
Second run	Identical output ✅
New top-level page	Appended at the end, `parent: null` ✅
New `version/7-2-compatibility.md`	`parent: version` ✅, slug is `7-2-compatibility`
New sub-directory with `index.md` + child page	Parent and child linked correctly ✅
Deleted page	Entry removed ✅
CRLF file, BOM file	Clean titles ✅
Duplicate key, invalid JSON	Clear error, manifest left untouched ✅

Suggestions, none of them blocking:

New pages get no order key. Every existing entry has one. Is a missing order safe for the importer? A default would be safer.
New version pages get the slug 7-2-compatibility, while the existing ones use 7-0 and 7-1. This will come up with WP 7.2 Compatibility docs #432. A log line that lists the newly added pages would remind people to set slug and order by hand.
Empty titles are silent. A page whose first line is not a # Heading gets "title": "" with no warning. Same as before this PR, but a WP_CLI::warning() would help.
Duplicate key error mixes formats: Duplicate manifest key 'security' for https://github.com/wordpress/hosting-handbook/blob/main/security.md and multisite/security.md. The first one is a URL, the second a relative path.
Final newline. The generator writes the file without a trailing newline, but .editorconfig sets insert_final_newline = true, so editors add one (this happened in Add AI Policy page #446) and the generator removes it again. Appending "\n" avoids that flip-flop.

AI Assisted: Claude was used to run the tests and draft this review.

…final newline Default order for new pages, warn on empty titles and new pages, add final newline

Handle JSON encode failure; warn on missing parents and removed pages

mindctrl

Looks good to me.

It would be good to update the PR description to clarify the changes related to JSON error handling, and when it triggers warnings.

- Error on invalid manifest JSON, JSON encoding failures (e.g. invalid UTF-8 in a heading), unreadable/unwritable files and duplicate keys. Previously, an encoding failure overwrote the manifest with an empty file.
- Warn when:
  - a page has no `# Heading` (empty title)
  - pages are added, listing their slug, parent and order for review
  - pages are removed because their files no longer exist
  - a page's `parent` is not in the manifest (e.g. a sub-directory without an `index.md`), which would break the import

Also worth noting it will no longer overwrite the manifest file with invalid JSON due to the above changes.

An unreadable bin/handbook-manifest.json was reported as "not valid JSON", which pointed at the wrong cause. Check file_get_contents() for failure and report "Unable to read" instead, and include json_last_error_msg() in the invalid JSON error.

mindctrl

This looks like a good improvement. I tried to break it, and the only thing I found with the current iteration is a small edge case where a folder exists and files are there but the files cannot be read for some reason, so they are removed from the manifest because they "no longer exist". I don't think that's worth addressing here.

Halt with error on unreadable handbook directories

sonali195 · 2026-10-09T06:01:49Z

What

Fixes wp hosting-handbook gen-all so regenerating the manifest no longer drops the handbook hierarchy.

Why

The generator rebuilt bin/handbook-manifest.json from scratch using only top-level .md files, so running it (as CONTRIBUTING.md instructs) removed the version/ pages and every parent/order value. On Windows checkouts it also added a trailing \r to every title.

Changes

Include pages in sub-directories (e.g. version/)
Keep existing slug, parent and order values; titles are still refreshed from each page's # Heading, as before
Keep existing page order; append new pages with the next free order and remove deleted ones
Strip CRLF/BOM when reading titles
Error on invalid manifest JSON, JSON encoding failures (e.g. invalid UTF-8 in a heading), unreadable/unwritable files, unreadable directories, and duplicate keys. It will no longer overwrite the manifest file with invalid JSON or an empty file due to encoding failures.
Warn when:
- a page has no # Heading (empty title)
- pages are added, listing their slug, parent and order for review
- pages are removed because their files no longer exist
- a page's parent is not in the manifest (e.g. a sub-directory without an index.md), which would break the import
Remove unused Mustache_Engine and WP_CLI\Utils imports
Use canonical URLs in get-involved.md (the old /handbook/handbook/ links worked only via redirect)
Document generator behaviour in CONTRIBUTING.md

Testing

Running the generator on this branch keeps all pages, slug, parent and order values and markdown_source links unchanged; titles match the original generator's output (without the \r)
Note: 6 manifest titles differ from their page headings and will update on the next regenerate, as they would with the original generator (e.g. "Upgrading" → "Upgrading WordPress"). The live handbook already shows the heading titles.
Tested with LF and CRLF checkouts, new/removed pages, a new version/ page, invalid JSON and duplicate keys
Tested JSON encoding error handling with non-UTF-8 characters
php -l passes

ekamran

Thanks @sonali195, this is a big improvement. I ran the generator on main and on this branch against the current manifest, plus new, renamed, deleted and moved pages, new folders, CRLF and BOM, bad UTF-8, bad JSON, duplicate keys and unreadable files. Everything matches the description.

One ordering case should be fixed before merge, inline below. Happy to approve once that is in.

List directory index.md first to ensure parent pages precede children

ekamran

Thanks @sonali195. Re-tested the latest push. The index-first change fixes the importer ordering case, including the hand-set parent case, and the current repo output remains byte-identical. The CONTRIBUTING wording now matches the generated slug behavior. I also reran the edge-case battery and cross-PHP checks.

One non-blocking follow-up: a README.md inside a sub-directory would still be picked up as a handbook page, while the root README is ignored. There is no such file today, so I would not block this PR on it.

Approving.

sonali195 · 2026-10-12T07:25:08Z

Thanks for the thorough review and testing, @Reviewer! That's a great catch on sub-directory README.md files. Since there are none in the repository today and this PR is approved, I'll let this land and open a quick follow-up PR to handle IGNORED_FILES in sub-directories as well

mindctrl · 2026-10-12T14:26:39Z

@sonali195 the AI assistance disclosure was removed from the PR description. Could you add that back before we merge?

sonali195 · 2026-10-12T17:00:54Z

"Added the AI assistance disclosure back to the PR description. Thanks!"

github-project-automation Bot added this to WordPress Hosting Team Sep 28, 2026

github-project-automation Bot moved this to Todo in WordPress Hosting Team Sep 28, 2026

sonali195 added 2 commits September 29, 2026 11:04

update contributiong file

f94c225

added change

mrx-arafat mentioned this pull request Oct 2, 2026

Tracking: Hosting Handbook audit (October 2026) - live site, links, versions and guidance #487

Open

21 tasks

This was referenced Oct 2, 2026

Add a WP-Cron and system cron page #491

Open

Add an HTTP security headers page #492

Open

Add a staging environments page #493

Open

abdur-emon mentioned this pull request Oct 2, 2026

Add a review checklist to CONTRIBUTING.md #500

Merged

Default order for new pages, warn on empty titles and new pages, add …

802b948

…final newline Default order for new pages, warn on empty titles and new pages, add final newline

This comment was marked as outdated.

Sign in to view

Handle JSON encode failure; warn on missing parents and removed pages

345abad

Handle JSON encode failure; warn on missing parents and removed pages

mindctrl approved these changes Oct 6, 2026

View reviewed changes

mindctrl approved these changes Oct 7, 2026

View reviewed changes

mindctrl requested a review from kittenkamala October 7, 2026 14:11

Halt with error on unreadable handbook directories

7d67028

Halt with error on unreadable handbook directories

ekamran reviewed Oct 9, 2026

View reviewed changes

Comment thread bin/command.php

Comment thread CONTRIBUTING.md Outdated

List directory index.md first to ensure parent pages precede children

15dd5f2

List directory index.md first to ensure parent pages precede children

ekamran approved these changes Oct 9, 2026

View reviewed changes

mindctrl merged commit 6f59864 into WordPress:main Oct 13, 2026

github-project-automation Bot moved this from Approved to Done in WordPress Hosting Team Oct 13, 2026

Navigation Menu

Repository navigation

Fix manifest generator dropping handbook hierarchy - #442

Fix manifest generator dropping handbook hierarchy#442
mindctrl merged 8 commits into
WordPress:mainfrom
sonali195:fix/handbook-manifest-generator

sonali195 commented Sep 28, 2026 •

edited

Loading

Uh oh!

vairafiq commented Oct 2, 2026

Uh oh!

This comment was marked as outdated.

mindctrl left a comment

Uh oh!

mindctrl left a comment

Uh oh!

sonali195 commented Oct 9, 2026

Uh oh!

ekamran left a comment

Uh oh!

Uh oh!

Uh oh!

ekamran left a comment

Uh oh!

sonali195 commented Oct 12, 2026

Uh oh!

mindctrl commented Oct 12, 2026

Uh oh!

sonali195 commented Oct 12, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

sonali195 commented Sep 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Why

Changes

Testing

AI Assistance Disclosure

Uh oh!

vairafiq commented Oct 2, 2026

Uh oh!

This comment was marked as outdated.

mindctrl left a comment

Choose a reason for hiding this comment

Uh oh!

mindctrl left a comment

Choose a reason for hiding this comment

Uh oh!

sonali195 commented Oct 9, 2026

What

Why

Changes

Testing

Uh oh!

ekamran left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

ekamran left a comment

Choose a reason for hiding this comment

Uh oh!

sonali195 commented Oct 12, 2026

Uh oh!

mindctrl commented Oct 12, 2026

Uh oh!

sonali195 commented Oct 12, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

sonali195 commented Sep 28, 2026 •

edited

Loading