Skip to content

Fix GH-23617: Keep a past-the-end internal pointer past the end on compaction - #23620

Open
darkdi wants to merge 1 commit into
php:masterfrom
darkdi:fix/preserve-array-internal-pointer
Open

darkdi wants to merge 1 commit into
php:masterfrom
darkdi:fix/preserve-array-internal-pointer

Conversation

@darkdi

@darkdi darkdi commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

An exhausted internal pointer now moves to the new end whenever holes get compacted, in zend_array_dup() and in zend_hash_rehash(), the same way past-the-end iterators already do. The test covers packed and mixed, with and without holes, full table included, each with and without separation.

Fixes GH-23617.

@devnexen

devnexen commented Sep 8, 2026

Copy link
Copy Markdown
Member

Hi @darkdi thanks for the patch but I m afraid the fix is really incorrect, I ll let ilaal review tough.

@iliaal

iliaal commented Sep 8, 2026

Copy link
Copy Markdown
Member

Direction looks right: matching the immutable path lines up with the MIN(nInternalPointer, nNumUsed) clamp tail deletion already uses, and an exhausted cursor picking up a later append is what the same code does with no separation at all. Two problems though.

The .phpt expectation is wrong and the test fails on this branch. end(); next(); then an append gives "d" at refcount 1 with no copy involved, so NULL is not a behaviour being restored. The test body prints "d", 4, 3, 4 here.

And it is incomplete for mixed arrays with holes, where zend_array_dup_elements() compacts and target->nNumUsed ends up below the preserved pointer:

array no separation separated, this PR
['a'=>1,'b'=>2,'c'=>3] 'zz' 'zz'
same, two keys unset 'zz' NULL
8 keys, 3 interior unset NULL NULL

Setting target->nInternalPointer = idx fixes row 2 but breaks row 3, since a full table rehashes on the append and zend_hash_rehash() does not remap an exhausted pointer either. So no single copied value agrees with the middle column everywhere; normalising the exhausted cursor wherever compaction happens, rehash included, looks like the actual fix.

The NULL expectation came from my issue text, which was wrong. I have corrected GH-23617.

… compaction

zend_array_dup() reset an exhausted internal pointer to 0, and both it and
zend_hash_rehash() left it at the old nNumUsed when holes were compacted, so
a later append was either skipped or the pointer jumped back to the first
element. Move it to the new end instead, the same way past-the-end
iterators are already migrated.

Signed-off-by: Dmitry Rantovov <rantovov5@gmail.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Puqi4fj3kWhLHanMyF1UNU
@darkdi
darkdi force-pushed the fix/preserve-array-internal-pointer branch from aa8c24f to 4792857 Compare September 11, 2026 16:29
@darkdi darkdi changed the title Preserve array internal pointer during copy-on-write separation Fix GH-23617: Keep a past-the-end internal pointer past the end on compaction Sep 11, 2026
@darkdi

darkdi commented Sep 11, 2026

Copy link
Copy Markdown
Contributor Author

Thanks, you were right on both. Reworked it the way you suggested: the exhausted cursor now goes to the new nNumUsed after compaction in both zend_array_dup_elements() and zend_hash_rehash(), not just copied. Built locally this time, your three rows plus packed with and without holes all give the appended element now, separated or not, and the new test fails on master for the holes cases. Zend/tests and ext/standard/tests/array pass

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Internal pointer past the end resets to the first element on copy-on-write separation

3 participants