Skip to content

[DomCrawler] Add DomCrawler to leverage PHP 8.4's HTML5-compliant DOM parser #61356

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
nicolas-grekas wants to merge 1 commit into from
Closed

[DomCrawler] Add DomCrawler to leverage PHP 8.4's HTML5-compliant DOM parser #61356

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions UPGRADE-7.4.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,11 @@ Console

* Deprecate `Symfony\Component\Console\Application::add()` in favor of `addCommand()`

BrowserKit
----------

* Leverage the native HTML5 parser when using PHP 8.4+
Copy link
Member

@stof stof Aug 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a BC break for code interacting with DomCrawler methods returning a raw node as the type has changed. this should be documented as such.


DependencyInjection
-------------------

Expand All @@ -29,6 +34,16 @@ DoctrineBridge

* Deprecate `UniqueEntity::getRequiredOptions()` and `UniqueEntity::getDefaultOption()`

DomCrawler
----------

* [BC BREAK] Type widening on public and protected methods/properties to support both
`Dom\*` and `DOM*` native classes for:
* properties `FormField::$document`, `$xpath`, `$node` and method `getLabel()`
* methods `Form::getFormNode()` and `addField()`
* property `AbstractUriElement::$node`, and methods `getNode()` and `setNode()`
* methods `Crawler::add()`, `addDocument()`, `addNodeList()`, `addNode()`, `getNode()` and `sibling()`
Copy link
Member

@stof stof Aug 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

type widening for getNode is a BC impacting caller code rather than child classes due to widening a return type instead of a parameter type (same for other getters in other classes). This should be highlighted IMO.

Copy link
Member

@stof stof Aug 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And this is not just theoretical. https://github.com/minkphp/MinkBrowserKitDriver will be totally broken by this BC break affecting consumers of the library (and immediately because of the BC break in BrowserKit switching to the BC-breaking implementation)


FrameworkBundle
---------------

Expand Down
7 changes: 4 additions & 3 deletions src/Symfony/Bundle/FrameworkBundle/Tests/Functional/ApiAttributesTest.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@

use PHPUnit\Framework\Attributes\DataProvider;
use Symfony\Component\DomCrawler\Crawler;
use Symfony\Component\DomCrawler\DomCrawler;
use Symfony\Component\HttpFoundation\JsonResponse;
use Symfony\Component\HttpFoundation\Request;
use Symfony\Component\HttpFoundation\Response;
Expand Down Expand Up @@ -426,7 +427,7 @@ public static function mapRequestPayloadProvider(): iterable
</request>
XML,
'responseAssertion' => static function (string $response) {
$crawler = new Crawler($response);
$crawler = \PHP_VERSION_ID >= 80400 && class_exists(DomCrawler::class) ? new DomCrawler($response) : new Crawler($response);

self::assertSame('https://symfony.com/errors/validation', $crawler->filterXPath('response/type')->text());
self::assertSame('Validation Failed', $crawler->filterXPath('response/title')->text());
Expand Down Expand Up @@ -642,7 +643,7 @@ public static function mapRequestPayloadProvider(): iterable
</request>
XML,
'responseAssertion' => static function (string $response) {
$crawler = new Crawler($response);
$crawler = \PHP_VERSION_ID >= 80400 && class_exists(DomCrawler::class) ? new DomCrawler($response) : new Crawler($response);

self::assertSame('https://symfony.com/errors/validation', $crawler->filterXPath('response/type')->text());
self::assertSame('Validation Failed', $crawler->filterXPath('response/title')->text());
Expand Down Expand Up @@ -860,7 +861,7 @@ public static function mapRequestPayloadProvider(): iterable
</request>
XML,
'responseAssertion' => static function (string $response) {
$crawler = new Crawler($response);
$crawler = \PHP_VERSION_ID >= 80400 && class_exists(DomCrawler::class) ? new DomCrawler($response) : new Crawler($response);

self::assertSame('https://symfony.com/errors/validation', $crawler->filterXPath('response/type')->text());
self::assertSame('Validation Failed', $crawler->filterXPath('response/title')->text());
Expand Down
99 changes: 65 additions & 34 deletions src/Symfony/Bundle/FrameworkBundle/Tests/Test/WebTestCaseTest.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
use Symfony\Component\BrowserKit\CookieJar;
use Symfony\Component\BrowserKit\History;
use Symfony\Component\DomCrawler\Crawler;
use Symfony\Component\DomCrawler\DomCrawler;
use Symfony\Component\HttpFoundation\Cookie as HttpFoundationCookie;
use Symfony\Component\HttpFoundation\Request;
use Symfony\Component\HttpFoundation\Response;
Expand Down Expand Up @@ -231,126 +232,156 @@ public function testAssertBrowserHistoryIsNotOnLastPage()

public function testAssertSelectorExists()
{
$this->getCrawlerTester(new Crawler('<html><body><h1>'))->assertSelectorExists('body > h1');
$crawler = \PHP_VERSION_ID >= 80400 && class_exists(DomCrawler::class) ? DomCrawler::class : Crawler::class;

$this->getCrawlerTester(new $crawler('<html><body><h1>'))->assertSelectorExists('body > h1');
$this->expectException(AssertionFailedError::class);
$this->expectExceptionMessage('matches selector "body > h1".');
$this->getCrawlerTester(new Crawler('<html><head><title>Foo'))->assertSelectorExists('body > h1');
$this->getCrawlerTester(new $crawler('<html><head><title>Foo'))->assertSelectorExists('body > h1');
}

public function testAssertSelectorNotExists()
{
$this->getCrawlerTester(new Crawler('<html><head><title>Foo'))->assertSelectorNotExists('body > h1');
$crawler = \PHP_VERSION_ID >= 80400 && class_exists(DomCrawler::class) ? DomCrawler::class : Crawler::class;

$this->getCrawlerTester(new $crawler('<html><head><title>Foo'))->assertSelectorNotExists('body > h1');
$this->expectException(AssertionFailedError::class);
$this->expectExceptionMessage('does not match selector "body > h1".');
$this->getCrawlerTester(new Crawler('<html><body><h1>'))->assertSelectorNotExists('body > h1');
$this->getCrawlerTester(new $crawler('<html><body><h1>'))->assertSelectorNotExists('body > h1');
}

public function testAssertSelectorCount()
{
$this->getCrawlerTester(new Crawler('<html><body><p>Hello</p></body></html>'))->assertSelectorCount(1, 'p');
$this->getCrawlerTester(new Crawler('<html><body><p>Hello</p><p>Foo</p></body></html>'))->assertSelectorCount(2, 'p');
$this->getCrawlerTester(new Crawler('<html><body><h1>This is not a paragraph.</h1></body></html>'))->assertSelectorCount(0, 'p');
$crawler = \PHP_VERSION_ID >= 80400 && class_exists(DomCrawler::class) ? DomCrawler::class : Crawler::class;

$this->getCrawlerTester(new $crawler('<html><body><p>Hello</p></body></html>'))->assertSelectorCount(1, 'p');
$this->getCrawlerTester(new $crawler('<html><body><p>Hello</p><p>Foo</p></body></html>'))->assertSelectorCount(2, 'p');
$this->getCrawlerTester(new $crawler('<html><body><h1>This is not a paragraph.</h1></body></html>'))->assertSelectorCount(0, 'p');
$this->expectException(AssertionFailedError::class);
$this->expectExceptionMessage('Failed asserting that the Crawler selector "p" was expected to be found 0 time(s) but was found 1 time(s).');
$this->getCrawlerTester(new Crawler('<html><body><p>Hello</p></body></html>'))->assertSelectorCount(0, 'p');
$this->getCrawlerTester(new $crawler('<html><body><p>Hello</p></body></html>'))->assertSelectorCount(0, 'p');
}

public function testAssertSelectorTextNotContains()
{
$this->getCrawlerTester(new Crawler('<html><body><h1>Foo'))->assertSelectorTextNotContains('body > h1', 'Bar');
$crawler = \PHP_VERSION_ID >= 80400 && class_exists(DomCrawler::class) ? DomCrawler::class : Crawler::class;

$this->getCrawlerTester(new $crawler('<html><body><h1>Foo'))->assertSelectorTextNotContains('body > h1', 'Bar');
$this->expectException(AssertionFailedError::class);
$this->expectExceptionMessage('matches selector "body > h1" and the text "Foo" of the node matching selector "body > h1" does not contain "Foo".');
$this->getCrawlerTester(new Crawler('<html><body><h1>Foo'))->assertSelectorTextNotContains('body > h1', 'Foo');
$this->getCrawlerTester(new $crawler('<html><body><h1>Foo'))->assertSelectorTextNotContains('body > h1', 'Foo');
}

public function testAssertAnySelectorTextContains()
{
$this->getCrawlerTester(new Crawler('<ul><li>Bar</li><li>Foo Baz'))->assertAnySelectorTextContains('ul li', 'Foo');
$crawler = \PHP_VERSION_ID >= 80400 && class_exists(DomCrawler::class) ? DomCrawler::class : Crawler::class;

$this->getCrawlerTester(new $crawler('<ul><li>Bar</li><li>Foo Baz'))->assertAnySelectorTextContains('ul li', 'Foo');
$this->expectException(AssertionFailedError::class);
$this->expectExceptionMessage('matches selector "ul li" and the text of any node matching selector "ul li" contains "Foo".');
$this->getCrawlerTester(new Crawler('<ul><li>Bar</li><li>Baz'))->assertAnySelectorTextContains('ul li', 'Foo');
$this->getCrawlerTester(new $crawler('<ul><li>Bar</li><li>Baz'))->assertAnySelectorTextContains('ul li', 'Foo');
}

public function testAssertAnySelectorTextSame()
{
$this->getCrawlerTester(new Crawler('<ul><li>Bar</li><li>Foo'))->assertAnySelectorTextSame('ul li', 'Foo');
$crawler = \PHP_VERSION_ID >= 80400 && class_exists(DomCrawler::class) ? DomCrawler::class : Crawler::class;

$this->getCrawlerTester(new $crawler('<ul><li>Bar</li><li>Foo'))->assertAnySelectorTextSame('ul li', 'Foo');
$this->expectException(AssertionFailedError::class);
$this->expectExceptionMessage('matches selector "ul li" and has at least a node matching selector "ul li" with content "Foo".');
$this->getCrawlerTester(new Crawler('<ul><li>Bar</li><li>Baz'))->assertAnySelectorTextSame('ul li', 'Foo');
$this->getCrawlerTester(new $crawler('<ul><li>Bar</li><li>Baz'))->assertAnySelectorTextSame('ul li', 'Foo');
}

public function testAssertAnySelectorTextNotContains()
{
$this->getCrawlerTester(new Crawler('<ul><li>Bar</li><li>Baz'))->assertAnySelectorTextNotContains('ul li', 'Foo');
$crawler = \PHP_VERSION_ID >= 80400 && class_exists(DomCrawler::class) ? DomCrawler::class : Crawler::class;

$this->getCrawlerTester(new $crawler('<ul><li>Bar</li><li>Baz'))->assertAnySelectorTextNotContains('ul li', 'Foo');
$this->expectException(AssertionFailedError::class);
$this->expectExceptionMessage('matches selector "ul li" and the text of any node matching selector "ul li" does not contain "Foo".');
$this->getCrawlerTester(new Crawler('<ul><li>Bar</li><li>Foo'))->assertAnySelectorTextNotContains('ul li', 'Foo');
$this->getCrawlerTester(new $crawler('<ul><li>Bar</li><li>Foo'))->assertAnySelectorTextNotContains('ul li', 'Foo');
}

public function testAssertPageTitleSame()
{
$this->getCrawlerTester(new Crawler('<html><head><title>Foo'))->assertPageTitleSame('Foo');
$crawler = \PHP_VERSION_ID >= 80400 && class_exists(DomCrawler::class) ? DomCrawler::class : Crawler::class;

$this->getCrawlerTester(new $crawler('<html><head><title>Foo'))->assertPageTitleSame('Foo');
$this->expectException(AssertionFailedError::class);
$this->expectExceptionMessage('matches selector "title" and has a node matching selector "title" with content "Bar".');
$this->getCrawlerTester(new Crawler('<html><head><title>Foo'))->assertPageTitleSame('Bar');
$this->getCrawlerTester(new $crawler('<html><head><title>Foo'))->assertPageTitleSame('Bar');
}

public function testAssertPageTitleContains()
{
$this->getCrawlerTester(new Crawler('<html><head><title>Foobar'))->assertPageTitleContains('Foo');
$crawler = \PHP_VERSION_ID >= 80400 && class_exists(DomCrawler::class) ? DomCrawler::class : Crawler::class;

$this->getCrawlerTester(new $crawler('<html><head><title>Foobar'))->assertPageTitleContains('Foo');
$this->expectException(AssertionFailedError::class);
$this->expectExceptionMessage('matches selector "title" and the text "Foo" of the node matching selector "title" contains "Bar".');
$this->getCrawlerTester(new Crawler('<html><head><title>Foo'))->assertPageTitleContains('Bar');
$this->getCrawlerTester(new $crawler('<html><head><title>Foo'))->assertPageTitleContains('Bar');
}

public function testAssertInputValueSame()
{
$this->getCrawlerTester(new Crawler('<html><body><form><input type="text" name="username" value="Fabien">'))->assertInputValueSame('username', 'Fabien');
$crawler = \PHP_VERSION_ID >= 80400 && class_exists(DomCrawler::class) ? DomCrawler::class : Crawler::class;

$this->getCrawlerTester(new $crawler('<html><body><form><input type="text" name="username" value="Fabien">'))->assertInputValueSame('username', 'Fabien');
$this->expectException(AssertionFailedError::class);
$this->expectExceptionMessage('matches selector "input[name="password"]" and has a node matching selector "input[name="password"]" with attribute "value" of value "pa$$".');
$this->getCrawlerTester(new Crawler('<html><head><title>Foo'))->assertInputValueSame('password', 'pa$$');
$this->getCrawlerTester(new $crawler('<html><head><title>Foo'))->assertInputValueSame('password', 'pa$$');
}

public function testAssertInputValueNotSame()
{
$this->getCrawlerTester(new Crawler('<html><body><input type="text" name="username" value="Helene">'))->assertInputValueNotSame('username', 'Fabien');
$crawler = \PHP_VERSION_ID >= 80400 && class_exists(DomCrawler::class) ? DomCrawler::class : Crawler::class;

$this->getCrawlerTester(new $crawler('<html><body><input type="text" name="username" value="Helene">'))->assertInputValueNotSame('username', 'Fabien');
$this->expectException(AssertionFailedError::class);
$this->expectExceptionMessage('matches selector "input[name="password"]" and does not have a node matching selector "input[name="password"]" with attribute "value" of value "pa$$".');
$this->getCrawlerTester(new Crawler('<html><body><form><input type="text" name="password" value="pa$$">'))->assertInputValueNotSame('password', 'pa$$');
$this->getCrawlerTester(new $crawler('<html><body><form><input type="text" name="password" value="pa$$">'))->assertInputValueNotSame('password', 'pa$$');
}

public function testAssertCheckboxChecked()
{
$this->getCrawlerTester(new Crawler('<html><body><form><input type="checkbox" name="rememberMe" checked>'))->assertCheckboxChecked('rememberMe');
$this->getCrawlerTester(new Crawler('<!DOCTYPE html><body><form><input type="checkbox" name="rememberMe" checked>'))->assertCheckboxChecked('rememberMe');
$crawler = \PHP_VERSION_ID >= 80400 && class_exists(DomCrawler::class) ? DomCrawler::class : Crawler::class;

$this->getCrawlerTester(new $crawler('<html><body><form><input type="checkbox" name="rememberMe" checked>'))->assertCheckboxChecked('rememberMe');
$this->getCrawlerTester(new $crawler('<!DOCTYPE html><body><form><input type="checkbox" name="rememberMe" checked>'))->assertCheckboxChecked('rememberMe');
$this->expectException(AssertionFailedError::class);
$this->expectExceptionMessage('matches selector "input[name="rememberMe"]:checked".');
$this->getCrawlerTester(new Crawler('<html><body><form><input type="checkbox" name="rememberMe">'))->assertCheckboxChecked('rememberMe');
$this->getCrawlerTester(new $crawler('<html><body><form><input type="checkbox" name="rememberMe">'))->assertCheckboxChecked('rememberMe');
}

public function testAssertCheckboxNotChecked()
{
$this->getCrawlerTester(new Crawler('<html><body><form><input type="checkbox" name="rememberMe">'))->assertCheckboxNotChecked('rememberMe');
$this->getCrawlerTester(new Crawler('<!DOCTYPE html><body><form><input type="checkbox" name="rememberMe">'))->assertCheckboxNotChecked('rememberMe');
$crawler = \PHP_VERSION_ID >= 80400 && class_exists(DomCrawler::class) ? DomCrawler::class : Crawler::class;

$this->getCrawlerTester(new $crawler('<html><body><form><input type="checkbox" name="rememberMe">'))->assertCheckboxNotChecked('rememberMe');
$this->getCrawlerTester(new $crawler('<!DOCTYPE html><body><form><input type="checkbox" name="rememberMe">'))->assertCheckboxNotChecked('rememberMe');
$this->expectException(AssertionFailedError::class);
$this->expectExceptionMessage('does not match selector "input[name="rememberMe"]:checked".');
$this->getCrawlerTester(new Crawler('<html><body><form><input type="checkbox" name="rememberMe" checked>'))->assertCheckboxNotChecked('rememberMe');
$this->getCrawlerTester(new $crawler('<html><body><form><input type="checkbox" name="rememberMe" checked>'))->assertCheckboxNotChecked('rememberMe');
}

public function testAssertFormValue()
{
$this->getCrawlerTester(new Crawler('<html><body><form id="form"><input type="text" name="username" value="Fabien">', 'http://localhost'))->assertFormValue('#form', 'username', 'Fabien');
$crawler = \PHP_VERSION_ID >= 80400 && class_exists(DomCrawler::class) ? DomCrawler::class : Crawler::class;

$this->getCrawlerTester(new $crawler('<html><body><form id="form"><input type="text" name="username" value="Fabien">', 'http://localhost'))->assertFormValue('#form', 'username', 'Fabien');
$this->expectException(AssertionFailedError::class);
$this->expectExceptionMessage('Failed asserting that two strings are identical.');
$this->getCrawlerTester(new Crawler('<html><body><form id="form"><input type="text" name="username" value="Fabien">', 'http://localhost'))->assertFormValue('#form', 'username', 'Jane');
$this->getCrawlerTester(new $crawler('<html><body><form id="form"><input type="text" name="username" value="Fabien">', 'http://localhost'))->assertFormValue('#form', 'username', 'Jane');
}

public function testAssertNoFormValue()
{
$this->getCrawlerTester(new Crawler('<html><body><form id="form"><input type="checkbox" name="rememberMe">', 'http://localhost'))->assertNoFormValue('#form', 'rememberMe');
$crawler = \PHP_VERSION_ID >= 80400 && class_exists(DomCrawler::class) ? DomCrawler::class : Crawler::class;

$this->getCrawlerTester(new $crawler('<html><body><form id="form"><input type="checkbox" name="rememberMe">', 'http://localhost'))->assertNoFormValue('#form', 'rememberMe');
$this->expectException(AssertionFailedError::class);
$this->expectExceptionMessage('Field "rememberMe" has a value in form "#form".');
$this->getCrawlerTester(new Crawler('<html><body><form id="form"><input type="checkbox" name="rememberMe" checked>', 'http://localhost'))->assertNoFormValue('#form', 'rememberMe');
$this->getCrawlerTester(new $crawler('<html><body><form id="form"><input type="checkbox" name="rememberMe" checked>', 'http://localhost'))->assertNoFormValue('#form', 'rememberMe');
}

public function testAssertRequestAttributeValueSame()
Expand Down
7 changes: 6 additions & 1 deletion src/Symfony/Component/BrowserKit/AbstractBrowser.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
use Symfony\Component\BrowserKit\Exception\LogicException;
use Symfony\Component\BrowserKit\Exception\RuntimeException;
use Symfony\Component\DomCrawler\Crawler;
use Symfony\Component\DomCrawler\DomCrawler;
use Symfony\Component\DomCrawler\Form;
use Symfony\Component\DomCrawler\Link;
use Symfony\Component\Process\PhpProcess;
Expand Down Expand Up @@ -510,7 +511,11 @@ protected function createCrawlerFromContent(string $uri, string $content, string
return null;
}

$crawler = new Crawler(null, $uri, null, $this->useHtml5Parser);
if (\PHP_VERSION_ID >= 80400 && $this->useHtml5Parser && class_exists(DomCrawler::class)) {
$crawler = new DomCrawler(null, $uri);
} else {
$crawler = new Crawler(null, $uri, null, $this->useHtml5Parser);
}
$crawler->addContent($content, $type);

return $crawler;
Expand Down
1 change: 1 addition & 0 deletions src/Symfony/Component/BrowserKit/CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ CHANGELOG

* Add `isFirstPage()` and `isLastPage()` methods to the History class for checking navigation boundaries
* Add PHPUnit constraints: `BrowserHistoryIsOnFirstPage` and `BrowserHistoryIsOnLastPage`
* Leverage the native HTML5 parser when using PHP 8.4+

6.4
---
Expand Down
Loading
Loading