Entirety of speculation rules should be filterable, not just the paths to exclude

@swissspidy

Feature Description

Currently the Speculative Loading plugin provides a plsr_speculation_rules_href_exclude_paths filter to exclude URLs (and URL patterns) from being speculatively loaded. However, two situations came up recently where this was not sufficient.

In Products added to cart in WooCommerce without clicking on Add to Cart button when Speculative Loading active #1140, WooCommerce was found to create non-idempotent add-to-cart links which add products to cart when prefetched/prerendered. WooCommerce seems to try to get around the non-idempotency by adding rel=nofollow to the links. However, there is no way to exclude links via such an attribute without manually modifying the default rules, which was done in Exclude rel=nofollow links from prefetch/prerender #1142. (This was to avoid having to add a WooCommerce-specific URLPattern to exclude URLs with an add-to-cart query param, since excluding links with rel=nofollow may be generally advisable: Should we exclude rel=nofollow by default? WICG/nav-speculation#309).
In a comment on the aforementioned issue, a user wanted to customize the speculation rules so that specific URLs were opted-in for eager prerendering. That is, instead of specifying a list of URLs/URLPatterns to exclude, they wanted to provide a list of URLs/URLPatterns to include. This is not possible at present either.

To account for these two use cases, I suggest that the entire set of speculation rules (speculation ruleset?) be filterable, doing something like this:

--- a/plugins/speculation-rules/helper.php
+++ b/plugins/speculation-rules/helper.php
@@ -109,5 +109,13 @@ function plsr_get_speculation_rules() {
 		);
 	}
 
-	return array( $mode => $rules );
+	$ruleset = array( $mode => $rules );
+
+	/**
+	 * Filters the entire speculation ruleset.
+	 *
+	 * @param array  $ruleset Ruleset.
+	 * @param string $mode    Mode used to apply speculative prerendering. Either 'prefetch' or 'prerender'.
+	 */
+	return (array) apply_filters( 'plsr_speculation_ruleset', $ruleset, $mode );
 }

Also, @swissspidy suggested in #1144 (comment) that a JSON Schema could be written which could validate whatever is being returned by the filter. If not valid, it could trigger a _doing_it_wrong().

@tunetheweb

I had thought that only one speculationrules is allowed on page, making it more important to allow the entire single ruleset to be filterable. But @tunetheweb corrected me by pointing out that Multiple speculation rules are indeed allowed. They all get merged together. So this lessens the need for this plugin's speculation rules to be filterable.

It would be great to have the ability to set a limit when using wildcards. For example, eagerly prerender /product/* up to 8. In this case, given the limit of 10, we would still have 2 available on hover "slots" provided by the Moderate setting.

@swissspidy

Also, @swissspidy suggested in #1144 (comment) that a JSON Schema could be written which could validate whatever is being returned by the filter. If not valid, it could trigger a _doing_it_wrong().

For context, my original use case was validation in unit tests, but runtime validation is interesting too. To make sure that's not too slow, I'd transform the JSON schema into a PHP file and then use that for the validation. Then there's no need to do any JSON parsing.

It would be great to have the ability to set a limit when using wildcards. For example, eagerly prerender /product/* up to 8. In this case, given the limit of 10, we would still have 2 available on hover "slots" provided by the Moderate setting.

This sounds like a feature request for https://github.com/WICG/nav-speculation/issues.

It would be great to have the ability to set a limit when using wildcards. For example, eagerly prerender /product/* up to 8. In this case, given the limit of 10, we would still have 2 available on hover "slots" provided by the Moderate setting.

This sounds like a feature request for https://github.com/WICG/nav-speculation/issues.

These are already separate limits. so you can prerender up to 10 eagerly, and 2 moderately.

These are already separate limits. so you can prerender up to 10 eagerly, and 2 moderately.

Check https://altsetup.com. I have:

and

<script type="speculationrules">
{
  "prerender": [{
    "where": {
      "and": [
        { "href_matches": "/*" },
        { "not": {"selector_matches": ".do-not-prerender"}},
        { "not": {"selector_matches": "[rel=nofollow]"}}
      ]
    },
    "eagerness": "eager"
  }]
}
</script>

The 10 eager prerenders happen, but the 2 moderate ones (on hover) don't. Why?

I believe because Chrome has a limit of 10 prerenders.

@tunetheweb

I believe because Chrome has a limit of 10 prerenders.

According to @tunetheweb, the limits are 10 eagerly + 2 moderately, and I can confirm it works like that on another site I'm testing it on. The issue is on https://altsetup.com and some other sites having similar stack. I can provide admin access to check it.

EDIT
It's like if an eager prerender is not performed ("failure" state) because the initiating page already has too many prerenders ongoing, then it is not a candidate anymore for the moderate prerendering on hover. It happens in case of href_matches": "/*"

I see Chrome is reporting this error with the prerenders:

Failure - The prerender whose eagerness is "eager" was not performed because the initiating page already has too many prerenders ongoing. Remove other speculation rules with "eager" to enable further prerendering.

I see there is only one URL which is "Not triggered", and that is the Contact page:

However, that /contact/ URL was also triggered and failed. So yeah, it seems that prevents it from being attempted.

Did you try excluding /contact/ from the eager rules? I believe this is it:

<script type="speculationrules">
{
  "prerender": [{
    "where": {
      "and": [
        { "href_matches": "/*" },
        { "not": { "href_matches": "/contact/" }},
        { "not": {"selector_matches": ".do-not-prerender"}},
        { "not": {"selector_matches": "[rel=nofollow]"}}
      ]
    },
    "eagerness": "eager"
  }]
}
</script>

I see there is only one URL which is "Not triggered", and that is the Contact page:

Actually all URLs having failure state are not eligible anymore to be triggered on hover. It's more clear if you navigate on any post page, where there are related posts URLs on the bottom. All those are not triggered on hover because excluded by the initial 10 eager prerenders without being excluded.

Did you try excluding /contact/ from the eager rules? I believe this is it:

I just replaced the script with yours, and now /contact/ prerender is triggered on hover, as you can see.

This is basically an extension of this bug: https://issues.chromium.org/issues/335277576. Once a speculation fails it is not retried.

But to be honest a high eagerness is best used with a targeted list of URLs where you KNOW there is a high probability the link will be used. The lower eagerness (moderate or conservative) is for URLs where this cannot be predicted.

The scattergun approach of trying to eagerly fetch ALL URLs, with the low eagerness backup for the same URLs is not recommended. Speculating 12 URLs per page load is a LOT of potential wastage which will affect both your hosting costs (every visitor is now effectively 12 visitors) and, more importantly, your visitors bandwidth and CPU resources. This is precisely why we have this 10 URL limit. And it be honest I think that's a little high and may be lowered if we see a lot of this scattergun approach.

I would suggest a more targeted approach for eagerly fetched URLs, or just using less eagerly fetched URLs.

What's the status here, are we planning on adding a filter at least in WordPress/wordpress-develop#7860? It would really be a best practice to give developers full control.

Correct.

If the default behaviour is to prerender and they want to do what you want, then to me they are asking for two extra things from the default:

prefetch eagerly
opt-out of their default to prerender.

Hence two classes. I don't think we need to create one class for all combinations just to avoid the need for two classes as they'll just get confusing.

Thanks!

Regarding the implementation of this, I think we should implement support for these classes as a follow-up enhancement via separate WordPress Trac ticket, once the initial feature has been merged. That keeps PRs manageable, and since this works well as a standalone enhancement, it makes sense. It could still be merged into the same WordPress Core release.

SGTM. We could also experiment with it in the Plugin in the meantime if we think that would be useful and/or the core implementation is still some way off.

@tunetheweb

@tunetheweb I agree with starting this as an experiment in the plugin, let's do that.

If we want to do that, we will need to be able to modify the overall speculation rules once speculative loading is in WordPress Core, so that brings me back to the original purpose of this issue.

For the previously mentioned reasons in #1156 (comment) I don't think a simple filter for the entire complex array is the right approach. Thinking about this further, I think a more explicit API would help reduce that complexity and also guide it to do the right thing. Instead of a filter, we could fire an action that receives a WP_Speculation_Rules object (based on a PHP class) to modify.

This action could be used something like this (taking one of the examples discussed above):

add_action(
	'wp_set_speculation_rules',
	function ( WP_Speculation_Rules $speculation_rules ) {
		$speculation_rules->add_rule(
			// The `$mode` parameter would be either 'prefetch' or 'prerender'.
			'prefetch',
			// The `$rule` parameter would be an array representing a single rule.
			array(
				'source'    => 'document',
				'where'     => array(
					'selector_matches' => '.eager-prefetch, .eager-prefetch a',
				),
				'eagerness' => 'eager',
			)
		);
	}
);

The WP_Speculation_Rules class could be very simple as a starting point:

two properties storing the array of rules, one for prefetch, one for prerender
method add_rule( string $mode, array $rule ) to add a new rule
method to_array() to return the entire speculation rules array representation
optional: helper methods e.g. to check for presence of a specific rule or to see whether there are any rules set for prefetch or prender specifically
more methods could be added in the future as needed, but the above would be enough for an MVP

WDYT about this idea @swissspidy @westonruter?

There seems to be something missing here, namely the ability to remove a speculation rule that has been added. Should there be a concept of a rule ID/handle/name which can be used to access previously-added rules?

Would there be a need to reorder speculation rules?

Note also that there wouldn't have to be a to_array() method as the entire WP_Speculation_Rules class could implement JsonSerializable and so instead of to_array() it could use jsonSerialize(). This would allow you to pass a WP_Speculation_Rules object directly into json_serialize() for printing in the SCRIPT. Optimization Detective should be doing this but isn't at present. It's currently doing:

performance/plugins/optimization-detective/storage/class-od-url-metrics-post-type.php

Lines 245 to 253 in d94977f

    
           $post_data['post_content'] = wp_json_encode( 
        
           	array_map( 
        
           		static function ( OD_URL_Metric $url_metric ): array { 
        
           			return $url_metric->jsonSerialize(); 
        
           		}, 
        
           		$group_collection->get_flattened_url_metrics() 
        
           	), 
        
           	JSON_UNESCAPED_SLASHES // No need for escaped slashes since not printed to frontend. 
        
           );

But it can be more simply:

$post_data['post_content'] = wp_json_encode(
	$group_collection->get_flattened_url_metrics(),
	JSON_UNESCAPED_SLASHES // No need for escaped slashes since not printed to frontend.
);

There seems to be something missing here, namely the ability to remove a speculation rule that has been added. Should there be a concept of a rule ID/handle/name which can be used to access previously-added rules?

That makes sense. Worth noting that this could always be added later - in theory it would make sense to add it, but I think we should come up with real use-cases for where removal of a rule would be helpful first, or otherwise wait for use-cases to come up if there are. With that said, your point on ID/handle/name would be crucial for that (and possibly other interactions like amending a rule), so I think it's a great idea to build that in from the start to make the API future-proof. I assume we could just use some unique string identifier that the caller would provide?

Note also that there wouldn't have to be a to_array() method as the entire WP_Speculation_Rules class could implement JsonSerializable and so instead of to_array() it could use jsonSerialize(). This would allow you to pass a WP_Speculation_Rules object directly into json_serialize() for printing in the SCRIPT.

I like that idea, let's do it.

I assume we could just use some unique string identifier that the caller would provide?

Yeah, any unique string. Similar to the IDs that are used for Customizer settings, controls, and sections.

I have added the above approach for adding further speculation rules to the Core PR, alongside a few other somewhat related fixes, see WordPress/wordpress-develop#7860 (comment).

This can be closed I think, as the request has been implemented in the WordPress Core API, which was merged last week. It's not available in the plugin's implementation, but that's not worth the effort at this point since it'll be phased out soon.

It's not available in the plugin's implementation, but that's not worth the effort at this point since it'll be phased out soon.

Phased out soon in what way? Won't this plugin stick around as the way to opt-in to moderate prerendering?

It's not available in the plugin's implementation, but that's not worth the effort at this point since it'll be phased out soon.

Phased out soon in what way? Won't this plugin stick around as the way to opt-in to moderate prerendering?

I meant that the plugin's "re-implementation" of the Core API will be phased out soon. As soon as we require 6.8 (which probably will happen in the next half year or so), there would be no point to maintain the plugin's own implementation anymore. The plugin of course yes, but the code path not using the Core API would be removed.

Navigation Menu

Repository navigation

Entirety of speculation rules should be filterable, not just the paths to exclude #1156

Feature Description

11 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	$post_data['post_content'] = wp_json_encode(
	array_map(
	static function ( OD_URL_Metric $url_metric ): array {
	return $url_metric->jsonSerialize();
	},
	$group_collection->get_flattened_url_metrics()
	),
	JSON_UNESCAPED_SLASHES // No need for escaped slashes since not printed to frontend.
	);

Entirety of speculation rules should be filterable, not just the paths to exclude #1156

Description

Feature Description

Activity

westonruter commented on Apr 16, 2024

masvil commented on Apr 17, 2024

swissspidy commented on Apr 23, 2024

tunetheweb commented on Apr 23, 2024

masvil commented on Apr 24, 2024

westonruter commented on Apr 24, 2024

masvil commented on Apr 24, 2024

westonruter commented on Apr 24, 2024

masvil commented on Apr 24, 2024

tunetheweb commented on Apr 24, 2024

swissspidy commented on Jan 14, 2025

11 remaining items

tunetheweb commented on Jan 28, 2025

felixarntz commented on Jan 28, 2025

tunetheweb commented on Jan 28, 2025

felixarntz commented on Jan 30, 2025

westonruter commented on Feb 1, 2025

felixarntz commented on Feb 3, 2025

westonruter commented on Feb 3, 2025

felixarntz commented on Feb 3, 2025

felixarntz commented on Feb 25, 2025

westonruter commented on Feb 25, 2025

felixarntz commented on Feb 25, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions