Included in estimation:

• drafting communications
• coordinating translations
• coordinating with ML on timeline (ideally we can agree on a tentative release for larger wikis)

We have several ways of considering (and reshuffling) the groups @OKarakaya-WMF defined in T404460#11200848.

Our first goal is to check if the model works well by collecting enough data. That data would help us compare the new model to the previous A/B tests we already conducted. This was already identified:

In T404460#11200848, @OKarakaya-WMF wrote:

The advantage of releasing small wikis first is that it will have less impact if something goes wrong.
The disadvantage is that it's easier to figure out problems on large wikis as they are used more.

My advice is to go with the larger wikis of the list (Chinese, Japanese, and Urdu) as a first phase. Testing at a smaller wiki would go nowhere, as they have a very limited number of daily edits.

We can start with a very limited release at these wikis to see if the model is not suggesting strange things, and after a week of test, we start the actual 50/50 test.

My suggestion is:
Round 1 with Chinese, Japanese, and Urdu:

1. Announce the test; let communities react (1 week)
2. Start the initial test, to check how it goes - 10% of users (1 week)
3. Start the A/B test (duration to be determined)
4. Release

Having comparison data would show if the model is good enough to replace the v1.

For the remaining wikis, as Round 2, I would start oafter Round 1's step 3:

1. Announce the deployment; let communities react (1 week)
2. Release

We can subdivide round 2 into sub-rounds, if needed. But given the size of the wikis, it is not necessary.

Thoughts?

ping @KStoller-WMF FYI

After a discussion with Özge, I updated the task description.

I added a step for Community Configuration: for the larger wikis, we should work with them to define a “List of excluded sections” for the Add a link task.
For the remaining wikis, we may need to continue work on: T345562: Generate excludedSections list for wikis missing in wiki_sections.jsonl.

One aspect of this plan that might need to change:
I don’t think we’ll be able to get product analytics support to run another A/B test for Add a link. We’ve already conducted three separate A/B tests for this task, and at this stage, it would be more practical to release it (potentially with more conservative Community Configuration limits) and perform a pre- and post-release analysis instead: T407455: Growth Team: monitor leading indicators for “Add a Link”.
We can still closely monitor task completion metrics, revert rates, and overall constructive edit rates, and communities will retain the ability to adjust or disable the task if needed.

While an A/B test would be preferable, it remains very time-consuming to complete the analysis and requires additional engineering effort to set up. Given our current constraints, I don’t think that approach is feasible for this iteration.

We will remove the 10% test part as:

• It won't show many edits to have a proper test
• We don't have the infrastructure to do it anymore.

Community configuration keeps the possibility of changing some thresholds if needed.

Keeping this task open we hve the second round of wikis to cover. The process is documented in T410469: Add a Link: Rollout "Add a Link" task to remaining Wikipedias that have V2 model support but don't yet have access to "Add a Link".