Jump to content

Edit check/Paste Check

Issue tracker: #EditCheck
From mediawiki.org
Translate this page
Languages:
Tracked in Phabricator
Task T359107
Wikimedia Foundation projects
Paste check
Prompt newcomers to consider whether the text they're pasting risks a copyright violation.
Group: Editing
Start: 2024-11-01
Team members: David Chan, David Lynch, Esther Akinloose, Marielle Volz, Rummana Yasmeen, Nicolas Ayoub, Benoît Evellin, Ed Sanders, Zoe, Caro Medelius
Backlog: #EditCheck
Management: Peter Pelberg (product), Valerie Puffet-Michel (engineering)

Paste Check is an Edit check that will appear when people paste text into an article they are likely not to have written.

This Check is an effort to increase the likelihood that the new content people are adding to Wikipedia is aligned with the Movement's commitment to offering information under a free content license.

To participate in and follow this project's development, we recommend adding this page to your watchlist.

Paste Check, like the broader Edit Check project, continues to be inspired by volunteer wishes and tools.[1][2]

Status

[edit]

Feedback needed

User experience: the team would value learning what you think the of proposed Paste Check user experience. Learn more here.

Current work

Instrumentation: the team is implementing the instrumentation needed to evaluate the impact of Paste Check through a controlled and time-bound experiment that is scheduled to start in or before September 2025.

Prevalence: to help evaluate the impact of Paste Check, the team is gathering data in order to estimate how often newcomers edits are reverted on the basis of WP:COPYVIO (and related policies).

Objectives

[edit]

Paste Check is meant to simultaneously:

  1. Cause newer volunteers acting in good faith to contribute content to Wikipedia that complies with the Movement's free content license.
  2. Increase the ease with which experienced volunteers can identify and patrol edits that could violate this free license.

Challenges

[edit]

Offering, "...free content that anyone cause use, edit, and distribute..." is a fundamental principle of Wikipedia.

Although, a significant portion of newcomers[3] are unaware of this content policy and, those that are aware of it, become confused about how to apply it.

In conversations with volunteers practiced with mentoring newcomers, we also learned that some new editors mistakenly assume that pasting content from – what they consider to be – a reliable source is an appropriate way to comply with WP:Verifiability.

Theory of Change

[edit]

Paste Check is grounded in the following hypothesis:

If we prompt new(er) volunteers pasting text from an external site to confirm whether they wrote the content they are attempting to add, then we will see a ≥4% decrease in the percentage of new content edits new(er) volunteers publish that are reverted on the grounds of WP:COPYVIO (and related policies).

Design

[edit]
Screenshot showing the Paste Check mobile user experience
Paste Check mobile user experience

Paste Check surfaces a prompt to people when they paste text into the visual editor.

The prompt offers context about why it is important for editors to consider the origin of content. It also includes information about the consequences of publishing text you do not have sufficient rights to use on to Wikipedia.

The prompt also asks that people indicate whether they have written the text they are pasting.

Diagram showing the proposed Paste Check user experience for September 2025 A/B test.
Proposed Paste Check user experience.

Evaluating impact

[edit]

The viability of Paste Check, like the broader Edit Check project, depends on the feature being able to simultaneously:

  1. Reduce the moderation workload experienced volunteers carry
  2. Increase the rate at which newcomers publish constructive edits.

To evaluate the extent to which Paste Check is effective at the above, the team will be conducting qualitative and quantitative experiments. See below for more information about both.

A/B Experiment

[edit]
Visual of Paste Check experiment design
Paste Check experiment design

Conclusions

[edit]

A 2025 analysis of Paste Check’s A/B experiment across 22 partner Wikipedias showed:

  1. Signs of reducing the new content revert rate and increasing the constructive edit rate. This finding is not statistically significant.
  2. The feature did not cause any meaningful regressions in the guardrail metrics we were monitoring. This finding is statistically significant.

Specifically, edits shown Paste Check in the test group were:

  • Reverted less frequently (-18% relative decrease) than edits in the control group, across desktop and mobile.
  • More likely to be constructive (2% relative increase), across desktop and mobile.

While these overall trends appear positive, per the above, the effect size was too small to confirm the statistical significance of these changes, both on an aggregate (desktop + mobile) and per platform basis, with the data available at the time of this analysis. This, combined with statistically significant evidence that Paste Check is not causing disruption (e.g. no increases in reverts or blocks and no decreases in edit completion rate), is causing us to move forward with scaling Paste Check to all Wikipedias. See Scenario 5 in T399669.

Bar graph showing Paste Check new content edit revert rates of edits made in the test and control groups
Paste Check experiment new content edit revert rate

Findings

[edit]
  1. New Content Edit Revert Rate
    1. We observed a -18% relative decrease [10.5% → 8.6%] in the revert rate for edits shown Paste Check compared to edits eligible but not shown Paste Check.
      1. The analysis indicates there is an 85.6% probability that edits shown Paste Check are reverted less frequently than eligible edits in the control group. While this falls short of the 95% certainty threshold, it’s a strong indicator that the feature is decreasing reverts.
    2. Trends differ by platform. For desktop, we observed a -27% decrease [2.6 percentage points; 9.6% → 7%] in the revert rate of edits shown Paste Check. While on mobile web, there was a +20% increase [3.1 percentage points; 15.5% → 18.6%].
      1. We are unable to confirm if either of these effects are statistically significant.
  2. Constructive Edit Rate
    1. Constructive edits increased by +2% for users in the test group shown Paste Check
      1. The analysis indicates there is an 85.6% probability that edits shown Paste Check are more constructive than eligible edits in the control group. While this falls short of the 95% certainty threshold, it’s a strong indicator that the feature is increasing constructive edits overall.
    2. Trends differ by platform. On desktop, we observed a +2.9% [90.4% → 93%] increase in constructive edit rate for edits shown Paste Check. While on mobile web, there was -3.7% decrease [84.5% → 81.4%].
      1. Both of these results are statistically inconclusive.
  3. Constructive Retention Rate
    1. There were no significant changes in the constructive edit rate for users shown Paste Check.
      1. 6% of contributors in both the test and the control group returned 7 to 14 days to make a constructive edit after making an edit where Paste Check was shown and eligible to be shown.
  4. Edit completion rate
    1. Paste check did not cause any significant changes in edit completion rate on either desktop or mobile web. Overall, we observed a -1.9% decrease [1.2 percentage points] in the completion rate of edits shown Paste Check.
  5. Guardrail metrics
    1. Edit completion rate: no significant decreases in edit completion rate.
    2. Revert rate: no significant decreases in revert rate.
    3. Check dismissal rates: people elected to keep the text they pasted in 56% of edits shown in Paste Check. This aligns with the Edit Check dismissal rate observed for Tone Check and slightly lower than rates observed for Reference Check.
    4. Block rate: people are not blocked at a higher rate after being shown Paste Check.

Experiment scope and design

[edit]
  • Wikis: 22 partner Wikipedias
  • Timing: 8 October – 28 November 2025
  • Participants: Unregistered users and registered editors with ≤100 edits
  • Platform(s): Desktop and mobile web
  • Primary metric: New-content edit revert rate (48 hours)
  • Secondary metrics: Constructive edit rate
  • Guardrail metrics: Completion rate, blocks, dismissal behavior

Leading indicators

[edit]

On 9 October 2025, an A/B experiment of Paste Check began at 22 Wikipedias. What follows, is an analysis of test events logged between 9 October 2025 and 22 October 2025. This analysis was meant to enable the team to decide the following: What – if any – adjustments/investigations will we prioritize for us to be confident moving forward with evaluating the Paste Check's impact?

Note: the findings that follow are not statistically significant. We expect to be able to share statistically significant conclusions by January 2026 via T399669 .

Findings

  • Activation frequency
    • Overall: Paste Check was shown at least once at 36% of all published new content edits by newer editors in the test group.
      • For reference, this is significantly higher than rates observed for Tone Check where 9% of all published new content edits were shown Tone Check.
    • Platform: a higher proportion of published edits on desktop are shown Paste Check (39%) compared to mobile (24%).
    • Experience level: Paste Check appears slightly more frequently for newcomers.
      • We observed a 15% increase in the proportion of published new content edits shown Paste Check when limited to users making their first edit on a Wikipedia.
  • Edit completion
    • Overall: Edits shown Paste Check are completed at a higher rate (52%) than edits in the control group that are eligible but not shown Paste Check (49%). This represents a 6% relative increase.
    • Number of Checks: we currently do not see any increase in edit abandonment rate even if a large number (>3) Paste Checks are shown in a single session.
    • Platform: there was a 15% increase (6 percentage points) in edit completion for edits where Paste Check was shown on mobile and a 4% increase (2 percentage points) on desktop.
    • Experience level: edit completion rate increased across all user experience types to differing degrees.
      • There was an 11% increase in edit completion rate for unregistered users and a 2.6% increase in edit completion rate among Newcomers (registered users making their fist edit).
  • Revert rates
    • Overall: Overall, new content edits shown Paste Check are reverted less frequently.
      • We've observed a -21.3% decrease in published edits where Paste Check was shown compared to edits eligible but not shown Paste Check.
    • Experience level: decreases in revert rate were observed across all reviewed user types (unregistered, newcomers, and Junior Contributors).
    • Platform: While we observed -28% decrease in revert rate among desktop edits in which Paste Check was shown, at the time of reporting, there was not enough data from mobile to share meaningful findings.

Next steps The Editing Team will proceed with the Paste Check A/B experiment without making adjustments to the intervention's user experience or experiment design.

The above is grounded in the fact that:

  1. Paste Check is being shown within a sufficient number of new content edits.
  2. Edits shown Paste Check are completed at a higher rate (52%) than eligible edits not shown Paste Check (49%).
  3. The overall revert rate among edits shown Paste Check are lower (-21.3% decrease) compared to eligible edits not shown Paste Check.

Qualitative evaluation

[edit]

To learn how effective newcomers found the proposed Paste Check user experience to be, the Editing Team conducted a series of unmoderated remote usability tests in August 2025.

10 English-speaking participants across a range of locations, genders, and age groups participated in total. 5 participants tried Paste Check on a desktop device while 5 participants tried Paste Check on a mobile device.

What follows is a summary of what the team learned through these tests and what actions the team prioritized in response.

Usability testing findings

Findings

Overall, test participants found the experience to be simple and clear as evidenced by their ability to complete the Paste Check flow without encountering issues.

People commented on the, "clear layout," "simple design," and "easy-to-understand text". However, they proposed some improvements for the copy, such as using a friendlier tone, clarify some terms, and mention potential consequences for publishing copied content.

However, we detected the following improvements:

  • Participants proposed some improvements for the copy, such as using a friendlier tone, clarify some terms, and mention potential consequences for publishing copied content.
  • One participant was surprised to learn that citing the paste or rephrasing it text did not cause the Paste Check issue to be resolved.

Next steps

In response to this feedback, the team prioritized a series of adjustments to the Paste Check copy.

Quantitative evaluation

[edit]

To evaluate the extent to which Paste Check is effective at causing improvements for newcomers and experienced volunteers, the Editing Team will be running a controlled experiment with a specified start and end date.

Primary Metrics

This table includes the main outcome Paste Check has been designed to cause and the metrics we will use to decide whether it is effective.

Hypothesis Decision(s) to be made Metrics for evaluation
If we prompt newcomers pasting text from an external site to confirm whether they wrote the content they are attempting to add, then we will see a ≥4% decrease in the percentage of new content edits they publish that are reverted on the grounds of WP:COPYVIO (and related policies). Decision A: Does showing people a prompt when pasting text from an external site lower the likelihood that new content edits include copyright violations?

Decision B: Do people intuitively interact with the Paste Check experience in ways that are NOT disruptive to them or the wikis?

1) Proportion of new content edits reverted on the grounds of WP:COPYVIO (and related policies).

2) Proportion of edits started (defined as reaching point that Paste Check was or would be shown) that are successfully published (not reverted).

Secondary metrics

Secondary metrics are used to learn about additional impact of Paste Check, but are not primary targets of the intervention. They reveal side effects (both positive and negative) of trying to improve the Primary Metrics.

ID Hypothesis Metric description
Curiosity #1 A larger proportion of new content edits by newcomers will be constructive because they will be shown a prompt to confirm whether they wrote the content they are attempting to add when pasting text from an external site. Proportion of published edits [i] by users with ≤100 cumulative edits that are constructive [ii]

--- i: We'll need to break edits out by platform as WE 1.1 is scoped to mobile-only.

ii: "Constructive edits" = edits to pages in any Wikipedia main namespace that are not reverted within 48 hours of being published

Curiosity #2 Newcomers will be more aware of the need to consider whether the text they're pasting from an external site into an article is at risk of copyright violations. The proportion of newcomers and Junior Contributors that publish at least one new content edit that was reverted due to copyright violations

Note: We’ll want to observe a decrease in this metric.

Curiosity #3 Newcomers will be more likely to return to publish a new content edit in the future that does not include copyright violations because Paste Check will have caused them to realize when they are at risk of this not being true. 1) Proportion of newcomers that publish an edit Paste Check was activated within and successfully return to make an unreverted edit to an article in the main namespace during the identified retention period.

2) Proportion of newcomers that publish an edit Paste Check was activated within and return to make a new content edit where Paste Check was not shown during the identified retention period.


Guardrails

Guardrail metrics are used to make sure Paste Check is not negatively impacting editors' experiences.

Guardrail Name Metric description
Edit quality decrease Proportion of published edits that add new content and are reverted within 48 hours
Edit completion rate drastically decreases Proportion of edits started (defined as reaching point that Paste Check was or would be shown) that are published.
People shown Paste Check are blocked at higher rates Proportion of contributors blocked after publishing an edit where Paste Check was shown, compared to contributors not shown Paste Check.
High false positive rate Proportion of published edits where a user declined a Paste Check prompt by indicating that it was irrelevant.

Decision matrix

ID Scenario Indicator(s) Plan of Action
1 Paste Check is disrupting, discouraging, or otherwise getting in the way of volunteers. ≥20% drop in edit completion rate in edit sessions where Paste Check is activated relative to edits that would have been shown Paste Check but were not. Pause scaling plans; If results indicate that significant decreases are only associated with a high number of Paste Checks shown, set a threshold for the maximum number checks that can be shown within a single session. If we observe significant decreases for both single and multiple checks presented in a single session, investigate changes to the UX.
2 Paste Check is increasing the likelihood that people will publish destructive edits. Increase in the proportion of published edits where Paste Check was activated that are reverted within 48 hours relative to edits that would have been shown Paste Check but were not. Increase in the proportion of contributors blocked after publishing an edit where Paste Check was shown, compared to contributors who were not shown Paste Check. Pause scaling plans, Review edits to try to identify any patterns in abuse and propose changes to UX to mitigate them.
3 Paste Check is causing people to publish edits that align with project policies Decrease in the proportion of edits Paste Check was activated within that are reverted within 48 hours on the grounds of WP:COPYVIO relative to edits that would have been shown Paste Check but were not. Move forward with scaling plans
4 Paste Check is effective at causing people to publish new content edits without pasted text from external sites, but those edits are still reverted. Increase in the proportion of edits where Paste Check was activated that were published without unmodified pasted text AND increase or no change in the proportion of these edits that are reverted within 48 hours on the grounds of WP:COPYVIO relative to edits that would have been shown Paste Check but were not. Pause scaling plans; Further investigation into methodology used to identify pasted text from an external site (e.g. might the false negative rate be too high); Analysis and manual review of reverted edits to understand why those edits were still reverted.
5 Paste Check is not effective at causing people to publish new content edits without pasted text from external sites but the check is not disrupting to volunteers. No change or decrease in the proportion of new content edits Paste Check was activated within that were published without unmodified pasted text from a non-Wikipedia HTML source AND A) no significant drop in edit completion rate or B) no significant spike in block or revert rates. Move forward with scaling plans

Configurability

[edit]

If the editor has pasted text that we can technically recognize as coming from another MediaWiki visual editor instance, MS Office, Libre Office, Google Docs, or is plain text, the check won't show up.

The initial explaination card offers a link to the Wikipedia Foundation's resolution regarding copyright. This link can be edited for a local help page using MediaWiki:Editcheck-copyvio-descriptionlink.

Strategy

[edit]

This work aligns with the longer-term Contributor Strategy and fits within the WMF 2025-2026 Annual Plan's Contributor Experience Objective, specifically the Wiki Experiences 1.1 Key Result:

Wiki Experiences 1.1: Increase the rate at which editors with ≤100 cumulative edits publish constructive edits on mobile web [i] by 4% [ii], as measured by controlled experiments (by the end of Q2).

---

i. "Constructive edits" = edits to pages in any Wikipedia main namespace that are not reverted within 48 hours of being published.

ii. T389403#10960480

Edit Check

[edit]

Paste Check sits within the larger Edit Check project – an effort to meet people while they are editing with actionable feedback about Wikipedia policies.

Edit Check is intended to simultaneously deliver impact for two key groups of people.

Experienced volunteers who need:

  1. Relief from repairing preventable damage
  2. Capacity to confront complexity

New(er) volunteers who need:

  1. Actionable feedback
  2. Compelling opportunities to contribute
  3. Clarity about what is expected of them

References

[edit]
  1. ↑ Wish: Warn when large amount of content has been copy-pasted
  2. ↑ User:SodiumJS/copying-check.js
  3. ↑ Analysis: What percentage of edits are reverted because of copyvio risk?
Retrieved from "https://www.mediawiki.org/w/index.php?title=Edit_check/Paste_Check&oldid=8216618"