Skip to content

Add comment value / relevance to Comment Moderation experiment #514

Description

@jeffpaul

What problem does this address?

The new Comment Moderation experiment introduced sentiment analysis and toxicity for comments, along with extensibility hooks around the analysis pipeline. During review discussion in #155, additional interest emerged around evaluating the overall value or relevance of a comment, not just whether it is toxic or negative. This could help identify low-value comments such as:

  • spammy or engagement-bait replies
  • off-topic responses
  • generic “thanks” / "+1" / low-context comments
  • comments that do not meaningfully contribute to discussion

In being able to detect those types of comments, it could support functions like:

  • moderation workflows
  • prioritization for community managers
  • surfacing high-quality discussion
  • future automation around comment handling

What is your proposed solution?

Update the Comment Moderation experiment to add a "Comment Value / Relevance" analysis feature that builds on the existing comment analysis infrastructure. Related discussion in #155 noted that some extensibility hooks already exist around system instructions and response schema, though additional JS extensibility may be needed for the full admin experience.

Potential capabilities:

  • Analyze comments for relevance and contribution quality
  • Store a normalized “value” or “relevance” score
  • Add a dedicated column in the Comments screen and pill on the Activity dashboard widget
  • Allow bulk processing similar to sentiment/toxicity analysis
  • Optionally flag low-value comments for moderation or review
  • Provide filters/hooks for custom scoring logic and thresholds

Activity

  1. added this to the Future Release milestone on May 6, 2026
  2. self-assigned this
    on May 6, 2026
  3. moved this from To do to In progress in WordPress AI Roadmapon Jun 8, 2026
  4. modified the milestones: Future Release, 1.2.0 on Jun 26, 2026
  5. modified the milestones: 1.2.0, 1.3.0 on Jul 13, 2026
  6. moved this from In progress to Needs review in WordPress AI Roadmapon Aug 10, 2026
  7. modified the milestones: 1.3.0, 1.4.0 on Aug 13, 2026
  8. moved this from Needs review to Done in WordPress AI Roadmapon Sep 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions