WikimediaDebugComponent
ActivePublic
Watch Project

Members

This project does not have any members.
View All

Watchers (1)

hashar (Antoine Musso)
Logistics
View All

Details

Source Repo: https://gerrit.wikimedia.org/g/performance/WikimediaDebug/

Description

WikimediaDebug is a set of tools for debugging and profiling MediaWiki web requests in a production environment. WikimediaDebug can be used through the accompanying browser extension, or from the command-line. Internally, it uses the X-Wikimedia-Debug header (also known as XWD).

Use this tag for issues about:

browser extension, the WikimediaDebug extension for Firefox and Chromium,
XHGui service at https://performance.wikimedia.org/xhgui/, including its Puppet provisioning and the xhgui database in the m2 database cluster.
Excimer UI service at https://performance.wikimedia.org/excimer/speedscope/, including its excimer database in the m2 database cluster.
configuration and profiling client for the above in operations/mediawiki-config.git: (e.g. src/XWikimediaDebug, src/Profiler, and src/PhpAutoPrepend.php).

Maintained by: SRE observability Team and Release-Engineering-Team

Documentation: https://wikitech.wikimedia.org/wiki/WikimediaDebug

For tasks about daily production flame graphs, use Arc-Lamp.
For tasks about Varnish/ATS routing of the X-Wikimedia-Debug header, see Traffic.
For tasks about the mwdebug servers, see serviceops.

Recent Activity
View All

Nov 16 2025

Pppery edited projects for T256039: Prometheus exporter for XHGui, added: Patch-Needs-Improvement; removed Patch-For-Review.

Nov 16 2025, 3:04 AM · Patch-Needs-Improvement, WikimediaDebug, observability

Oct 30 2025

MSantos closed T182827: Support X-Wikimedia-Debug header for services as Declined.

@A_smart_kitten thanks for tagging us.

Oct 30 2025, 2:24 PM · Content-Transform-Team, WikimediaDebug, Mobile-Content-Service

Oct 26 2025

A_smart_kitten reopened T182827: Support X-Wikimedia-Debug header for services as "Open".

CTT: tagging you as per decline note. This seems like a more general request than one that's specific to the now-decommissioned Mobile-Content-Service?

Oct 26 2025, 11:34 AM · Content-Transform-Team, WikimediaDebug, Mobile-Content-Service

Sep 20 2025

ori closed T391516: https://performance.wikimedia.org/php-profiling/ leads to 404 for all listed sources as Resolved.

I cleaned up the residual instances manually on arclamp1001 and arclamp2001.

Sep 20 2025, 4:09 PM · SRE Observability (FY2025/2026-Q1), Regression, observability, Arc-Lamp, WikimediaDebug

Sep 10 2025

aaron moved T354042: WikimediaDebug features (e.g. logging and profiling) for MediaWiki JobQueue jobs from Incoming (Needs Triage) to Needs Further Discussion on the MW-Interfaces-Team board.

Sep 10 2025, 5:44 AM · WikimediaDebug, MW-Interfaces-Team, WMF-JobQueue

Sep 9 2025

Seddon moved T403824: Add WikimediaDebug functionality to the Wikimedia Android app from Needs Triage to Up Next on the Wikipedia-Android-App-Backlog board.

Sep 9 2025, 4:12 PM · Wikipedia-Android-App-Backlog, WikimediaDebug

Seddon triaged T403824: Add WikimediaDebug functionality to the Wikimedia Android app as Medium priority.

Sep 9 2025, 4:12 PM · Wikipedia-Android-App-Backlog, WikimediaDebug

Sep 7 2025

A_smart_kitten added a project to T354042: WikimediaDebug features (e.g. logging and profiling) for MediaWiki JobQueue jobs: WikimediaDebug.

Sep 7 2025, 7:43 AM · WikimediaDebug, MW-Interfaces-Team, WMF-JobQueue

Sep 5 2025

Tgr updated the task description for T403824: Add WikimediaDebug functionality to the Wikimedia Android app.

Sep 5 2025, 1:43 PM · Wikipedia-Android-App-Backlog, WikimediaDebug

Tgr created T403824: Add WikimediaDebug functionality to the Wikimedia Android app.

Sep 5 2025, 1:42 PM · Wikipedia-Android-App-Backlog, WikimediaDebug

Sep 2 2025

Maintenance_bot removed a project from T267248: create mwdebug1003 - ganeti VM with buster and appserver role: Patch-For-Review.

Sep 2 2025, 6:30 PM · Performance-Team, WikimediaDebug, serviceops

Aug 25 2025

Aklapper closed T182827: Support X-Wikimedia-Debug header for services as Declined.

As part of the decommission of Mobile-Content-Service, this task is no longer valid. See T328036 for reference.
If you believe this is reproducible as part of Page Content Service nowadays used, please reach out to Content-Transform-Team. Thanks.

Aug 25 2025, 11:47 AM · Content-Transform-Team, WikimediaDebug, Mobile-Content-Service

Aug 20 2025

Reedy added a comment to T315111: Make WikimediaDebug status more sticky.

With the rollout of PHP 8.3... Especially in these very early stages, I'd love to be able to stick my activity onto the PHP 8.3 hosts, and frankly leave it there for the near term future.

Aug 20 2025, 9:27 PM · WikimediaDebug

Aug 12 2025

Aklapper removed Impacted on T340713: Seeking Advice and Assistance: Upgrading Debian Buster XHGui Instances.

Aug 12 2025, 3:47 PM · WikimediaDebug, Performance-Team, Security

Aug 5 2025

Maintenance_bot removed a project from T401152: Switch wmf-config/Profiler from Tideways to XHProf: Patch-For-Review.

Aug 5 2025, 7:31 PM · MediaWiki-Platform-Team, serviceops, WikimediaDebug, MediaWiki-Core-Profiler

Krinkle added a parent task for T401152: Switch wmf-config/Profiler from Tideways to XHProf: T398245: Prepare WMF PHP 8.3 packages for bullseye.

Aug 5 2025, 7:03 PM · MediaWiki-Platform-Team, serviceops, WikimediaDebug, MediaWiki-Core-Profiler

Krinkle removed a subtask for T401152: Switch wmf-config/Profiler from Tideways to XHProf: T399884: Configure production MediaWiki image builds for PHP 8.3.

Aug 5 2025, 7:03 PM · MediaWiki-Platform-Team, serviceops, WikimediaDebug, MediaWiki-Core-Profiler

Krinkle added a subtask for T401152: Switch wmf-config/Profiler from Tideways to XHProf: T399884: Configure production MediaWiki image builds for PHP 8.3.

Aug 5 2025, 7:02 PM · MediaWiki-Platform-Team, serviceops, WikimediaDebug, MediaWiki-Core-Profiler

Krinkle closed T401152: Switch wmf-config/Profiler from Tideways to XHProf as Resolved.

Aug 5 2025, 7:01 PM · MediaWiki-Platform-Team, serviceops, WikimediaDebug, MediaWiki-Core-Profiler

Stashbot added a comment to T401152: Switch wmf-config/Profiler from Tideways to XHProf.

Mentioned in SAL (#wikimedia-operations) [2025-08-05T19:01:11Z] <krinkle@deploy1003> Finished scap sync-world: Backport for [[gerrit:1175621|Profiler: Remove support for php-tideways_xhprof (T401152)]] (duration: 14m 54s)

Aug 5 2025, 7:01 PM · MediaWiki-Platform-Team, serviceops, WikimediaDebug, MediaWiki-Core-Profiler

Stashbot added a comment to T401152: Switch wmf-config/Profiler from Tideways to XHProf.

Mentioned in SAL (#wikimedia-operations) [2025-08-05T18:48:04Z] <krinkle@deploy1003> krinkle: Backport for [[gerrit:1175621|Profiler: Remove support for php-tideways_xhprof (T401152)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Aug 5 2025, 6:48 PM · MediaWiki-Platform-Team, serviceops, WikimediaDebug, MediaWiki-Core-Profiler

Stashbot added a comment to T401152: Switch wmf-config/Profiler from Tideways to XHProf.

Mentioned in SAL (#wikimedia-operations) [2025-08-05T18:46:17Z] <krinkle@deploy1003> Started scap sync-world: Backport for [[gerrit:1175621|Profiler: Remove support for php-tideways_xhprof (T401152)]]

Aug 5 2025, 6:46 PM · MediaWiki-Platform-Team, serviceops, WikimediaDebug, MediaWiki-Core-Profiler

gerritbot added a comment to T401152: Switch wmf-config/Profiler from Tideways to XHProf.

Change #1175621 merged by jenkins-bot:

[operations/mediawiki-config@master] Profiler: Remove support for php-tideways_xhprof

https://gerrit.wikimedia.org/r/1175621

Aug 5 2025, 6:46 PM · MediaWiki-Platform-Team, serviceops, WikimediaDebug, MediaWiki-Core-Profiler

gerritbot added a comment to T401152: Switch wmf-config/Profiler from Tideways to XHProf.

Change #1175623 merged by Scott French:

[operations/puppet@production] mediawiki: install php8.1-xhprof on beta cluster and mwdebug

https://gerrit.wikimedia.org/r/1175623

Aug 5 2025, 5:40 PM · MediaWiki-Platform-Team, serviceops, WikimediaDebug, MediaWiki-Core-Profiler

Stashbot added a comment to T401152: Switch wmf-config/Profiler from Tideways to XHProf.

Mentioned in SAL (#wikimedia-operations) [2025-08-05T17:28:43Z] <swfrench@deploy1003> Finished scap sync-world: Migrate debug and cli images to xhprof - T401152 (duration: 22m 02s)

Aug 5 2025, 5:28 PM · MediaWiki-Platform-Team, serviceops, WikimediaDebug, MediaWiki-Core-Profiler

Stashbot added a comment to T401152: Switch wmf-config/Profiler from Tideways to XHProf.

Mentioned in SAL (#wikimedia-operations) [2025-08-05T17:08:40Z] <swfrench@deploy1003> swfrench: Migrate debug and cli images to xhprof - T401152 synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Aug 5 2025, 5:08 PM · MediaWiki-Platform-Team, serviceops, WikimediaDebug, MediaWiki-Core-Profiler

Stashbot added a comment to T401152: Switch wmf-config/Profiler from Tideways to XHProf.

Mentioned in SAL (#wikimedia-operations) [2025-08-05T17:07:02Z] <swfrench@deploy1003> Started scap sync-world: Migrate debug and cli images to xhprof - T401152

Aug 5 2025, 5:07 PM · MediaWiki-Platform-Team, serviceops, WikimediaDebug, MediaWiki-Core-Profiler

CodeReviewBot added a comment to T401152: Switch wmf-config/Profiler from Tideways to XHProf.

make-container-image: switch tideways to xhprof

Aug 5 2025, 5:06 PM · MediaWiki-Platform-Team, serviceops, WikimediaDebug, MediaWiki-Core-Profiler

Stashbot added a comment to T401152: Switch wmf-config/Profiler from Tideways to XHProf.

Mentioned in SAL (#wikimedia-operations) [2025-08-05T17:05:09Z] <krinkle@deploy1003> Finished scap sync-world: Backport for [[gerrit:1175620|Profiler: Add php-xhprof support besides php-tideways_xhprof (T401152)]] (duration: 11m 15s)

Aug 5 2025, 5:05 PM · MediaWiki-Platform-Team, serviceops, WikimediaDebug, MediaWiki-Core-Profiler

Krinkle added a comment to T401152: Switch wmf-config/Profiler from Tideways to XHProf.

Apply https://gerrit.wikimedia.org/r/c/1175623 (Puppet) in Beta Cluster. Which uninstalls php8.1-tideways_xhprof, and installs php8.1-xhprof. At this point, WikimediaDebug with XHGui is a no-op (the permalink at performance.wikimedia.beta.wmcloud.org/xhgui shows no results).
Initiate scap backport for https://gerrit.wikimedia.org/r/1175620 (wmf-config).
- Once merged, git pull in krinkle@deployment-deploy04:/srv/mediawiki-staging, scap pull on krinkle@deployment-mediawiki14, and confirm krinkle@deployment-mediawiki14:~$ cat /srv/mediawiki/src/Profiler.php | grep xhprof.
- Verify that WikimediaDebug now works again in Beta Cluster, using php-xhprof.
Once the scap backport is on testserver, verify via mwdebug in prod that WikimediaDebug/XHGui still works there via php-tideways_xhprof.

Aug 5 2025, 5:04 PM · MediaWiki-Platform-Team, serviceops, WikimediaDebug, MediaWiki-Core-Profiler

Stashbot added a comment to T401152: Switch wmf-config/Profiler from Tideways to XHProf.

Mentioned in SAL (#wikimedia-operations) [2025-08-05T16:55:40Z] <krinkle@deploy1003> krinkle: Backport for [[gerrit:1175620|Profiler: Add php-xhprof support besides php-tideways_xhprof (T401152)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Aug 5 2025, 4:55 PM · MediaWiki-Platform-Team, serviceops, WikimediaDebug, MediaWiki-Core-Profiler

Stashbot added a comment to T401152: Switch wmf-config/Profiler from Tideways to XHProf.

Mentioned in SAL (#wikimedia-operations) [2025-08-05T16:53:54Z] <krinkle@deploy1003> Started scap sync-world: Backport for [[gerrit:1175620|Profiler: Add php-xhprof support besides php-tideways_xhprof (T401152)]]

Aug 5 2025, 4:54 PM · MediaWiki-Platform-Team, serviceops, WikimediaDebug, MediaWiki-Core-Profiler

gerritbot added a comment to T401152: Switch wmf-config/Profiler from Tideways to XHProf.

Change #1175620 merged by jenkins-bot:

[operations/mediawiki-config@master] Profiler: Add php-xhprof support besides php-tideways_xhprof

https://gerrit.wikimedia.org/r/1175620

Aug 5 2025, 4:53 PM · MediaWiki-Platform-Team, serviceops, WikimediaDebug, MediaWiki-Core-Profiler

CodeReviewBot added a comment to T401152: Switch wmf-config/Profiler from Tideways to XHProf.

swfrench opened https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/200

Aug 5 2025, 3:29 PM · MediaWiki-Platform-Team, serviceops, WikimediaDebug, MediaWiki-Core-Profiler

gerritbot added a comment to T401152: Switch wmf-config/Profiler from Tideways to XHProf.

Change #1175623 had a related patch set uploaded (by Krinkle; author: Krinkle):

[operations/puppet@production] mediawiki: install php8.1-xhprof

https://gerrit.wikimedia.org/r/1175623

Aug 5 2025, 1:22 AM · MediaWiki-Platform-Team, serviceops, WikimediaDebug, MediaWiki-Core-Profiler

gerritbot added a comment to T401152: Switch wmf-config/Profiler from Tideways to XHProf.

Change #1175621 had a related patch set uploaded (by Krinkle; author: Krinkle):

[operations/mediawiki-config@master] Profiler: Remove support for php-tideways_xhprof

https://gerrit.wikimedia.org/r/1175621

Aug 5 2025, 1:06 AM · MediaWiki-Platform-Team, serviceops, WikimediaDebug, MediaWiki-Core-Profiler

gerritbot added a project to T401152: Switch wmf-config/Profiler from Tideways to XHProf: Patch-For-Review.

Aug 5 2025, 1:06 AM · MediaWiki-Platform-Team, serviceops, WikimediaDebug, MediaWiki-Core-Profiler

gerritbot added a comment to T401152: Switch wmf-config/Profiler from Tideways to XHProf.

Change #1175620 had a related patch set uploaded (by Krinkle; author: Krinkle):

[operations/mediawiki-config@master] Profiler: Support php-xhprof besides php-tideways-xhprof

https://gerrit.wikimedia.org/r/1175620

Aug 5 2025, 1:06 AM · MediaWiki-Platform-Team, serviceops, WikimediaDebug, MediaWiki-Core-Profiler

Aug 4 2025

Scott_French added a comment to T401152: Switch wmf-config/Profiler from Tideways to XHProf.

Thanks for filing this @Krinkle.

Aug 4 2025, 9:52 PM · MediaWiki-Platform-Team, serviceops, WikimediaDebug, MediaWiki-Core-Profiler

Krinkle added a project to T401152: Switch wmf-config/Profiler from Tideways to XHProf: MediaWiki-Platform-Team.

Aug 4 2025, 9:19 PM · MediaWiki-Platform-Team, serviceops, WikimediaDebug, MediaWiki-Core-Profiler

Krinkle updated the task description for T401152: Switch wmf-config/Profiler from Tideways to XHProf.

Aug 4 2025, 9:18 PM · MediaWiki-Platform-Team, serviceops, WikimediaDebug, MediaWiki-Core-Profiler

Krinkle updated the task description for T401152: Switch wmf-config/Profiler from Tideways to XHProf.

Aug 4 2025, 9:09 PM · MediaWiki-Platform-Team, serviceops, WikimediaDebug, MediaWiki-Core-Profiler

Krinkle updated the task description for T401152: Switch wmf-config/Profiler from Tideways to XHProf.

Aug 4 2025, 9:09 PM · MediaWiki-Platform-Team, serviceops, WikimediaDebug, MediaWiki-Core-Profiler

Krinkle created T401152: Switch wmf-config/Profiler from Tideways to XHProf.

Aug 4 2025, 9:09 PM · MediaWiki-Platform-Team, serviceops, WikimediaDebug, MediaWiki-Core-Profiler

Aug 1 2025

Tgr closed T397439: X-Wikimedia-Debug cookie not routed correctly in Kubernetes on POST requests as Resolved.

Thanks!

Aug 1 2025, 12:01 AM · Traffic, MediaWiki-Platform-Team, WikimediaDebug

Jul 31 2025

Krinkle added a comment to T397439: X-Wikimedia-Debug cookie not routed correctly in Kubernetes on POST requests.

wikimedia-sre

<sukhe> Krinkle: I can simply merge that one if that's fine.
<sukhe> (doing now)
<Krinkle> Thx!
…
<sukhe> Krinkle: all done.
<Krinkle> sukhe: ack. works as expected.

Jul 31 2025, 5:36 PM · Traffic, MediaWiki-Platform-Team, WikimediaDebug

Krinkle assigned T397439: X-Wikimedia-Debug cookie not routed correctly in Kubernetes on POST requests to Tgr.

Jul 31 2025, 5:35 PM · Traffic, MediaWiki-Platform-Team, WikimediaDebug

Maintenance_bot removed a project from T397439: X-Wikimedia-Debug cookie not routed correctly in Kubernetes on POST requests: Patch-For-Review.

Jul 31 2025, 5:31 PM · Traffic, MediaWiki-Platform-Team, WikimediaDebug

gerritbot added a comment to T397439: X-Wikimedia-Debug cookie not routed correctly in Kubernetes on POST requests.

Change #1174685 merged by Ssingh:

[operations/puppet@production] varnish: Fix X-Wikimedia-Debug cookie VCL rule

https://gerrit.wikimedia.org/r/1174685

Jul 31 2025, 5:07 PM · Traffic, MediaWiki-Platform-Team, WikimediaDebug

Krinkle added a project to T397439: X-Wikimedia-Debug cookie not routed correctly in Kubernetes on POST requests: Traffic.

Jul 31 2025, 4:26 PM · Traffic, MediaWiki-Platform-Team, WikimediaDebug