Takeaway and next step from @KStoller-WMF that I agree with:

Ok, I added an entry to the annotation for Edits and Editors.
fingers crossed it shows up for November 2025 in a few days :) will check back.

hey @GGoncalves-WMF , thanks for that info! I wasnt aware that temp edits and temp editors are being counted under Anonymous. I want to say then that no direct impact from temp accounts project or missclassification other than the fact that this metric could get extra attention because of the rollout.
I looked at Edits hourly which also shows a gradual decrease in anonymous edits since June 2021 (the drop is sharper in the last 2 months because mwh and edits_hourly have separate fields for temp and anonymous). When you add the anonymous and temp edits it checks out with Wikistats.

This is all I can think of for the scope for this task right now, but needs further discussion with DPE and possible changes to AQS - https://wikitech.wikimedia.org/wiki/Data_Platform/Systems/Wikistats_2

Going forward we will get updates from DPE Team on this task about progress on dbt - the new tool we are using to develop a standard for storing metric calculation logic. I will add these updates to my Asana updates every Friday.

thank you @JAllemandou !

@JAllemandou , do we have webrequest data retained from the pageviews backfill? it would be great if we could backfill this field.

Hey everyone we discussed this a bit more and have decided to go with a new category called external (ai_chatbot) to include some of the top chabot referrers. In addition I’ve proposed a few changes and additions to existing categories.

hey @JAllemandou/ @Ahoelzl , Ive chimed in here T383088#11308163 about the modification to referer - since most of it is being tagged as automated there is low impact to user pageviews currently.

+1 to @Isaac . I re ran the query that @Krinkle ran using pageview_actor for a day in Oct and Sep 2025. Most of the requests are now tagged as 'automated'. I think this could be a result of the new heuristic we put into place in T395934.

1. the proportion of non https requests are much smaller than the ones tagged correctly as search_engine referer
2. of the ones tagged as external or unknown, majority are automated ie user pageviews arent impacted
3. also, looking at browser_family, the automated requests are mostly coming from 'Chrome' or 'Other' browser family. Firefox isnt dominating this anymore.

Iteration 1 of Draft Dashboard - https://superset.wikimedia.org/superset/dashboard/p/eNLvg06Bgl5/

@JAllemandou , +1 on changing the IP referer to unknown.

hey @Isaac, more and more search engines are integrating conversational AI features into the search experience, and yes, I agree that the boundaries are getting blurry. so the way I see it, it means we may not need a new classifier for chat agents. Traditional chat agents like ChatGPT would fit into the search engine category, and even more now since the launch of Atlas.
May be some time in the future we may just have to rename or modify the definition of search engine referer to accommodate this nuance.

Hi Everyone, thank you for your inputs! this is helpful as we begin to think about upgrading the referer column. However, this should be done in phases as we can make many improvements and I see quite a few suggestions in this task. I'll try to curate them in a document and share soon.

week of Oct 10 - OOO - no update

hmm, apparently this was done as a part of T334104#8759879, but I wonder if we encountered a bug later and the data stopped loading from hive to druid?

Oct 3, 2025
Progress update for the week

• Met with MI team and Omari to review Contributors dashboard and continue iterating
• Contributors dashboard prototype with 4 MVP metrics is available here - https://superset.wikimedia.org/superset/dashboard/p/xn8BnA4vzYy/
• Global trends mid year check in - discussed sharing with P&T and reached out to some collaborators
• August metrics: Shared the monthly report and provided retained newcomer data for Commons & Wikidata split to Marshall

We continue to see these trends in August as well.
Wikidata seems to be pulling most of the weight in the new active editor growth in July and Aug 2025. here's why -
in July 2025, Wikidata accounted for more than half the growth in new active ed increases; and in Aug 2025 Wikidata increases were more than the overall new active editor increases.

speaking for our needs, I think that's totally fine! :)

Progress update for the week

• Prepared and Presented the Mid year global trends to department leadership
• Created the Contributors Metric dashboard T403305
  • Added retained newcomers and activations data
• Did some WE5 metric modeling with Jaime
• Met with Comms team for updating the public comms stats

Backfill is happening in T405667

Sep 19, 2025
Progress update for the week

• Prototyped the Contributors metrics on Superset - Retained Editors by wiki and access method (example chart), and Account Registrations by wiki and access method (sample chart) T403305
• Met with Andreas to request prioritzing DBT implementation; and met with Fundraising team to learn about their experience with DBT in FR-analytics
• Met with Jaime to continue building a measurement strategy for WE 5 Infrastructure Re use objective

@Hghani is going to add the 3rd category for all charts that had the Emerging vs Developed split in the August run.
good luck @Hghani :D

thank you @mpopov ! any action item for Movement Insights here?

Sep 12, 2025
Progress update for the week

• Published July metrics report T402060
• Met with product managers to gather requirements and made user stories. Started scripting first few simple queries to test out T403305
• Met with Guilherme to consider using DBT T403305
• Discussed with the team about adding new category for wiki project T404401

Done. https://analytics.wikimedia.org/published/reports/movement-metrics/archive/2025-07.html

Conducted Requirement gathering session with PMs and Stakeholders this week. and built user stories.
Next up meet with Andreas and Guilherme to request implementation with DBT.
While also working on the manual creation of tables and visualizing them on Superset.

Im re thinking this.. can we add the charts for referral source and regional active editors ? I feel they will be useful. we are getting questions on referal source again now, and regional split of active editors could be useful for the contributors strategy.

In the coming week, we'll rollout to test.wikipedia.org (T401595).
It is fine for testwiki if it isn't live yet, and we will of course wait for this and T401666 to resolve before the next phase of rollouts. I am asking so that, if it happens to be live already, I can do some testing in the hour after the switch on pageview stats. Otherwise I will do that at another time.

This should ne problem for webrequest/pageview, but will impact unique-devices. As long as @maya, @nshahquinn-wmf and @Hghani are ok, I'm ok :)

Sep 5, 2025
Progress update for the week

• Generated July metrics T402060
• Did Stakeholder mapping with Jaime and Megan; and planned for PM meeting for Contributors Metrics - Reporting and Operationalization T403305
• Did a Brainstorming session for Movement metrics report improvements and collected a good backlog of ideas and improvements T369225

Aug 29, 2025
Progress update for the week

• Deprecated superset metrics dashboards T359700
• Created brainstorming plan for Movement metrics reporting improvements T369225
• Analyzed test results with Hamid and Approved reader metric fix for deployment T395934
• Analyzed access_method and domain removal changes for unique devices T401666

hey @JAllemandou , unless there is a bigger Foundation wide initiative to get rid of Turnilo I don't think we should remove this dataset from druid.
As much as I agree on the efficiency use case we are aware of many users (and who are part of Leadership team) that use Turnilo regularly. We would want this data to be available in Turnilo and Druid.

Are you ok with us taking advantage of this breaking change to remove both the druid datasource and the hive datasource, keeping only the iceberg one? The idea is to simplify by removing datasources that are the same :)

I will be meeting with Movement insights team to brainstorm improvements to metric reporting.

The 3 dashboards are now deleted. Amen

No longer needed because we will T359700

Knowledge gap metrics are available now
We will adding new Contributor pipeline metrics -

• Experienced Active Editors,
• Retained Newcomers
• the new Core metric for the Contributors’ strategy, Retained Editors

Moving it back to Doing as I'm working on deprecating the dashboards and preserving the queries.

Post production: We have 63 unique visitors to the movement metrics folder! And close to 50 who accessed (i.e. read) the current month report.

Aug 22, 2025
Progress update for the week

• Started Experiment with improving Data_glossary page and creating mock-ups T402576
• Calculated unique visitors to the new Quarto report
• Analyzed test data for Reader metric fix T395934
• Closed out haproxy data loss impact T401246