I'm not sure we should re-use the existing database, but rather set up a new one and import the existing records into it. This is the current structure:

+-----------------------------------+
| Tables_in_p50380g50816__pop_stats |
+-----------------------------------+
| pop_Apr11                         |
| pop_Apr12                         |
| pop_Apr13                         |
| pop_Apr14                         |
| pop_Apr15                         |
| pop_Apr16                         |
| pop_Aug11                         |
| pop_Aug12                         |
| pop_Aug13                         |
| pop_Aug14                         |
| pop_Aug15                         |
| pop_Dec11                         |
| pop_Dec12                         |
| pop_Dec13                         |
| pop_Dec14                         |
| pop_Dec15                         |
| pop_Feb11                         |
| pop_Feb12                         |
| pop_Feb13                         |
| pop_Feb14                         |
| pop_Feb15                         |
| pop_Feb16                         |
| pop_Jan11                         |
| pop_Jan12                         |
| pop_Jan13                         |
| pop_Jan14                         |
| pop_Jan15                         |
| pop_Jan16                         |
| pop_Jul11                         |
| pop_Jul12                         |
| pop_Jul13                         |
| pop_Jul14                         |
| pop_Jul15                         |
| pop_Jun11                         |
| pop_Jun12                         |
| pop_Jun13                         |
| pop_Jun14                         |
| pop_Jun15                         |
| pop_Jun16                         |
| pop_Mar11                         |
| pop_Mar12                         |
| pop_Mar13                         |
| pop_Mar14                         |
| pop_Mar15                         |
| pop_Mar16                         |
| pop_May11                         |
| pop_May12                         |
| pop_May13                         |
| pop_May14                         |
| pop_May15                         |
| pop_May16                         |
| pop_Nov11                         |
| pop_Nov12                         |
| pop_Nov13                         |
| pop_Nov14                         |
| pop_Nov15                         |
| pop_Oct10                         |
| pop_Oct11                         |
| pop_Oct12                         |
| pop_Oct13                         |
| pop_Oct14                         |
| pop_Oct15                         |
| pop_Sep11                         |
| pop_Sep12                         |
| pop_Sep13                         |
| pop_Sep14                         |
| pop_Sep15                         |
+-----------------------------------+

And each table looks like:

CREATE TABLE `pop_Sep15` (
  `ns` int(11) NOT NULL,
  `title` varchar(255) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL,
  `hits` int(10) NOT NULL DEFAULT '0',
  `project_assess` text,
  UNIQUE KEY `ns_title` (`ns`,`title`),
  FULLTEXT KEY `project_asssess` (`project_assess`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1 ROW_FORMAT=DYNAMIC

@Niharika you've started building the bot side of things haven't you? How are you handling the database?

I'm not handling the database at all, at the moment. It generates report on the fly and puts it on the wiki page. See https://en.wikipedia.org/wiki/User:NiharikaKohli/Test_popular_pages for example.

Reading more into the task, it seems like the only reason for the web end is to store config information. How about we do it on a wiki page? It should be fairly easy to retrieve config information from a wiki page, which can be protected to prevent abuse.

@Samwilson: We should definitely not re-use the existing pageview stat tables. They aren't accurate and don't include mobile views. It will be easier to just use the monthly pageviews API.

@Niharika: I'm open to the idea of trying an on-wiki config, but it should probably have indefinite auto-confirmed protection. Also we will need to provide good instructions for using it so that people don't break it unintentionally.

An on-wiki config is a neat idea, but how would we check for the existence of a config? You could use templates and look for transclusions, but there are 800+ wikis to check. Also there's not a user-friendly way of indicating any issues to the user, such as when they enter an invalid WikiProject or category, or they made the page size too big. In my opinion a web interface is more intuitive, but it's certainly a lot more work, especially if you want to including auditing like the current system has.

Hmm, there are definitely advantages to either approach. Maybe we should stick with a web interface since that's what people are already used to using and it will be less easy for people to break it by entering bad data. There are two existing config tables we can use:

• s51401__pop_data.project_config - The current config data
• s51401__pop_data.config_history - The history of changes to the config

If we use these, at some point we'll need to add columns for the wikis, e.g. 'enwiki', but we can use them how they are for now.

In T156856#3017977, @MusikAnimal wrote:

An on-wiki config is a neat idea, but how would we check for the existence of a config? You could use templates and look for transclusions, but there are 800+ wikis to check. Also there's not a user-friendly way of indicating any issues to the user, such as when they enter an invalid WikiProject or category, or they made the page size too big. In my opinion a web interface is more intuitive, but it's certainly a lot more work, especially if you want to including auditing like the current system has.

Not sure what you're thinking, but I was imagining something like https://en.wikipedia.org/wiki/User:NKohli_(WMF)/Test1.
It takes care of invalid values because we're asking them to enter links and not plain strings so they'd know something is wrong when they see a red link and we can also indicate this in text above the config. It's straightforward to modify the table and add a new row for a new wiki project.
Why do we need to use templates and look for transclusions? The page could live somewhere like [[Wikipedia:Popular_pages_report/Config]] on each wiki which we can look up when generating reports for the given wiki.
Having it on-wiki is also nicer because we don't have to invent a system to keep track of config history. Wiki does it for you.

All this is not to say I object to a web interface. If that's what's preferred, I'm fine with that.

@czar, @Stevietheman, @Framawiki, @Doc_James: Any opinions on what would be better? An on-wiki configuration interface like https://en.wikipedia.org/wiki/User:NKohli_(WMF)/Test1 or a Tool Labs interface like https://tools.wmflabs.org/popularpages/list.php? (Note that either way, we will reuse the existing config data.)

I appreciate the deference to keeping it the way it is, but the on-wiki config works for me, although if anyone jumps up and down and wants a lookalike to the previous GUI, I won't object as the GUI was well-designed. I think a server-side process for handling errors and perhaps messaging someone who made a mistake in the wiki table is perfectly reasonable. I think those tending to request these pages will be editors with reasonable wiki know-how.

The on-wiki table format looks good to me! That shouldn't be too difficult to parse. The template idea I had was similar to how archiving bots work, where you add it to the page you want the bot to edit, but it makes more sense to keep it centralized. If we wanted to do error reporting I would have the bot add it directly to the config page, rather than message the user who made the mistake. Something simple like "Unable to parse config table" with a link to contact the bot operators. This wiki solution to me seems significantly easier than implementing a web app, and we wouldn't need a database at all.

I think wiki-based config sounds like a better idea. We can get the list of config pages via a Wikidata item, and to start with it's not going to be very many because it's limited to wikis with PageAssessments. And as Stevietheman says, it's mostly going to be table-savvy users editing the config, so breakages should hopefully be rare.

Whichever is easiest—both are fine, but the on-wiki looks the cleanest right now. I imagine the off-wiki site would require more design/maintenance work, so best not to bother with that.

I have a question about the current config. What's the purpose of "Number of pages"?

'Number of pages' is the number of top-viewed pages the wikiproject wants ranked. So if they select 1000, they want to see the Top 1,000 viewed pages from their project. (1,000 was the upper limit in the previous incarnation of this tool)

Thank you, @Stevietheman. Would it simplify things if we defaulted to 1000 for every project without there being a config to change it? It isn't computationally expensive as it was with the prior bot.

Technically, doing that would simplify things, but I don't believe the issue was computational expense. I think some WikiProjects simply wanted shorter lists. I can't speak for all WikiProjects on this, so I don't know if everyone would agree to a top 1,000 list.

On this point, I was hoping to be able to produce a second shorter result for a WikiProject I work on, basically a Top 10 list I can show on the project's front page. Things like this add to the project's public showcase.

In T156856#3022280, @Stevietheman wrote:

On this point, I was hoping to be able to produce a second shorter result for a WikiProject I work on, basically a Top 10 list I can show on the project's front page. Things like this add to the project's public showcase.

Feel free to create a ticket and tag it with Community-Tech. We'll look at it in our next sprint meeting and see what we can do. :)

It sounds like the consensus is to do an on-wiki config for now. Maybe at User:Mr.Z-bot/PopularPagesConfig? It would be good to set it up under User:Mr.Z-bot since that's the user account it will be running under, at least for now (since it already has the Bot approval).

In T156856#3022680, @kaldari wrote:

It sounds like the consensus is to do an on-wiki config for now. Maybe at User:Mr.Z-bot/PopularPagesConfig? It would be good to set it up under User:Mr.Z-bot since that's the user account it will be running under, at least for now (since it already has the Bot approval).

Sounds good, but I still question using the bot account of a more or less retired editor. Per bot policy, "the [bot] account's name should identify the operator or bot function". Users are naturally going to go Mr.Z with questions or concerns, which I don't think is fair. Community_Tech_bot would be more appropriate, but even "PopularPagesBot" or the like would do, paving the way for any maintainer to take control moving forward. If you do some test runs in the bot's userspace, which you can do without approval, and present it at a fresh BRFA, the task should be quickly approved.

@MusikAnimal: That makes sense. It looks like User:PopularPagesBot is available. Let's create a new account and submit it for a BRFA.

In T156856#3022460, @Niharika wrote:

In T156856#3022280, @Stevietheman wrote:

On this point, I was hoping to be able to produce a second shorter result for a WikiProject I work on, basically a Top 10 list I can show on the project's front page. Things like this add to the project's public showcase.

Feel free to create a ticket and tag it with Community-Tech. We'll look at it in our next sprint meeting and see what we can do. :)

Why, when I could use the result of this ticket? Users are accustomed to setting the number of results for their list (up to 1,000), so there shouldn't be any obstruction to me adding a row to cover this top-10 list.

Note that there's no consensus to deviate from having a "number of pages" field, and I would oppose deviating from that. It would be unfair to the many WikiProjects not represented in this discussion, for starters.

Agree we should keep the "number of pages" field.

In T156856#3023656, @Stevietheman wrote:

In T156856#3022460, @Niharika wrote:

In T156856#3022280, @Stevietheman wrote:

On this point, I was hoping to be able to produce a second shorter result for a WikiProject I work on, basically a Top 10 list I can show on the project's front page. Things like this add to the project's public showcase.

Feel free to create a ticket and tag it with Community-Tech. We'll look at it in our next sprint meeting and see what we can do. :)

Why, when I could use the result of this ticket? Users are accustomed to setting the number of results for their list (up to 1,000), so there shouldn't be any obstruction to me adding a row to cover this top-10 list.

Ah, I mistook that to mean that we use the bot to display the Top 10 viewed pages of a wikiproject on the project main page.

@kaldari How about we use Community Tech bot for now since we're the ones maintaining it?

@Niharika: Sounds good to me.

In T156856#3024025, @Niharika wrote:

Ah, I mistook that to mean that we use the bot to display the Top 10 viewed pages of a wikiproject on the project main page.

Yes, the bot sends the results to a subpage of the project's choosing. In this case, I would create a different subpage with its own top/surround formatting, so that subpage could be transcluded into the project's front page. It should really require no changes from what is already proceeding.

I think the only question is one of process. of whether wikiprojects can request more than one popular pages output.

@Stevietheman @czar: I'd like to know your thoughts about having the config page as JSON rather than wikitext.

Advantages:

• It's much easier for the bot to parse it i.e. the bot wouldn't break if someone, say, messed up the table syntax.
• Harder to break it because the editor won't let you save if it's wrong
• The syntax is pretty straightforward
• Hopefully it'll deter vandals
• Can be protected as we like it

Disadvantages:

• Might not be user-friendly for many people
• More difficult for one to know that the wikiproject title/path they entered was correct/incorrect because of absence of red/blue links. (One hacky workaround to this would be that the bot post an error message with wrong links on the talk page/elsewhere periodically)

And just to give an example of what the config might look like:

{
  "Wikipedia:WikiProject Academic Journals": {
    "report": "Wikipedia:WikiProject Academic Journals/Popular pages",
    "category": "Category:Academic Journal articles",
    "limit": 500
  }
}

And if you wanted to add one, just separate the new hash with a comma:

{
  "Wikipedia:WikiProject Academic Journals": {
    "report": "Wikipedia:WikiProject Academic Journals/Popular pages",
    "category": "Category:Academic Journal articles",
    "limit": 500
  },
  "Wikipedia:WikiProject Spiders": {
    "report": "Wikipedia:WikiProject Spiders/Popular pages",
    "category": "Category:Spider articles",
    "limit": 1000
  },
}

The important thing are the stupid commas. You can't have a comma at the end or it will break the bot (or the JSON decoding, specifically). However again if you use the default JSON editor (Special:Preferences > Editing > "Enable enhanced editing toolbar"), it will indicate any errors with the syntax.

When viewing the config it actually will look quite nice, e.g. https://en.wikipedia.org/w/index.php?title=User:MusikAnimal/JSON_sandbox&oldid=765835983

So I'm sort of pushing for JSON too, it's just the best way to have an on-wiki configuration for bots. It's not like the config would be changed that much, so even if there are only a handful of people who understand JSON, they wouldn't be bothered too often. We would be happy to update the config for folks, too, with a simple request on the Meta talk page.

Above all, JSON is arguably easier to learn than wikitext :)

Since nobody raised any objections, I'll shortly be moving the config page to JSON instead of the current wikitext table.

The JSON config can be found here. I do intend to keep the wikitext table updated using the bot.

Having both a JSON config and a WikiText table isn't going to work well. I can tell you from experience that people will edit the WikiText table and expect that to control the configuration. I can also tell you that no one is going to feel comfortable editing the JSON unless they are a programmer. Both WikiLove and PageTriage have JS-based on-wiki configuration and no one from the communities ever edits them, ever. They always ask someone from the WMF to edit them because they are scared of breaking them. We shouldn't ask editors to learn JSON in order to use popular pages, IMO. If we're worried that the WikiText table is going to be too fragile, we should provide a dedicated UI on Tool Labs instead.

In T156856#3044120, @kaldari wrote:

Having both a JSON config and a WikiText table isn't going to work well. I can tell you from experience that people will edit the WikiText table and expect that to control the configuration. I can also tell you that no one is going to feel comfortable editing the JSON unless they are a programmer. Both WikiLove and PageTriage have JS-based on-wiki configuration and no one from the communities ever edits them, ever. They always ask someone from the WMF to edit them because they are scared of breaking them. We shouldn't ask editors to learn JSON in order to use popular pages, IMO. If we're worried that the WikiText table is going to be too fragile, we should provide a dedicated UI on Tool Labs instead.

I agree that we shouldn't have a wikitable shown if there is also a JSON config. But as for JSON, I stand by my points above that this won't need to be changed much, and with good documentation it isn't hard to edit it. I mean, JSON is definitely easier than a wikitable, right? :) The "description" Niharika put in at Popular pages config.json looks good, except we didn't point out the issue with commas, which is my biggest concern. The last entry can't have a comma at the end or it is invalid JSON that I assume will break the bot. So it is tricky and potentially fragile. I'll note however other bots use JSON configs, including my own, and the community has edited them (IABot, for example). These configs also have talk pages that go with them, where those less comfortable can make an edit request there.

I'd like to bring up the idea of a template again. Communities are maybe more comfortable with that, as that's how they set up archiving bots. That's still not really bot-friendly, but perhaps it's a little easier to parse than a wikitable.

For this project I think auditing is important, where you can see who changed the configuration. The wiki does this natively, and I cringe at re-inventing the wheel with a web interface. Maybe we could have the web interface that works on that JSON page, redirecting them to the on-wiki edit interface with the new content put in, like reFill does. That way we don't need to add OAuth and what not.

I mean, JSON is definitely easier than a wikitable, right?

Definitely not for Wikipedia editors (most of which are not programmers).

@Niharika: It doesn't look like the current config data matches the historical config data. For example, your data includes WikiProject Alphabet, which isn't in the historical list, and the historical list includes WikiProject Anime and manga, which isn't in your list. We should be working off the historical subscription list as closely as possible. Please check on this before performing more edits.

What about a pop-up oojs-ui dialog that can edit individual entries or add new ones? Then there wouldn't be a problem with JSON syntax. I assume it's possible for a gadget to inject HTML into the pretty-printed JSON data table output (for adding edit buttons etc)?

Would be more work than letting people edit directly, but less than a whole external UI on labs.

Maybe something like FormWizard, but then we end up having to implement a separate gadget on every wiki we want to deploy on.

In T156856#3044983, @kaldari wrote:

Maybe something like FormWizard, but then we end up having to implement a separate gadget on every wiki we want to deploy on.

That's true, and people would have to turn the gadget on before editing the config page. So it might not get that much use — easier to link to an external tool i guess. Or just fix breakages when they happen.

(FormWizard would also have to be modified to edit more than just templates, which would probably be a good feature for it anyway. )

The wikitext table I intend to keep updated would look like:

And I'd make it clear in the table header that the bot will update the page after every run cycle, so don't edit it. Does that still seem like a bad idea? I thought it'd serve as an index of sorts.

In T156856#3044120, @kaldari wrote:

I can also tell you that no one is going to feel comfortable editing the JSON unless they are a programmer. Both WikiLove and PageTriage have JS-based on-wiki configuration and no one from the communities ever edits them, ever. They always ask someone from the WMF to edit them because they are scared of breaking them. We shouldn't ask editors to learn JSON in order to use popular pages, IMO. If we're worried that the WikiText table is going to be too fragile, we should provide a dedicated UI on Tool Labs instead.

I ran a query to check how often the config for projects in Mr.Z-bot changed and it's changed only 39 times (by 13 different editors) over 2.5 years (12 editors if you exclude Mr.Z-man). Most of them made batch-edits so really the number is 19 times. Most of the wikiprojects have never had their original config changed. Going by that, I don't really expect too many changes to the config.

In T156856#3044777, @MusikAnimal wrote:

The "description" Niharika put in at Popular pages config.json looks good, except we didn't point out the issue with commas, which is my biggest concern. The last entry can't have a comma at the end or it is invalid JSON that I assume will break the bot.

I've updated the page to hopefully make it clearer. Feel free to edit! :)

For this project I think auditing is important, where you can see who changed the configuration. The wiki does this natively, and I cringe at re-inventing the wheel with a web interface. Maybe we could have the web interface that works on that JSON page, redirecting them to the on-wiki edit interface with the new content put in, like reFill does. That way we don't need to add OAuth and what not.

That's an interesting idea. We could do validation checks for JSON on the interface and show better error messages than "Invalid content". Should be straightforward to implement. What say?

In T156856#3044854, @kaldari wrote:

@Niharika: It doesn't look like the current config data matches the historical config data. For example, your data includes WikiProject Alphabet, which isn't in the historical list, and the historical list includes WikiProject Anime and manga, which isn't in your list. We should be working off the historical subscription list as closely as possible. Please check on this before performing more edits.

Yes, I'm aware of that. I was hoping to talk to you sometime this week about it. The config I have on wiki is generated from the projects we have in PageAssessments database. I have no clue where Mr.Z-bot's config was generated from. There are a few projects in its list that are not there in PageAssessments. I am guessing that some of those projects are dead, or no longer active (or maybe deleted?) so I was going to go over them today.
Also yes, the current config has a bunch of projects which don't currently have a Popular pages report. I thought it would be okay to create one for them. Is that going to be a problem?
Lastly, some (<10) projects have their popular pages reports at non-standard urls such as /Popularpages or /popular_pages etc. I was going to just create their reports at /Popular_pages, like all others, and ask those projects to redirect their old reports to the new one to maintain some consistency across projects. I thought this would be a good opportunity to do that. Is that okay?

Oh, and if someone writes invalid JSON (e.g. trailing commas), they're prompted with "The document contains errors. Are you sure you want to save?" when they try to save. So that should prevent many syntax errors.

In T156856#3044777, @MusikAnimal wrote:
The "description" Niharika put in at Popular pages config.json looks good, except we didn't point out the issue with commas, which is my biggest concern. The last entry can't have a comma at the end or it is invalid JSON that I assume will break the bot.

I've updated the page to hopefully make it clearer. Feel free to edit! :)

In a startling revelation, it seems to be impossible to save the page if the JSON is invalid. And the more interesting thing - if we add a trailing comma at the end, of the last field or the last wikiproject block, it saves the JSON and removes the comma on its own. Feel free to test it out on this shorter version of that page.

There are a few projects in its list that are not there in PageAssessments. I am guessing that some of those projects are dead, or no longer active (or maybe deleted?) so I was going to go over them today.

Some of them are deleted and some may not be using the WPBannerMeta template.

Also yes, the current config has a bunch of projects which don't currently have a Popular pages report. I thought it would be okay to create one for them. Is that going to be a problem?

If we're just going to generate a report for every WikiProject in PageAssessments, I don't really see a good reason to even have a config system. We might as well just use a template (like MusikAnimal was suggesting) to control the size of each report and get the rest of the info dynamically from PageAssessments. If we are going to allow configuring a specific list, we should start with the list in the original config and not create reports for projects that didn't ask for them.

I'm not clear how the Template version would work. Would the report be stored in a database and if the template is used on a page, it loads the report on it somehow?

I don't mind taking off projects which don't already have a report either.
Whatever everyone else prefers.

This feels like it's getting further and further away from the original request, which was basically just to fix the existing bot. Can we just switch to using the old config list? If the assessments don't work, that's not a big deal.

I haven't looked at this ticket for a week, and it looks like a lot of discussion has flown past me. Anyway, as a former programmer, JSON is simple for me to update, but most Wikipedians aren't programmers.

Maybe it would be possible to let wiki editors edit a wiki table, but the bot maintainer periodically uses the wiki table to update their config data in whatever form they prefer (JSON, database table, etc.). There doesn't have to be an immediate on-switch for new popular pages anyway. Perhaps the bot maintainer could transfer the data once a week or as frequently as they choose.

Perhaps running straight off the table isn't a good idea anyway. Does any bot run straight off a table anyone can edit?

In T156856#3045503, @Niharika wrote:

In a startling revelation, it seems to be impossible to save the page if the JSON is invalid. And the more interesting thing - if we add a trailing comma at the end, of the last field or the last wikiproject block, it saves the JSON and removes the comma on its own. Feel free to test it out on this shorter version of that page.

NICE!!! Nevermind me then! I now 100% !vote for JSON.

In T156856#3048159, @Stevietheman wrote:

Maybe it would be possible to let wiki editors edit a wiki table, but the bot maintainer periodically uses the wiki table to update their config data in whatever form they prefer (JSON, database table, etc.). There doesn't have to be an immediate on-switch for new popular pages anyway. Perhaps the bot maintainer could transfer the data once a week or as frequently as they choose.

That could work, but it's a little confusing. I think instead users should simply ask that a WikiProject be added on the config talk page.

Perhaps running straight off the table isn't a good idea anyway. Does any bot run straight off a table anyone can edit?

Not that I'm aware of. Templates and JSON seem to be the standard.

In T156856#3045701, @Niharika wrote:

I'm not clear how the Template version would work. Would the report be stored in a database and if the template is used on a page, it loads the report on it somehow?

The bot would use the Transcludein API to find all pages that transclude the template, and the template parameter and values would act as the config. E.g.

{{User:Community Tech bot/Popular pages
| name = New York City
| report = Wikipedia:WikiProject New York City/Popular pages
| limit = 1000
}}

Though you'd probably put this template on the page the bot would write the report to, and do without the report= parameter. The template itself doesn't have to render anything. For example see User:MiszaBot/config. This approach to bot configs is common and shouldn't be too difficult for users to grasp, but I still think JSON is the best and easiest route. It's super bot-friendly, syntax errors cannot be saved (yay!), wouldn't need to be changed often, and anyone who's afraid of editing it can request an addition on the talk page or reach out to the bot operators.

P4971 - Projects in Z-bot config and not in Page assessments table (235 listed)

In T156856#3045444, @Niharika wrote:

The wikitext table I intend to keep updated would look like:

Wikiproject title	Report link	Limit	Updated on
Wikipedia:WikiProject_Spiders	Wikipedia:WikiProject_Spiders/Popular_pages	500	21 Feb 2017
Wikipedia:WikiProject_Biology	Wikipedia:WikiProject_Biology/Popular_pages	1000	21 Feb 2017

And I'd make it clear in the table header that the bot will update the page after every run cycle, so don't edit it. Does that still seem like a bad idea? I thought it'd serve as an index of sorts.

Thoughts on this?

I guess that's fine as long as there is a prominent warning at the top to not edit the table, but to edit the JSON page instead.

Yeah it seems like a nicely formatted reference. I especially like the "Updated on" so we can easily see if it failed for one the projects.

In T156856#3058896, @MusikAnimal wrote:

Yeah it seems like a nicely formatted reference. I especially like the "Updated on" so we can easily see if it failed for one the projects.

Wait... Shouldn't we be using a logfile of some sort so that we can troubleshoot failures?

It looks like Mr. Z-Man had some ad hoc logfiles in his code that are disabled (perhaps for performance reasons). If someone could provide some consulting assistance (I'm not at all familiar with the tools.wmflabs.org environment), I'd be more than happy to troubleshoot the immediate problem. That would save you the trouble of performing a rewrite that, while well-intentioned, nobody asked for.

In T156856#3058760, @kaldari wrote:

I guess that's fine as long as there is a prominent warning at the top to not edit the table, but to edit the JSON page instead.

The first rule of user interface design is "don't make the end users perform more work than they have to". This approach of introducing yet another editor seems to be more about satisfying the needs of a particular developer than maintaining mistake-proofed existing functionality for the end users.

In T156856#3018411, @kaldari wrote:

Hmm, there are definitely advantages to either approach. Maybe we should stick with a web interface since that's what people are already used to using and it will be less easy for people to break it by entering bad data. There are two existing config tables we can use:

• s51401__pop_data.project_config - The current config data
• s51401__pop_data.config_history - The history of changes to the config

If we use these, at some point we'll need to add columns for the wikis, e.g. 'enwiki', but we can use them how they are for now.

There is wisdom in not reinventing the wheel. It would be great if you all would reconsider your desire to re-write everything.

In T156856#3063471, @DanielPenfield wrote:

Wait... Shouldn't we be using a logfile of some sort so that we can troubleshoot failures?

It looks like Mr. Z-Man had some ad hoc logfiles in his code that are disabled (perhaps for performance reasons). If someone could provide some consulting assistance (I'm not at all familiar with the tools.wmflabs.org environment), I'd be more than happy to troubleshoot the immediate problem. That would save you the trouble of performing a rewrite that, while well-intentioned, nobody asked for.

I think the wiki table is more for user reference, not for our purposes. We do have server-side logging of errors, etc.

In T156856#3063482, @DanielPenfield wrote:

In T156856#3058760, @kaldari wrote:

I guess that's fine as long as there is a prominent warning at the top to not edit the table, but to edit the JSON page instead.

The first rule of user interface design is "don't make the end users perform more work than they have to". This approach of introducing yet another editor seems to be more about satisfying the needs of a particular developer than maintaining mistake-proofed existing functionality for the end users.

The wiki will be updated by a bot. I'm guessing by "editor" you mean the need to edit the JSON config? That's the only thing that requires editing, which we're happy to do so on behalf of users, and we will make this clear. They could also use the talk page, similar to other bots operating off of an on-wiki config.

There is wisdom in not reinventing the wheel. It would be great if you all would reconsider your desire to re-write everything.

I'm not sure which route you're arguing for. The web interface is nice, but it will require some rebuilding to meet our needs. The big must-haves are a database and auditing system so we can see who changed what – both a wiki does natively, so in that sense we would be reinventing the wheel. Users could also monitor the config via watchlist which wouldn't be possible with a Tool Labs tool. As for a quick formatted reference like this, we'd have the bot-generated wiki table.

Considering this done. The bot is working off of the JSON config on wiki.

I'll just chime in to say that I'm comfortable with where this part is heading. Thanks everyone.