Skip to content

[ticket/14257] Reparse after update#4005

Merged
Nicofuma merged 10 commits into
phpbb:masterfrom
Elsensee:ticket/14257
Dec 5, 2015
Merged

[ticket/14257] Reparse after update#4005
Nicofuma merged 10 commits into
phpbb:masterfrom
Elsensee:ticket/14257

Conversation

@Elsensee

Copy link
Copy Markdown
Contributor

One new cron task for each reparser.
Advantage of this approach is that the decision which reparser will run in the cron job is made before running it.
Disadvantage is that extension authors have to create their own cron task but since they only have to define that in the services.yml and they have to create their own reparser class anyway, I don't think this is a big issue.
So basically no disadvantages. 😀

Cron job and console command can't interfere with eachother because the reparser acquires a lock, of course.
However, cron job and console command will be synced. So the console command can resume where the cron job stopped and vice versa.

This also means unfortunately that an administator can set weird values by using the console command (weird start, end, size IDs) which might lead to an unfinished reparsing. I don't know if this is something we have to care about?

And since this is your baby, @JoshyPHP, please have a look as well. :) ;) Thank you! :)

PHPBB3-14257

@prototech

Copy link
Copy Markdown
Contributor

I don't think we really need cron tasks for the forum data, contact admin info, and group descriptions. We should be able to reparse these from a single migration. The contact admin info is just a single config entry for example.

@Elsensee

Copy link
Copy Markdown
Contributor Author

Indeed, I didn't thought about that. ^^

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is that conditional correct? Looks like it should not be negated.

Either way, I think it would be better to simply have a conditional with only one branch that throws an exception and no else, e.g..

if (!$this->reparse_lock->acquire())
{
    throw new Exception;
}

// ...the rest executes as usual

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Argh, thanks. That's the way I wanted to write it first, but then I changed it (mainly to be consistent with everything else that acquires a lock) and forgot to change the conditional too. :/

@JoshyPHP

Copy link
Copy Markdown
Contributor

What do you think of moving the loading/saving of resume data into its own class? Perhaps even the locking mechanism? Not sure about that one. Maybe it could have a simplified API to schedule reparsing via the cron. It would be useful for extension authors I guess. You could have something like that:

namespace phpbb\textreparser;
class manager
{
    public function get_resume_data($name);
    public function update_resume_data($name, $min, $current, $size);
    public function schedule($name);
}

@Elsensee

Copy link
Copy Markdown
Contributor Author

Well, that would reduce code duplication, so I guess we can do that.

The locking mechanism already reuses the phpbb\lock\db class. Also why simplifying it further? I think the $reparse_lock->acquire(); and $reparse_lock->release(); should be simple enough?

Extension authors actually don't have to write any class for this. For the cronjob they can just reuse this one and change the call in the service definition, while for the CLI command adding the tag to their reparser class is enough.

More concerning should be that an administrator can do weird stuff in the console command which can have unexpected results for the cron job, right? What do you think about that?

@Elsensee Elsensee changed the title [ticket/14257] Add cron tasks for reparsing text [ticket/14257] Reparse after update Oct 24, 2015
@JoshyPHP

Copy link
Copy Markdown
Contributor

What kind of weird stuff can they do? Reparse a single post and interrupt the cron maybe?

I don't know. I think that people who use CLI commands should know better, and if they interrupt the cron (by overwriting the resume data) they'll be able to restart reparsing. I don't have a strong opinion.

@Elsensee

Copy link
Copy Markdown
Contributor Author

What kind of weird stuff can they do? Reparse a single post and interrupt the cron maybe?

Exactly.
Well I think, that we maybe could just remove the possibility to define a start and end ID in the CLI command?

@Elsensee

Copy link
Copy Markdown
Contributor Author

@JoshyPHP Writing the manager resume_data_manager right now. What's the schedule() supposed to do actually?

@JoshyPHP

Copy link
Copy Markdown
Contributor

schedule('post_text') would create/enable/update a cron task that reparses all post_text. Like the reparser CLI, instead of post_text it could be the name of a reparser service such as text_reparser.poll_option. schedule_all() would cause every available reparser service to be scheduled to be run via cron.

@Elsensee

Copy link
Copy Markdown
Contributor Author

Added the text_reparser manager now. (I also fixed stuff so console command and cron task now can really work together ^^)
I guess I'll remove the start, end, size options from the CLI command and then.. what's left then? I think that's it. 😁

@Nicofuma

Copy link
Copy Markdown
Member

Why would you remove these options from the CLI command? This command isn't meant only for the update, it shouldn't be a command you can use only once in the life of your forum but anytime you need (or think you need) to reparse the posts (or a sub set) of your forum.

The console is meant to be used by advanced user, please keep it flexible

Comment thread phpBB/language/en/acp/common.php Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a very technical message, you should explain what the issue is (ie. an other process might already ben using the reparser)

@Nicofuma

Copy link
Copy Markdown
Member

@phpbb/native ^

@Elsensee
Elsensee force-pushed the ticket/14257 branch 2 times, most recently from 0bc3bb1 to 762b383 Compare November 21, 2015 18:42

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about Another process is already running reparsing. or Another process is already reparsing.?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another reparsing process is running ?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another reparsing process is already running?

@marc1706

Copy link
Copy Markdown
Member

👍

@Nicofuma
Nicofuma merged commit 762b383 into phpbb:master Dec 5, 2015
Nicofuma pushed a commit that referenced this pull request Dec 5, 2015
[ticket/14257] Reparse after update

* Elsensee/ticket/14257:
  [ticket/14257] Add tests for reparser manager
  [ticket/14257] Fix CLI reparser and set cron interval
  [ticket/14257] Fix CLI error message
  [ticket/14257] Fix phpdoc in CLI command
  [ticket/14257] Add text_reparser manager
  [ticket/14257] Use migrations instead of cron job for some reparsers
  [ticket/14257] Fix lock acquire in CLI command
  [ticket/14257] Fix if condition
  [ticket/14257] Add reparse_lock to CLI command
  [ticket/14257] Add cron tasks for reparsing text
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants