Skip to content

Set $interactive = true when output is STDOUT#198

Closed
aidvu wants to merge 3 commits intowp-cli:masterfrom
aidvu:fix-oom-export-stdout
Closed

Set $interactive = true when output is STDOUT#198
aidvu wants to merge 3 commits intowp-cli:masterfrom
aidvu:fix-oom-export-stdout

Conversation

@aidvu
Copy link

@aidvu aidvu commented Jun 17, 2021

Changes to run_mysql_command made it load process output
into variables, which for db export tends to OOM and completely breaks piping.
The old behavior is retained when $interactive = true
for run_mysql_command. When we're outputting to STDOUT, interactive
mode makes sense as it defaults to STDIN/STDOUT which in my opinion
is the expected behavior.

Fixes #195

Changes to run_mysql_command load command output
into strings, which for DB export commands tends to OOM.
Seems the old behavior is retained when $interactive = true
for run_mysql_command. When we're outputting to STDOUT, interactive
mode makes sense and fixes the problem with exports and piping.
@aidvu aidvu requested a review from a team as a code owner June 17, 2021 10:40
@aidvu aidvu changed the title DB Export: Set $interactive = true when output is STDOUT db export: Set $interactive = true when output is STDOUT Jun 17, 2021
@schlessera schlessera changed the title db export: Set $interactive = true when output is STDOUT Set $interactive = true when output is STDOUT Jul 1, 2021
@schlessera
Copy link
Member

I've been trying to wrap my head around this fix for a while now. I don't think this works as expected.

When trying to export to STDOUT, having $interactive be true would mean that any prompts end up in the pipe and go over the target server/app.

Also, when trying to export to STDOUT, you cannot have $send_to_shell be false, as your pipe will then be empty.

I'm wondering whether you've tested this and whether I am maybe misunderstanding something...

@aidvu
Copy link
Author

aidvu commented Jul 2, 2021

When trying to export to STDOUT, having $interactive be true would mean that any prompts end up in the pipe and go over the target server/app.

This change is ONLY for wp db export -. All prompts would go over if there were any. This is how it worked before the updates in 2.5.0: wp-cli/wp-cli@f0db86c#diff-44579845db87c0c3a4c0b98809b72e841722fae9cf67edd1948c338b1fd95396R481.

Also, when trying to export to STDOUT, you cannot have $send_to_shell be false, as your pipe will then be empty.

When $interactive = true you're starting the process with these descriptiors: https://github.com/wp-cli/wp-cli/blob/c3bd5bd76abf024f9d492579539646e0d263a05a/php/utils.php#L506. So the process simply outputs to STDOUT. This also means that both $stdout and $stderr are going to be empty strings since we're not setting them because of $interactive = true, so printing them doesn't make much sense.

I'm wondering whether you've tested this and whether I am maybe misunderstanding something...

Made a manual wp-cli 2.5.0 build with this patch, and it's in production for 2-3 days now.

@schlessera
Copy link
Member

@aidvu Thanks for the work on this, but I'm closing this unmerged in favor of wp-cli/wp-cli#5546. That one is pretty similar in terms of how it works, but fixes the issue in a more central way, without creating an exception for a single command.

And thanks for the help on debugging the above PR!

@schlessera schlessera closed this Jul 20, 2021
@aidvu aidvu deleted the fix-oom-export-stdout branch July 21, 2021 09:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Export to STDOUT loads entire DB into memory at once

2 participants