Skip to content

Replace all occurrences in large search-replace operations #473

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
scribu merged 8 commits into from
May 30, 2013
Merged

Replace all occurrences in large search-replace operations #473

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
d351534
Add a test for large search/replace failure.
cyberhobo May 27, 2013
fba2f4e
Use the STDOUT saving technique so tests can pass.
cyberhobo May 28, 2013
132bf37
Add a test for the case when the replacement contains the search string.
cyberhobo May 28, 2013
45770c2
Adjust chunking based on whether the replacement contains the search …
cyberhobo May 28, 2013
611afd7
Generate only as many posts as needed to test chunking.
cyberhobo May 28, 2013
05a8929
Revert "Adjust chunking based on whether the replacement contains the…
cyberhobo May 29, 2013
6196545
Add a dry run test to make sure the iterator doesn't depend on updates.
cyberhobo May 29, 2013
b58290c
Reduce chunk offset to account for updated rows.
cyberhobo May 29, 2013
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 47 additions & 0 deletions features/search-replace.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,50 @@ Feature: Do global search/replace
"""
guid
"""

Scenario: Large guid search/replace where replacement contains search
Given a WP install

When I run `wp option get siteurl`

And save STDOUT as {SITEURL}

And I run `wp post generate --count=1000`

And I run `wp search-replace {SITEURL} {SITEURL}/subdir`
Then STDOUT should be a table containing rows:
"""
Table Column Replacements
wp_posts guid 1002
"""
Scenario: Large guid search/replace where replacement does not contain search
Given a WP install

When I run `wp option get siteurl`

And save STDOUT as {SITEURL}

And I run `wp post generate --count=1000`

And I run `wp search-replace {SITEURL} http://newdomain.com`
Then STDOUT should be a table containing rows:
"""
Table Column Replacements
wp_posts guid 1002
"""

Scenario: Large guid search/replace dry run where replacement does not contain search
Given a WP install

When I run `wp option get siteurl`

And save STDOUT as {SITEURL}

And I run `wp post generate --count=1000`

And I run `wp search-replace --dry-run {SITEURL} http://newdomain.com`
Then STDOUT should be a table containing rows:
"""
Table Column Replacements
wp_posts guid 1002
"""
28 changes: 28 additions & 0 deletions php/WP_CLI/Iterators/Query.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,12 @@ class Query implements \Iterator {

private $chunk_size;
private $query = '';
private $count_query = '';

private $global_index = 0;
private $index_in_results = 0;
private $results = array();
private $row_count = 0;
private $offset = 0;
private $db = null;
private $depleted = false;
Expand All @@ -34,12 +36,38 @@ class Query implements \Iterator {
*/
public function __construct( $query, $chunk_size = 500 ) {
$this->query = $query;

$this->count_query = preg_replace( '/^.*? FROM /', 'SELECT COUNT(*) FROM ', $query, 1, $replacements );
if ( $replacements != 1 )
$this->count_query = '';

$this->chunk_size = $chunk_size;

$this->db = $GLOBALS['wpdb'];
}

/**
* Reduces the offset when the query row count shrinks
*
* In cases where the iterated rows are being updated such that they will no
* longer be returned by the original query, the offset must be reduced to
* iterate over all remaining rows.
*/
private function adjust_offset_for_shrinking_result_set() {
if ( empty( $this->count_query ) )
return;

$row_count = $this->db->get_var( $this->count_query );

if ( $row_count < $this->row_count )
$this->offset -= $this->row_count - $row_count;

$this->row_count = $row_count;
}

private function load_items_from_db() {
$this->adjust_offset_for_shrinking_result_set();

$query = $this->query . sprintf( ' LIMIT %d OFFSET %d', $this->chunk_size, $this->offset );
$this->results = $this->db->get_results( $query );

Expand Down