Repository navigation
Purge git history for irrelevant files from all repos #5594
Description
WP-CLI started out as a singe repository in git and was split up into multiple packages many years later.
When splitting up the packages, we tried to make sure that no historical knowledge is lost, so each subsplit package started out as the original main package and then removed the files that were not needed.
However, it seems like the purging we did afterward to remove unused history was not thorough enough.
Right now, each package contains the history of all WP-CLI files across all commands up to the point of the split. This makes most statistics and contribution data useless and drastically increases the size of the individual VCS repos. When cloning a full environment, each command repo has about 10MB of historical data of which most is useless.
To solve this, a more thorough purge should be done:
git checkout master
git ls-files > keep-these.txt
git ls-files | while read -r line; do (git log --follow --raw --diff-filter=R --pretty=format:%H "$line" | while true; do if ! read hash; then break; fi; IFS=$'\t' read mode_etc oldname newname; read blankline; echo $oldname; done); done >> keep-these.txt
git filter-branch --force --index-filter "git rm --ignore-unmatch --cached -qr . ; cat $PWD/keep-these.txt | tr '\n' '\0' | xargs -d '\0' git reset -q \$GIT_COMMIT --" --prune-empty --tag-name-filter cat -- --all
rm -rf .git/refs/original/
git reflog expire --expire=now --all
git gc --aggressive --prune=nowThen, the master branch needs to be force-pushed to overwrite the current master branch history.
Therefore, this should be worked upon for each repo separately, while first ensuring no PRs are open against the master branch anymore.
TODO:
- wp-cli/wp-cli-bundle
- wp-cli/cache-command
- wp-cli/checksum-command
- wp-cli/config-command
- wp-cli/core-command
- wp-cli/cron-command
- wp-cli/db-command
- wp-cli/embed-command
- wp-cli/entity-command
- wp-cli/eval-command
- wp-cli/export-command
- wp-cli/extension-command
- wp-cli/i18n-command
- wp-cli/import-command
- wp-cli/language-command
- wp-cli/maintenance-mode-command
- wp-cli/media-command
- wp-cli/package-command
- wp-cli/rewrite-command
- wp-cli/role-command
- wp-cli/scaffold-command
- wp-cli/scaffold-package-command
- wp-cli/search-replace-command
- wp-cli/server-command
- wp-cli/shell-command
- wp-cli/super-admin-command
- wp-cli/widget-command
- wp-cli/wp-cli
- wp-cli/wp-cli-tests
- wp-cli/doctor-command
- wp-cli/profile-command
- wp-cli/handbook
- wp-cli/wp-cli.github.com
Activity
wojsmol commentedon Dec 24, 2021
@schlessera If you are already doing task requiring 0 open PR's against
masterthen maybe at the same time changemasterbranch tomainortrunk?schlessera commentedon Jan 5, 2022
@wojsmol Yes, agree, this should be done in tandem with #5598
danielbachhuber commentedon Nov 18, 2022
Prior to purging the history, I'm capturing the code to each open pull request in this manner:
If the pull request doesn't have an issue, I'll open that too.
danielbachhuber commentedon Nov 18, 2022
On Mac,
xargsdoesn't have-d. The easy workaround is tobrew install findutilsforgxargs:The before and after of
du -h -d 0 .gitshould decrease substantially in size.Once you're triple-sure of your changes, run
git push -f origin master. You might need to enable force pushes beforehand.danielbachhuber commentedon Nov 18, 2022
🚢