ROX-20769: Use pg_upgrade in init-entrypoint#14447
Conversation
|
Skipping CI for Draft Pull Request. |
|
Images are ready for the commit at 95d5a7d. To use with deploy scripts, first |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #14447 +/- ##
==========================================
- Coverage 49.26% 48.98% -0.29%
==========================================
Files 2528 2550 +22
Lines 184979 187198 +2219
==========================================
+ Hits 91128 91690 +562
- Misses 86622 88262 +1640
- Partials 7229 7246 +17
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
398431f to
6f4dd31
Compare
porridge
left a comment
There was a problem hiding this comment.
TBH I must be missing something, because this script seems to:
- make a backup in container's root FS, which IIRC disappears as soon as the container exits
- run the old version against another copy of the data
- run the upgrade
4419d70 to
9c3ee42
Compare
|
@porridge thanks for checking out.
It's a temporary approach for testing, until a persistent volume is added.
For testing, to verify that the backup works. |
e23bf12 to
286d8a1
Compare
286d8a1 to
0e43d99
Compare
Turns out postgresql-upgrade wrapper doesn't allow to inspect failure logs, since it wipes out the new data directory on the upgrade failure.
There was a problem hiding this comment.
Hey @erthalion - I've reviewed your changes - here's some feedback:
Overall Comments:
- Consider adding a feature flag to control the execution of the upgrade logic.
- Consider using a dedicated tool for managing file system operations, such as
rsync, instead oftar.
Here's what I looked at during the review
- 🟡 General issues: 2 issues found
- 🟡 Security: 1 issue found
- 🟢 Testing: all looks good
- 🟢 Complexity: all looks good
- 🟢 Documentation: all looks good
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
It will be bumped separatelly
|
@erthalion: The following tests failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
Introduce a major upgrade logic into CentralDB init-db script. The implementation is based on the pg_upgrade [1] and postgresql-container [2]. We can't use the latter upgrade scripts directly, because they require to use provided entrypoints, which will need some more work from our side. We also couldn't use `postgresql-upgrade` script directly, because it turns out it doesn't allow to keep logs after upgrade if something went wrong. The upgrade scenario looks like this: * Check if the binaries version is newer than the data version and the upgrade is needed. * Make sure the cluster was shutdown properly. * Figure out if there is enough disk space available for the upgrade. The disk space needs to accomodate one mandatory backup and one temporary restored database to verify the backup. Hence the available size must be more than two times existing database size. * Take a physical backup, and verify it via restoring a temporary copy of the database. * Spin up a new PG15 database. * Do the upgrade check first. * If successfull, do the full upgrade, then swap old and new databases. The script knows about a backup volume, and tries to use it for making a backup. If it's not present, the main data volume will be used. The script is not idempotent: * If a backup was already taken, it will not be recreated. * If a new PG15 database already exists, the upgrade will be aborted. * If there is not enough disk space available, the upgrade will be aborted. [1]: https://www.postgresql.org/docs/current/pgupgrade.html [2]: https://github.com/sclorg/postgresql-container/
Experimental trigger for major version upgrade.
User-facing documentation
Testing and quality
Automated testing
How I validated my change
change me!