Skip to content

Freeze 3.73 schema in migrator#3769

Merged
c-du merged 6 commits intomasterfrom
cong/sep
Nov 16, 2022
Merged

Freeze 3.73 schema in migrator#3769
c-du merged 6 commits intomasterfrom
cong/sep

Conversation

@c-du
Copy link
Copy Markdown
Contributor

@c-du c-du commented Nov 10, 2022

Description

This PR wont merge before 3.73 branching out.
Freeze 3.73 schema in migrator.
I did not remove the auto generator because I want to keep that in a clean PR dedicated for stopping generators so we can conveniently re-use that PR in case we want to make massive change.

Checklist

  • Investigated and inspected CI test results
  • [ ] Unit test and regression tests added
  • [ ] Evaluated and added CHANGELOG entry if required
  • [ ] Determined and documented upgrade steps
  • [ ] Documented user facing changes (create PR based on openshift/openshift-docs and merge into rhacs-docs)

If any of these don't apply, please comment below.

Testing Performed

TODO(replace-me)
Use this space to explain how you tested your PR, or, if you didn't test it, why
you did not do so. Valid reasons include, for example, "CI is sufficient",
"No testable changes". Feel free to attach JSON snippets, curl commands,
screenshots.

In addition to reviewing your code, reviewers must also review your testing
instructions and make sure they are sufficient.

@c-du
Copy link
Copy Markdown
Contributor Author

c-du commented Nov 10, 2022

/test all

@openshift-ci
Copy link
Copy Markdown

openshift-ci bot commented Nov 10, 2022

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@c-du
Copy link
Copy Markdown
Contributor Author

c-du commented Nov 14, 2022

/test all

@c-du c-du requested a review from dashrews78 November 14, 2022 21:43
@ghost
Copy link
Copy Markdown

ghost commented Nov 14, 2022

Images are ready for the commit at c58006a.

To use with deploy scripts, first export MAIN_IMAGE_TAG=3.72.x-612-gc58006ab4d.

"github.com/stackrox/rox/migrator/types"
pkgMigrations "github.com/stackrox/rox/pkg/migrations"
pkgSchema "github.com/stackrox/rox/pkg/postgres/schema"
pkgSchema "github.com/stackrox/rox/migrator/migrations/frozenschema/v73"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should call this frozenSchema to make it stand out.

@dashrews78
Copy link
Copy Markdown
Contributor

a README or something might be useful to explain the reasoning for separating the schema as well as how migrations work going forward. Things like when we lock the migration schema, how we work migrations for the next release. How that impacts the generated code.

@c-du c-du marked this pull request as ready for review November 15, 2022 17:22
@c-du
Copy link
Copy Markdown
Contributor Author

c-du commented Nov 15, 2022

a README or something might be useful to explain the reasoning for separating the schema as well as how migrations work going forward. Things like when we lock the migration schema, how we work migrations for the next release. How that impacts the generated code.

Nice idea. Will do

@c-du c-du added the do-not-merge A change which is not meant to be merged label Nov 15, 2022
@c-du c-du requested a review from dashrews78 November 15, 2022 20:30
Copy link
Copy Markdown
Contributor

@dashrews78 dashrews78 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@c-du c-du requested a review from a team as a code owner November 15, 2022 21:22
Copy link
Copy Markdown
Contributor

@md2119 md2119 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The frozen schema seems to be differ than the ones in pkg/postgres/schema.

Comment on lines +21 to +24
GroupsSchema = func() *walker.Schema {
schema := walker.Walk(reflect.TypeOf((*storage.Group)(nil)), "groups")
return schema
}()
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The schema seems different. The registration to global table is missing:

GroupsSchema = func() *walker.Schema {
		schema := GetSchemaForTable("groups")
		if schema != nil {
			return schema
		}
		schema = walker.Walk(reflect.TypeOf((*storage.Group)(nil)), "groups")
		RegisterTable(schema, CreateTableGroupsStmt)
		return schema
	}()

Copy link
Copy Markdown
Contributor Author

@c-du c-du Nov 16, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is on purpose. The dependency are pre-defined and the tables are not to be reused. The schemas are local to this section of migrations, we do not have global schema in migrator.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay. What I am worried about is that event if the migrator and its schema is frozen, one can still make changes to the schema/proto which actually will go unhandled. I think we should add a check which scan pkg/schema/ dir for any changes. If the changes are desirable, the file is added to a whitelist. WDYT?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The migration should just have the schema at 3.73. It should not contain future changes to schema. That is what we expect.

@c-du c-du requested a review from md2119 November 16, 2022 05:10
Copy link
Copy Markdown
Contributor

@md2119 md2119 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving because I do not want to block the work, but I feel a required check that fails the CI would be better for maintenance than dropping schema into new pkg.

@c-du c-du removed the do-not-merge A change which is not meant to be merged label Nov 16, 2022
@c-du c-du merged commit 5e1288d into master Nov 16, 2022
@c-du c-du deleted the cong/sep branch November 16, 2022 21:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants