Repository navigation
fix(water-level-csv): set observation release status (BDMS-1560) - #1055
Open
ksmuczynski wants to merge 2 commits into
Conversation
The water level CSV importer creates each groundwater level observation as draft, even when its well is public. The public EDR waterlevels collection (ogc_waterlevels) shows an observation only when both the observation and its well are public. So the collection includes none of the observations this importer creates. The importer now gives each observation it creates the same release status as its well. It uses observation_release_status, which BDMS-1559 added for the well inventory importer. That function copies public and private, and returns draft for any other status. Public and private are the only statuses that indicate whether the well's data may be published. The well inventory importer creates each well and sets the well's status to public, private or draft. This importer adds observations to existing wells, which may have a status other than public, private or draft. An observation on such a well stays draft. The importer sets the status whenever it creates an observation. It creates one in two cases: - The row matches no existing sample, so the importer creates a sample and an observation. - The row matches an existing sample, by the name the importer builds for the row, but that sample has no groundwater level observation yet. When a re-import matches an existing observation, the importer updates the observation's values but leaves its status unchanged. Staff can change an observation's status through the API, and a re-import should not overwrite their change. The BDMS-1561 backfill sets the status of observations this importer created before this fix, so the importer does not need to change them. Because the importer sets the status only when it creates an observation, a draft well that staff later make public keeps its imported observations at draft. The EDR collection does not include them until someone makes them public too. This commit also changes the BDMS-1561 backfill test test_water_level_release_status_publishes_a_real_csv_import. The test checks that the backfill changes an observation this importer created before this fix. It imports a row through this importer and expects the new observation to be draft. The test now replaces observation_release_status with a function that returns draft, so the import creates a draft observation, as the importer did before this fix. The test could not instead run a normal import and then set the observation to draft. The import writes an observation_version row with status public, and the backfill skips any observation whose observation_version rows include a status other than draft. The backfill would leave that observation unchanged, and the test would fail. Refs BDMS-1560
Before this fix, the tests gave these results: - The tests failed when the well was public or private, both when the import created a new sample and when it added the observation to an existing sample. The importer left the observation as draft, and the tests expect public or private. - The tests passed when the well was draft or provisional. The importer already left those observations as draft, which is the correct status. These cases will fail if a later change gives those observations any other status. - The re-import test passed. The importer did not change an existing observation's status. The test will fail if a later change makes a re-import copy the well's status onto an existing observation. The well status test runs once for each well status: public, private, draft and provisional. Each run imports one row and checks that: - The observation's status is public for a public well, private for a private well, and draft for the other two. - The sample, field activity and field event stay draft. - The observation appears in ogc_waterlevels only when the well is public. The fix exists to make these observations appear there, so the test checks the collection itself. A later change to the view's filter that excludes them fails the test. The re-import test imports a row on a public well, sets the observation to private, and imports the same row again. It checks that the observation is still private. The importer would never give an observation on a public well the status private, so a re-import that copied the well's status would change it. The existing-sample test creates a sample with only a pH observation. The sample has the name the importer builds for the row, so the import adds a groundwater level observation to that sample instead of creating a new sample. The test runs with a public well and a private well. It checks that the new observation's status equals the well's. It also checks that the pH observation stays draft, because the importer did not create that observation. Refs BDMS-1560
Contributor
Coverage✅ 82.03% total — gate is 75%. Coverage for the Python files changed in this PR
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR makes the water level CSV importer give each groundwater level observation it creates the same release status as its well. After this change, an observation the importer creates for a public well appears in the public EDR waterlevels collection.
Why
This PR addresses the following problem / context:
ogc_waterlevelsshows an observation only when both the observation and its well are public, so these observations do not appear there.How
Implementation summary - the following was changed / added / removed:
services/water_level_csv.pyto set the release status of each new observation withobservation_release_status, which BDMS-1559 added.test_water_level_release_status_publishes_a_real_csv_importso that its import still creates a draft observation, like an import from before this fix.ogc_waterlevelsincludes the observation only when the well is public.Notes
Any special considerations, workarounds, or follow-up work to note?
observation_release_status. It targets the BDMS-1559 branch until BDMS-1559 merges, so the diff shows only this ticket's changes.forceoption.NGWMN_WaterLevels) requires those three records to be public as well, so it does not include these observations. A separate ticket should be made to cover NGWMN.last_observation_dateon the publicwater_wellslayer now includes them. Materialized layers, such asogc_well_water_column, include them after their next refresh.Refs BDMS-1560