Skip to content

feat(trends): add fixed window water level trend layers 5 10 20 years (DIE-400) - #154

Open
chasetmartin wants to merge 3 commits into
mainfrom
DIE-400-add-fixed-window-water-level-trend-layers-5-10-20-years
Open

chasetmartin wants to merge 3 commits into
mainfrom
DIE-400-add-fixed-window-water-level-trend-layers-5-10-20-years

Conversation

@chasetmartin

@chasetmartin chasetmartin commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Why

This PR addresses the following problem / context:

How

Implementation summary: the following was changed / added / removed:

  • Added nm_waterlevel_trends_5yr, _10yr and _20yr. Each window covers whole calendar months before the run month and is the same for every well.
  • All trends now use one value per month: the median for water levels, still using mean for analytes (until further user research).
  • Ethan, Monica, Geoff, and Amy from AMMP all confirmed this monthly median method as worth trying.
  • The minimum-data rule is unchanged (10 points, or 4 points spanning at least 2 years) and is applied inside each window.
  • nm_waterlevel_trends is retitled "Full Record" to mark it as the overall trend. Its id is unchanged.
  • New fields: trend_period, window_years, window_start, window_end, insufficient_reason.
  • Added unit tests and updated the product docs.

Author Checklist

  • Changes are limited to the intended scope.
  • Tests were added or updated where appropriate.
  • Relevant tests and lint checks pass.
  • No secrets, credentials, or sensitive data were publicly exposed.
  • Documentation was updated where necessary.

Reviewer Checklist

  • The window layers and the new fields look right.
  • Keeping the nm_waterlevel_trends id while retitling it is acceptable.

Notes

Any special considerations, known limitations, risks, workarounds, or follow-up work to note?

  • Monthly values don't remove seasonality entirely. A seasonality-adjusted test is a possible follow-up or another method to add alongside within the same trend products.

Every trend is now tested on one value per calendar month: the median for
depth-to-water, the mean for analytes. A single analyte flag picks both the
aggregate and the method text, so they always agree. Trend collections can
be limited to a trailing window of whole calendar months before the run
month, using one run date per Dagster run. Features carry trend_period,
window_years, window_start/window_end and an insufficient_reason when the
minimum-data rule fails.
Add nm_waterlevel_trends_5yr/_10yr/_20yr. nm_waterlevel_trends is relabeled
as the full-record (overall) trend; its id is unchanged. Analyte trend
abstracts now describe the monthly mean.
@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

Your pull request is automatically being deployed to Dagster Cloud.

Location Status Link Updated
die-orchestration View in Cloud Nov 11, 2026 at 12:20 AM (UTC)

@chasetmartin chasetmartin changed the title Die 400 add fixed window water level trend layers 5 10 20 years feat(trends): add fixed window water level trend layers 5 10 20 years (DIE-400) Nov 11, 2026

@jacob-a-brown jacob-a-brown left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall I think it looks! I made one comment about slope in feet per year. If you agree with my comment then the same update needs to be applied elsewhere. I just didn't want to litter this PR with the same comment throughout.

Comment on lines +101 to +102
with the Mann-Kendall test (alpha = 0.05), with the rate given as a Theil-Sen
slope in feet per year. Wells need at least 10 monthly values, or at least 4

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if this is tested on monthly data wouldn't it be slope per month?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The series is now aggregated to monthly, but I believe the Theil-Sen slope is fit against year timestamps slope_per_year = float(theilslopes(values, years)[0]), so the slopes are ft/yr.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants