Merge coastal to ocean/develop: COMPASS and sediment additions by mark-petersen · Pull Request #485 · MPAS-Dev/MPAS-Model

mark-petersen · 2020-03-24T03:30:24Z

Additions for the coastal needs. Includes three new analysis members:

harmonic_analysis
sediment_flux_index
sediment_transport

and changes to the hurricane and drying slope COMPASS test cases and init mode. Does not change forward mode simulations.

mark-petersen · 2020-03-24T03:39:21Z

@xylar and @pwolfram, we need some of these changes for COMPASS, so I thought we would try a merge. If it passes our normal tests, I'll go ahead and merge it. That is better than taking the small pieces we need.

@xylar, could you resolve the conflicts on

testing_and_setup/compass/README_ocean.md
testing_and_setup/compass/ocean/jigsaw_to_MPAS/coastal_tools.py

The resolve conflicts button above looks very easy. I can fix the other two, I'll take the ocean/coastal version but fix the indenting.

xylar · 2020-03-24T08:10:01Z

@mark-petersen and @pwolfram, all the conflicts were related to change I made so I have resolved them using resolve conflicts. This seems to have required a commit to merge ocean/develop into ocean/coastal, so maybe that's something we want to be careful about and do on the command line instead in the future?

pwolfram · 2020-04-02T01:49:59Z

@mark-petersen, when were you planning on doing this merge?

mark-petersen · 2020-04-02T12:51:56Z

In 2-4 weeks. There is no rush, it's just good practice to update occasionally. We have more pressing items to merge to ocean/develop right now.

xylar · 2020-04-02T13:17:57Z

@mark-petersen, was the basemap to cartopy change already merged somewhere else? Because I keep running into basemap on ocean/develop but maybe I haven't rebased and that's why. Anyway, that's one of the things that this PR would bring in.

pwolfram · 2020-04-02T14:00:26Z

@xylar, we need to at this point ensure that basemap is gone. If not, can you please fill out issues as you see them?

xylar · 2020-04-02T14:10:48Z

#504 should take care of this in ocean/develop. It's not a question of having missed some, it's the fact that #441 only went into ocean/coastal, not ocean/develop so far.

pwolfram · 2020-04-30T15:23:36Z

@mark-petersen, can we make sure #520 is in this PR? Plan to merge this week if you can afford to wait.

mark-petersen · 2020-05-04T11:39:02Z

There are other merges going in this week, so this PR will not go it right now. Feel free to update the coastal branch as needed.

xylar · 2020-06-01T18:42:00Z

Related to #550 and #577, when will be a good time for this merge to happen? @pwolfram and @mark-petersen?

xylar · 2020-07-21T11:44:01Z

It's been more than a month an a half since I pinged on this last. It's holding up my work. @sbrus89 and @mark-petersen, can we make a point of getting coastal/develop and ocean/develop synced up soon?

mark-petersen · 2020-08-12T17:45:24Z

@xylar and @sbrus89 this one is next in line for E3SM. Possible merge in the next few days. Please update with any recent changes.

xylar · 2020-08-12T18:05:55Z

Okay great!

xylar · 2020-08-12T18:06:31Z

Looks like we need a rebase?

sbrus89 · 2020-08-12T18:19:31Z

@xylar, I've re-rebased and am running some tests today.

xylar · 2020-08-12T18:20:14Z

Jinx!

sbrus89 · 2020-08-13T15:42:22Z

I'm currently sorting out some non-BFB differences after the latest rebase.

sbrus89 · 2020-08-14T20:11:12Z

@mark-petersen, I'm able to get BFB matches with debug on. Some of the differences with it off are bigger than I expected:

Hurricane case:
(I am concerned with the max differences here.)

['atmosphericPressure', 'kineticEnergyCell', 'layerThickness', 'ssh', 'windSpeedMagnitude', 'xtime']
testing atmosphericPressure
-- atmosphericPressure is bit identical.
testing kineticEnergyCell
-- ERROR kineticEnergyCell field is not bit-zero for min!
min   0.0 0.0
max   2.6658805796448863 2.6658767208070935
mean  0.0015096423405791648 0.0015096433392686555
min  diff  0.0
max  diff  0.020858563410840768
mean diff  1.300848115398338e-07
testing layerThickness
-- ERROR layerThickness field is not bit-zero for min!
min   0.0 0.0
max   60.05766648047755 60.05766661684283
mean  26.114428512582155 26.114428513244743
min  diff  0.0
max  diff  0.011561682297346986
mean diff  1.0587988321994339e-07
testing ssh
-- ERROR ssh field is not bit-zero for min!
min   -8.77784715062625 -8.777856817362744
max   10.030321191467657 10.030312193263235
mean  0.3486454752188095 0.34864554148394344
min  diff  0.0
max  diff  0.03468504689204055
mean diff  1.0587985468994867e-05
testing windSpeedMagnitude
-- windSpeedMagnitude is bit identical.

Drying slope case:
(these differences seem much more reasonable)

['angleEdge', 'areaCell', 'areaTriangle', 'bottomDepth', 'cellsOnCell', 'cellsOnEdge', 'cellsOnVertex', 'dcEdge', 'dvEdge', 'edgesOnCell', 'edgesOnEdge', 'edgesOnVertex', 'fCell', 'fEdge', 'fVertex', 'indexToCellID', 'indexToEdgeID', 'indexToVertexID', 'kiteAreasOnVertex', 'latCell', 'latEdge', 'latVertex', 'layerThickness', 'lonCell', 'lonEdge', 'lonVertex', 'maxLevelCell', 'meshDensity', 'nEdgesOnCell', 'nEdgesOnEdge', 'normalVelocity', 'refBottomDepth', 'restingThickness', 'salinity', 'ssh', 'temperature', 'tidalInputMask', 'tracer1', 'tracer2', 'tracer3', 'verticesOnCell', 'verticesOnEdge', 'weightsOnEdge', 'xCell', 'xEdge', 'xVertex', 'xtime', 'yCell', 'yEdge', 'yVertex', 'zCell', 'zEdge', 'zMid', 'zVertex']
testing angleEdge
-- angleEdge is bit identical.
testing areaCell
-- areaCell is bit identical.
testing areaTriangle
-- areaTriangle is bit identical.
testing bottomDepth
-- bottomDepth is bit identical.
testing cellsOnCell
-- cellsOnCell is bit identical.
testing cellsOnEdge
-- cellsOnEdge is bit identical.
testing cellsOnVertex
-- cellsOnVertex is bit identical.
testing dcEdge
-- dcEdge is bit identical.
testing dvEdge
-- dvEdge is bit identical.
testing edgesOnCell
-- edgesOnCell is bit identical.
testing edgesOnEdge
-- edgesOnEdge is bit identical.
testing edgesOnVertex
-- edgesOnVertex is bit identical.
testing fCell
-- fCell is bit identical.
testing fEdge
-- fEdge is bit identical.
testing fVertex
-- fVertex is bit identical.
testing indexToCellID
-- indexToCellID is bit identical.
testing indexToEdgeID
-- indexToEdgeID is bit identical.
testing indexToVertexID
-- indexToVertexID is bit identical.
testing kiteAreasOnVertex
-- kiteAreasOnVertex is bit identical.
testing latCell
-- latCell is bit identical.
testing latEdge
-- latEdge is bit identical.
testing latVertex
-- latVertex is bit identical.
testing layerThickness
-- ERROR layerThickness field is not bit-zero for min!
min   0.0 0.0
max   0.11050196508421674 0.1105019649790773
mean  0.02824951689551029 0.02824951688787687
min  diff  0.0
max  diff  1.156308749271362e-08
mean diff  1.284696455379052e-10
testing lonCell
-- lonCell is bit identical.
testing lonEdge
-- lonEdge is bit identical.
testing lonVertex
-- lonVertex is bit identical.
testing maxLevelCell
-- maxLevelCell is bit identical.
testing meshDensity
-- meshDensity is bit identical.
testing nEdgesOnCell
-- nEdgesOnCell is bit identical.
testing nEdgesOnEdge
-- nEdgesOnEdge is bit identical.
testing normalVelocity
-- ERROR normalVelocity field is not bit-zero for min!
min   -1e+34 -1e+34
max   2.2069732799790662 2.2069734295389245
mean  -2.861608844834164e+32 -2.861608844834164e+32
min  diff  0.0
max  diff  8.611260027047152e-06
mean diff  3.468266232559119e-08
testing refBottomDepth
-- refBottomDepth is bit identical.
testing restingThickness
-- restingThickness is bit identical.
testing salinity
-- ERROR salinity field is not bit-zero for min!
min   -1e+34 -1e+34
max   35.00000000004821 35.00000000004821
mean  -4.585714285714286e+33 -4.585714285714286e+33
min  diff  0.0
max  diff  1.1368683772161603e-12
mean diff  3.3547774671864714e-14
testing ssh
-- ERROR ssh field is not bit-zero for min!
min   -9.945894583899534 -9.945894583899534
max   0.07351375558951694 0.07351375485354014
mean  -2.5909731205882407 -2.5909731213515816
min  diff  0.0
max  diff  1.121619273902752e-06
mean diff  1.2846964640479065e-08
testing temperature
-- ERROR temperature field is not bit-zero for min!
min   -1e+34 -1e+34
max   20.000000000024585 20.000000000024585
mean  -4.585714285714286e+33 -4.585714285714286e+33
min  diff  0.0
max  diff  6.643574579356937e-13
mean diff  2.0025199297374744e-14
testing tidalInputMask
-- tidalInputMask is bit identical.
testing tracer1
-- ERROR tracer1 field is not bit-zero for min!
min   -1e+34 -1e+34
max   1.0000000000013645 1.0000000000013645
mean  -4.585714285714286e+33 -4.585714285714286e+33
min  diff  0.0
max  diff  3.419486915845482e-14
mean diff  8.589558719946789e-16
testing tracer2
-- ERROR tracer2 field is not bit-zero for min!
min   -1e+34 -1e+34
max   1.0000000000013645 1.0000000000013645
mean  -4.585714285714286e+33 -4.585714285714286e+33
min  diff  0.0
max  diff  3.419486915845482e-14
mean diff  8.589558719946789e-16
testing tracer3
-- ERROR tracer3 field is not bit-zero for min!
min   -1e+34 -1e+34
max   1.0000000000013645 1.0000000000013645
mean  -4.585714285714286e+33 -4.585714285714286e+33
min  diff  0.0
max  diff  3.419486915845482e-14
mean diff  8.589558719946789e-16
testing verticesOnCell
-- verticesOnCell is bit identical.
testing verticesOnEdge
-- verticesOnEdge is bit identical.
testing weightsOnEdge
-- weightsOnEdge is bit identical.
testing xCell
-- xCell is bit identical.
testing xEdge
-- xEdge is bit identical.
testing xVertex
-- xVertex is bit identical.
testing yCell
-- yCell is bit identical.
testing yEdge
-- yEdge is bit identical.
testing yVertex
-- yVertex is bit identical.
testing zCell
-- zCell is bit identical.
testing zEdge
-- zEdge is bit identical.
testing zMid
-- ERROR zMid field is not bit-zero for min!
min   -10.045394683399486 -10.045394683399486
max   0.01826277304740863 0.018262772364001423
mean  -4.381724764843089 -4.381724765206794
min  diff  0.0
max  diff  1.1158377883901949e-06
mean diff  4.734809050939188e-09
testing zVertex
-- zVertex is bit identical.

mark-petersen · 2020-08-17T23:00:11Z

@sbrus89 how long does your hurricane case run here, where you see 0.02 differences in kinetic energy?

sbrus89 · 2020-08-18T01:21:18Z

The case takes 3.5 hours on 72 cores. I'm checking to see where in the domain the largest differences are.

sbrus89 · 2020-08-18T01:31:19Z

The differences are confined to the high-resolution region:

It seems like the most obvious candidate for the differences is wetting and drying, but the drying slope case seems to check out okay...

SedimentFluxIndex - I SedimentFluxIndex -I add the following files - Registry_sediment_flux_index.xml - mpas_con_sediment_flux_index.F sedimentFluxIndex - I update add four variables - sedimentFluxIndexVAX : Vertically-Averaged sFI in X-direction - sedimentFluxIndexVAY : Vertically-Averaged sFI in Y-direction - sedimentFluxIndexBX: Bottom sFI in X-direction - sedimentFluxIndexBY: Bottom sFI in Y-direction Adds sediment flux calculation calculate sediment fall velocity by Goldstein and Coco 2013 calculate ratio "a" of sediment flux Q and U3 by Grawe et al 2014 calculate vertically-averaged and close bottom sediment flux (kg/m/s) test the AM on the case ziso/2.5km/default Add Soulsby and Damgaard bedload transport formulae Goldstein and Coco's fall velocity equation seems to overestimate Ws, might be repaced by VanRijn [1989] or Zhu & Cheng [1993] in next commit

Note, time step may be too conservative and it may be possible to increase dt.

mark-petersen · 2020-08-21T13:01:31Z

Tested on grizzly with both gnu and intel, debug and optimized. Passes nightly regression suite, and was bfb with ocean/develop, as expected, because this PR does not change forward mode other than for coastal test cases.

@sbrus89 I am comfortable with the differences you saw above after your recent rebase. The performance changes in #513 make non-bfb changes with compiler optimization that can lead to larger changes after a long run. The fact that you got BFB match with debug on means it's the same differences we had seen when testing #513.

mark-petersen · 2020-08-21T13:03:03Z

@sbrus89 and @xylar please approve when ready. I can merge today and set up the E3SM PR.

xylar · 2020-08-21T13:06:37Z

@mark-petersen, I think we need to figure out #658 first and what broke it.

sbrus89

@mark-petersen, I approve based on my testing of the coastal test cases.

xylar · 2020-08-21T14:10:20Z

In the future, I don't think we want to be rebasing or editing commit history for any branch that has PRs into it, including ocean/coastal. The connection between the commits and the PRs here has now been lost. That will make debugging issues in the future potentially more challenging. Instead, I think conflicts need to be solved only with additional commits.

mark-petersen · 2020-08-21T14:16:50Z

@xylar thanks for your comment. @sbrus89 and I saw that yesterday, and agree with you.

In the future we will merge ocean/develop back into this branch to resolve merge conflicts. Another option is to rebase with --preserve-merges, but then the commit hashes listed on PR pages get changed.

I'd like to leave this one be, as it is not worthwhile changing now. We will leave the merges in, in the future.

xylar · 2020-08-21T14:34:19Z

I looked through the code (only skimming the many COMPASS additions) and it looks okay to me. I just want to double check that you are 100% sure none of the changes in shared are on by default in E3SM.

I ran the QU240 init, test and daily_output test cases and all looks good, but I'm relying on @mark-petersen to do the bit-for-bit comparison with ocean/develop there. Am I right that you will be doing that?

I would like to make sure the SOwISC12to60, ECwISC30to60 and QUwISC240 work after these changes, which means I need to wait on the bug fix that @mark-petersen is working on, since I don't think they work before these changes at the moment. Once that's sorted out, I will approve.

xylar

I'm approving based on tests without ice-shelf cavities (and the promise that the wISC tests shouldn't be affected).

mark-petersen · 2020-08-21T15:28:11Z

@sbrus89 and @caozd999 let's consider the ocean/coastal branch closed for a week, i.e. no more PRs until #563 and a few other things are sorted out. Then we can reset ocean/coastal to ocean/develop, and your existing coastal PR branches will need to be rebased on that.

* ocean/develop: (59 commits) Merged ocean/coastal. See PR #485

…n/develop fix Rayleigh drag logic on implicit vertical mix #676 Fix if statement for Rayleigh drag introduced in #485. When Rayleigh drag was on, it turned off implicit vertical mixing. We only use Rayleigh drag for spin-up, so this does not change normal forward model runs or E3SM simulations.

…nto ocean/develop fix Rayleigh drag logic on implicit vertical mix MPAS-Dev#676 Fix if statement for Rayleigh drag introduced in MPAS-Dev#485. When Rayleigh drag was on, it turned off implicit vertical mixing. We only use Rayleigh drag for spin-up, so this does not change normal forward model runs or E3SM simulations.

mark-petersen added Ocean COMPASS coastal labels Mar 24, 2020

mark-petersen requested review from pwolfram and xylar March 24, 2020 03:30

mark-petersen self-assigned this Mar 24, 2020

sbrus89 force-pushed the ocean/coastal branch from 1d6da01 to b88a37f Compare August 3, 2020 20:45

sbrus89 force-pushed the ocean/coastal branch from 5f1875b to f53d3f0 Compare August 14, 2020 22:02

pwolfram and others added 6 commits August 19, 2020 11:37

Adjusts time for synthetic hurricane definition

c2acaad

Adds 500m Delaware hurricane case

e4849bd

Adds 250m Delaware hurricane case

79d3606

Adds 100m Delaware hurricane case

863db49

Note, time step may be too conservative and it may be possible to increase dt.

Updates interpolation file to avoid memory errors

848d1a7

Modify marsh_flooding test case files for merge

acea49c

sbrus89 force-pushed the ocean/coastal branch from 025c036 to acea49c Compare August 19, 2020 18:00

sbrus89 mentioned this pull request Aug 19, 2020

Adds free surface gradient forcing for idealized 1D turbulence #615

Closed

xylar mentioned this pull request Aug 21, 2020

Move jigsaw_to_MPAS to mpas_tools #563

Merged

mark-petersen requested review from sbrus89 and removed request for pwolfram August 21, 2020 12:46

sbrus89 approved these changes Aug 21, 2020

View reviewed changes

xylar approved these changes Aug 21, 2020

View reviewed changes

mark-petersen merged commit 5e71e29 into ocean/develop Aug 21, 2020

mark-petersen added a commit that referenced this pull request Aug 21, 2020

Merge branch 'ocean/develop' into e3sm/develop

8efdce0

* ocean/develop: (59 commits) Merged ocean/coastal. See PR #485

xylar mentioned this pull request Aug 24, 2020

Add prerequisite tags to all necessary test cases #560

Closed

sbrus89 mentioned this pull request Aug 24, 2020

Fix eighteen broken coastal test cases #662

Merged

mark-petersen mentioned this pull request Aug 29, 2020

fix Rayleigh drag logic on implicit vertical mix #676

Merged

matthewhoffman added coastal COMPASS Ocean labels Mar 17, 2021

Conversation

mark-petersen commented Mar 24, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mark-petersen commented Mar 24, 2020

Uh oh!

xylar commented Mar 24, 2020

Uh oh!

pwolfram commented Apr 2, 2020

Uh oh!

mark-petersen commented Apr 2, 2020

Uh oh!

xylar commented Apr 2, 2020

Uh oh!

pwolfram commented Apr 2, 2020

Uh oh!

xylar commented Apr 2, 2020

Uh oh!

pwolfram commented Apr 30, 2020

Uh oh!

mark-petersen commented May 4, 2020

Uh oh!

xylar commented Jun 1, 2020

Uh oh!

xylar commented Jul 21, 2020

Uh oh!

mark-petersen commented Aug 12, 2020

Uh oh!

xylar commented Aug 12, 2020

Uh oh!

xylar commented Aug 12, 2020

Uh oh!

sbrus89 commented Aug 12, 2020

Uh oh!

xylar commented Aug 12, 2020

Uh oh!

sbrus89 commented Aug 13, 2020

Uh oh!

sbrus89 commented Aug 14, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mark-petersen commented Aug 17, 2020

Uh oh!

sbrus89 commented Aug 18, 2020

Uh oh!

sbrus89 commented Aug 18, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mark-petersen commented Aug 21, 2020

Uh oh!

mark-petersen commented Aug 21, 2020

Uh oh!

xylar commented Aug 21, 2020

Uh oh!

sbrus89 left a comment

Choose a reason for hiding this comment

Uh oh!

xylar commented Aug 21, 2020

Uh oh!

mark-petersen commented Aug 21, 2020

Uh oh!

xylar commented Aug 21, 2020

Uh oh!

xylar left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mark-petersen commented Aug 21, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

mark-petersen commented Mar 24, 2020 •

edited

Loading

sbrus89 commented Aug 14, 2020 •

edited

Loading

sbrus89 commented Aug 18, 2020 •

edited

Loading

xylar left a comment •

edited

Loading