Skip to content

New Mesh: EC30to60kmL60E3SMv2r02 and change to 60 layers as default#668

Merged
mark-petersen merged 3 commits intoMPAS-Dev:ocean/developfrom
vanroekel:ocean/updateEC6030_default_layers
Sep 4, 2020
Merged

New Mesh: EC30to60kmL60E3SMv2r02 and change to 60 layers as default#668
mark-petersen merged 3 commits intoMPAS-Dev:ocean/developfrom
vanroekel:ocean/updateEC6030_default_layers

Conversation

@vanroekel
Copy link
Contributor

@vanroekel vanroekel commented Aug 26, 2020

This PR bumps the revision number on the new 6030 grid, and includes all improvements to islands, channels, and passages for v2 of the E3SM water cycle campaign.

The default number of vertical layers is also switched to 60.

@vanroekel vanroekel requested a review from xylar August 26, 2020 19:51
@vanroekel vanroekel marked this pull request as draft August 26, 2020 19:52
@xylar xylar changed the title EC6030r2 and changes to 60 layers as default New Mesh: EC30to60kmL60E3SMv2r02 and change to 60 layers as default Aug 26, 2020
@xylar
Copy link
Collaborator

xylar commented Aug 26, 2020

Looks good so far. If you make config_initial_state.xml a copy rather than a symlink, you can edit it to change the initial condition data set to be the 1900 MetOffice data set you mentioned.

@vanroekel vanroekel mentioned this pull request Aug 26, 2020
Merged
@xylar
Copy link
Collaborator

xylar commented Aug 26, 2020

If you make config_initial_state.xml a copy rather than a symlink, you can edit it to change the initial condition data set to be the 1900 MetOffice data set you mentioned.

Oh, right, that's for r04.

@mark-petersen
Copy link
Contributor

mark-petersen commented Aug 26, 2020

@vanroekel FYI I was testing the current EC60to30 and the prep_spin_up1 step was having some trouble crashing. I'm testing with a smaller timestep now.

@vanroekel vanroekel mentioned this pull request Aug 27, 2020
Closed
@mark-petersen
Copy link
Contributor

mark-petersen commented Aug 27, 2020

@vanroekel I tried a shorter time step and larger Rayleigh damping, but the EC60to30 still crashes in prep_spin_up1. Did you run that in the last few weeks for your recent low-res test, and did you adjust something to get it started?

@vanroekel
Copy link
Contributor Author

vanroekel commented Aug 27, 2020

@mark-petersen I don't remember changing anything for pre_spinup1. I'll try today with your culled mesh and let you know.

@mark-petersen
Copy link
Contributor

mark-petersen commented Aug 27, 2020

I'll work on it today too. It's not clear if this is a settings problem or actual bug.

@vanroekel
Copy link
Contributor Author

vanroekel commented Aug 27, 2020

@mark-petersen when does the 6030 pre_spinup1 crash and what is the error output? I just started that run and it has run 1 day so far.

@vanroekel
Copy link
Contributor Author

vanroekel commented Aug 27, 2020

@xylar I think I have the sequence of PRs for the new mesh as close as I can for now. ping me when you are ready to work on these and I can jump back in.

@xylar
Copy link
Collaborator

xylar commented Aug 27, 2020

@vanroekel, the plan of swapping the symlinks for 60 and 64 layers isn't going to work quite right. You also renamed the steps in their respective files in testing_and_setup/compass/ocean/global_ocean/config_files/, which is going to break other test cases that use these symlinks.

My suggestion is just to make copies of these files and modify them in the test case itself.

@vanroekel
Copy link
Contributor Author

vanroekel commented Aug 27, 2020

@xylar good point, it made the other PRs easier and seems much safer here. I'll make the fix.

@vanroekel vanroekel force-pushed the ocean/updateEC6030_default_layers branch from 7ba231c to b96ee2b Compare August 27, 2020 19:24
@xylar
Copy link
Collaborator

xylar commented Aug 27, 2020

I tested a rebase of this on #577 and there were no conflicts (unsurprisingly). I also don't think that PR will change the results here, those are organizational changes that should be bit-for-bit. So I think this can proceed without worrying about those changes. Likewise, it shouldn't matter if we use the current compass environment (0.1.9) or the one I'm working on for that PR (0.1.10) because the changes are only organizational.

@xylar
Copy link
Collaborator

xylar commented Aug 27, 2020

@vanroekel, could you rebase onto the current ocean/develop following @mark-petersen's recent merges? You shouldn't have any conflicts but it will make sure your build is compatible with the latest compass and has the fixes to the plotting script in initial_state that were giving me trouble before.

@vanroekel vanroekel force-pushed the ocean/updateEC6030_default_layers branch from b96ee2b to c06e15e Compare August 27, 2020 20:52
@vanroekel
Copy link
Contributor Author

vanroekel commented Aug 27, 2020

Of course., done.

This was referenced Sep 1, 2020
Merged
Merged
@vanroekel vanroekel marked this pull request as ready for review September 3, 2020 21:47
@vanroekel
Copy link
Contributor Author

vanroekel commented Sep 3, 2020

@mark-petersen and @xylar any update on when this will be ready so I can spin up the mesh for WC runs?

@mark-petersen
Copy link
Contributor

mark-petersen commented Sep 3, 2020

I will merge #577 and #680 in a bit. Xylar will rebase and check over these mesh PRs, and we can merge tomorrow morning.

@vanroekel
Copy link
Contributor Author

vanroekel commented Sep 4, 2020

okay great. Thanks for the update.

@xylar
Copy link
Collaborator

xylar commented Sep 4, 2020

@vanroekel, I'm getting started on this as soon as I install an update compass environment on all our machines.

This was referenced Sep 4, 2020
Merged
Merged
xylar
xylar approved these changes Sep 4, 2020
View reviewed changes
Copy link
Collaborator

@xylar xylar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I merged this with #676 and ocean/develop and tested on Grizzly with Intel and SCORPIO. The spin-up went fine. Results are here:

/lustre/scratch4/turquoise/xylar/test_EC30to60kmL60E3SMv2r02/

I verified that the e3sm_coupling step ran fine, showing the expected number of layers an mesh name:

- ice_shelf_cavities set to False
- author's name autodetected from git config: Xylar Asay-Davis
- author's name specified in config file: xylar@lanl.gov
- date string autodetected from today's date: 200904
- creation date autodetected from today's date: 09/04/2020 06:07:38
- maximum ocean depth autodetected mesh file: 5500.0
- number of vertical levels in the ocean autodetected mesh file: 60
- mesh long name specified in config file: EC30to60kmL60E3SMv2r02
- mesh short name specified in config file: EC30to60E2r02

Here are a few images showing the resulting T, S, bottom depth and maxLevelCell as sanity checks:
temperature
salinity
bottomDepth
maxLevelCell

@xylar
Copy link
Collaborator

xylar commented Sep 4, 2020

I'm rerunning with gnu to see if I get the same problems above as @vanroekel. I'm also moving on to #669

@xylar xylar force-pushed the ocean/updateEC6030_default_layers branch from c06e15e to f3f4438 Compare September 4, 2020 16:02
@mark-petersen
Copy link
Contributor

mark-petersen commented Sep 4, 2020

tested with gnu 6.4.0 optimized on badger. Results are here:

/lustre/scratch3/turquoise/mpeterse/runs/200904_EC30to60kmL60E3SMv2r02_PR668_final/ocean/global_ocean/EC60to30

ran through prep_spin_up1 (10 days) and simulation (10 more days).

mesh and IC histogram images:

cellWidthGlobal
initial_state

@mark-petersen
Copy link
Contributor

mark-petersen commented Sep 4, 2020

mesh images

image

image
image
image

T&S after 20 day spinup

image
image
image
image

these are at

/lustre/scratch3/turquoise/mpeterse/runs/200904_EC30to60kmL60E3SMv2r02_PR668_final/ocean/global_ocean/EC60to30/spin_up/simulation/vtk*

@mark-petersen mark-petersen self-requested a review September 4, 2020 17:40
mark-petersen
mark-petersen approved these changes Sep 4, 2020
View reviewed changes
Copy link
Contributor

@mark-petersen mark-petersen left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything looks good from init and spin-up on badger with gnu 6.4.0 and compass_0.1.10

@mark-petersen mark-petersen merged commit f9e3d78 into MPAS-Dev:ocean/develop Sep 4, 2020
@xylar
Copy link
Collaborator

xylar commented Sep 4, 2020 •
edited
Loading

After @mark-petersen merges this, I'm going to rebase #669 so the PR is cleaner. Is that okay, @vanroekel?

Update: I'm going to leave this to @vanroekel.

@mark-petersen mark-petersen mentioned this pull request Sep 8, 2020
Merged
mark-petersen added a commit that referenced this pull request Sep 9, 2020
@mark-petersen
Merge branch 'ocean/develop' into e3sm/develop
b61d423 
New Mesh: EC30to60kmL60E3SMv2r02 and change to 60 layers as default #668
New mesh: EC30to60kmL60E3SMv2r03 and change to gebco bathymetry #669
Fix validation for QU240 test step #660
Change block tests to partition tests for QU240 RK4 and SE #657
Update to version 0.1.11 of the compass environment #688
@mark-petersen mark-petersen mentioned this pull request Sep 9, 2020
Merged
jonbob added a commit to E3SM-Project/E3SM that referenced this pull request Sep 9, 2020
@jonbob
Merge branch 'mark-petersen/update_mpas_200909' into next (PR #3818)
7bab0e0 
Update MPAS-Source: COMPASS and documentation only

This PR brings in a new mpas-source submodule with changes only to the
ocean core. The changes are limited to internal coupling and
documentation so it should have no impact on E3SM. It includes:
* new EC30to60kmL60E3SMv2r02 mesh with 60 layers as default
  (MPAS-Dev/MPAS-Model/pull/668)
* new EC30to60kmL60E3SMv2r03 mesh with change to gebco bathymetry
  (MPAS-Dev/MPAS-Model/pull/669)
* switch from Travis-CI to Azure Pipelines
  (MPAS-Dev/MPAS-Model/pull/681)
* fix validation for QU240 test step
  (MPAS-Dev/MPAS-Model/pull/660)
* change block tests to partition tests for QU240 RK4 and SE
  (MPAS-Dev/MPAS-Model/pull/657)
* update to version 0.1.11 of the compass environment
  (MPAS-Dev/MPAS-Model/pull/688)

[BFB]
jonbob added a commit to E3SM-Project/E3SM that referenced this pull request Sep 10, 2020
@jonbob
Merge branch 'mark-petersen/update_mpas_200909' (PR #3818)
85999dc 
pdate MPAS-Source: COMPASS and documentation only

This PR brings in a new mpas-source submodule with changes only to the
ocean core. The changes are limited to internal coupling and
documentation so it should have no impact on E3SM. It includes:
* new EC30to60kmL60E3SMv2r02 mesh with 60 layers as default
  (MPAS-Dev/MPAS-Model/pull/668)
* new EC30to60kmL60E3SMv2r03 mesh with change to gebco bathymetry
  (MPAS-Dev/MPAS-Model/pull/669)
* switch from Travis-CI to Azure Pipelines
  (MPAS-Dev/MPAS-Model/pull/681)
* fix validation for QU240 test step
  (MPAS-Dev/MPAS-Model/pull/660)
* change block tests to partition tests for QU240 RK4 and SE
  (MPAS-Dev/MPAS-Model/pull/657)
* update to version 0.1.11 of the compass environment
  (MPAS-Dev/MPAS-Model/pull/688)

[BFB]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mark-petersen mark-petersen mark-petersen approved these changes

@xylar xylar xylar approved these changes

Assignees

No one assigned

Labels

COMPASS mesh Ocean

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@vanroekel @xylar @mark-petersen @matthewhoffman