New Mesh: EC15to60E2r4 #740

xylar · 2020-11-03T22:33:14Z

Long Name: EC15to60kmL60E3SMv2r04

MPAS Eddy Closure mesh for E3SM version2 with enhanced resolution around the equator (30 km), South pole (35 km), Greenland (15 km ), 60-km resolution at mid latitudes, and 60 vertical levels. Bathymetry is from GEBCO 2019, combined with BedMachineAntarctica around Antarctica. The default initial condition is from EN4 1900, but PHC is also an option.

The motivation for this new mesh is enhanced resolution around Greenland, similar to the EC15to60E2r1 mesh (#522), that should hopefully increase the strength of the AMOC, which was weak in E3SM v1 and is even weaker in E3SM v2.

xylar · 2020-11-03T22:38:26Z

I think I should rebase this to take out all the EC30to60 changes.

xylar · 2020-11-03T22:58:29Z

First draft

From my laptop:

vanroekel · 2020-11-03T23:08:58Z

Thanks @xylar this is looking great. Thanks for pulling this together so quickly. @maltrud @proteanplanet @milenaveneziani @lconlon could you take a look as well?

xylar · 2020-11-03T23:12:31Z

I will post some plots of the culled mesh as soon as I have it.

proteanplanet · 2020-11-03T23:22:09Z

I will post some plots of the culled mesh as soon as I have it.

Please point me to the mesh file when you have it.

xylar · 2020-11-03T23:25:27Z

Please point me to the mesh file when you have it.

I'll do that. Grizzly is down (or at least I can't log in) so I'll be making the mesh on Anvil. Hopefully, the fact that there's no space won't kill this work...

xylar · 2020-11-04T03:50:49Z

Results on Anvil

The E3SM files with metadata are not quite ready yet. The file system seems unusually slow, perhaps because the disk is quite full?

The culled mesh is here:
/lcrc/group/acme/ac.xylar/mpas/spinup_EC15to60E2r4/ocean/global_ocean/EC15to60/build_mesh/culled_mesh/culled_mesh.nc

The initial state (before any spin-up) is here:
/lcrc/group/acme/ac.xylar/mpas/spinup_EC15to60E2r4/ocean/global_ocean/EC15to60/spin_up_EN4_1900/initial_state/initial_state.nc

The restart file after a 1-hour spin-up is here:
/lcrc/group/acme/ac.xylar/mpas/spinup_EC15to60E2r4/ocean/global_ocean/EC15to60/spin_up_EN4_1900/simulation/restarts/restart.0001-01-01_01.00.00.nc

I'll update here once the ocean/sea ice data in the format needed for E3SM and with metadata for the mesh are ready.

Update: The ocean and sea-ice initial conditions are now also ready, available here:
/lcrc/group/acme/ac.xylar/mpas/spinup_EC15to60E2r4/ocean/global_ocean/EC15to60/files_for_e3sm/files_for_e3sm/assembled_files_for_upload/

proteanplanet · 2020-11-04T04:16:22Z

@xylar Please can you change permissions on the initial_state.nc file above?

xylar · 2020-11-04T04:24:38Z

@proteanplanet, the permissions look correct to me:

$ ls -lah -lah initial_state.nc 
-rw-r-----+ 1 ac.xylar cels 2.5G Nov  3 20:59 initial_state.nc

I'm guessing the problem was that I use a link from my home directory to get to /lcrc/group/acme/ac.xylar/ and you won't have access to that. I changed the paths above to the absolute path without that link. Could you see if you have access to that path?

proteanplanet · 2020-11-04T05:06:51Z

@xylar I can't work this out. I can't access /lcrc/group/acme/ac.xylar, but can access other of my colleague's directories at the same place. We are in the same groups. If you can copy the initial_state.nc to /lcrc/group/acme/ac.afroberts, please do.

xylar · 2020-11-04T05:26:27Z

@proteanplanet, I changed the group for /lcrc/group/acme/ac.xylar from climate to cels, so maybe that will help.

Just in case, I also copied initial_state.nc to: /lcrc/group/acme/ac.afroberts/EC15to60E2r4_initial_state.nc.

xylar · 2020-11-04T05:27:07Z

It is also 6:30 am and I'm becoming a bit delirious so I don't think it's safe for me to work on this anymore.

proteanplanet · 2020-11-04T07:00:47Z

Looks good to me. Glad you went to bed @xylar

jonbob · 2020-11-04T16:42:06Z

@xylar - please let me know when you feel like there's a mesh file I can start making mapping files from. And thanks for all your hard work!

xylar · 2020-11-04T17:23:20Z

@jonbob, the info you need should be in place. Hopefully, what you need is in:

/lcrc/group/acme/ac.xylar/mpas/spinup_EC15to60E2r4/ocean/global_ocean/EC15to60/files_for_e3sm/files_for_e3sm/assembled_files_for_upload/

Please let me know if that isn't what you need.

jonbob · 2020-11-04T17:24:53Z

All I really need is the initial file -- I can make everything else from that. So I'll get started!

xylar · 2020-11-04T17:43:31Z

@jonbob, it would be great if the files that end up on the LCRC server have the metadata that gets added during the files_for_e3sm step form COMPASS. Do you know if that metadata is preserved in the operations that you do?
This is the metadata that I mean:

		:MPAS_Mesh_Short_Name = "EC15to60E2r04" ;
		:MPAS_Mesh_Long_Name = "EC15to60kmL60E3SMv2r04" ;
		:MPAS_Mesh_Description = "MPAS Eddy Closure mesh for E3SM versio
n 2 with enhanced resolution around the equator (30 km), South pole (35 km), Gre
enland (15 km), 60-km resolution at mid latitudes, and 60 vertical levels.  Bath
ymetry is from GEBCO 2019, combined with BedMachineAntarctica around Antarctica.
 The initial condition is from EN4 1900.  The mesh is documented at https://gith
ub.com/MPAS-Dev/MPAS-Model/pull/740" ;
		:MPAS_Mesh_E3SM_Version = 2 ;
		:MPAS_Mesh_EC_Version = "04" ;
		:MPAS_Mesh_EC_Version_Author = "Xylar Asay-Davis" ;
		:MPAS_Mesh_EC_Version_Author_E-mail = "xylar@lanl.gov" ;
		:MPAS_Mesh_EC_Version_Creation_Date = "11/03/2020 21:34:39" ;
		:MPAS_Mesh_EC_Minimum_Resolution_km = 15. ;
		:MPAS_Mesh_EC_Maximum_Resolution_km = 60. ;
		:MPAS_Mesh_EC_Maximum_Depth_m = 5500. ;
		:MPAS_Mesh_EC_Number_of_Levels = 60. ;
		:MPAS_Mesh_Ice_Shelf_Cavities = "OFF" ;
		:MPAS_Mesh_Runoff_Description = "<<<Spreading function described
 here>>>" ;
		:MPAS_Mesh_COMPASS_Version = "0.1.11" ;
		:MPAS_Mesh_JIGSAW_Version = "0.9.12" ;
		:MPAS_Mesh_JIGSAW-Python_Version = "0.2.1" ;
		:MPAS_Mesh_MPAS-Tools_Version = "0.0.13" ;
		:MPAS_Mesh_NCO_Version = "4.9.4" ;
		:MPAS_Mesh_ESMF_Version = "8.0.1" ;
		:MPAS_Mesh_geometric_features_Version = "0.1.11" ;
		:MPAS_Mesh_Metis_Version = "5.1.0" ;
		:MPAS_Mesh_pyremap_Version = "0.0.7" ;

jonbob · 2020-11-04T17:49:30Z

@xylar - I'll check and modify my workflow if needed to get the metadata. For what it's worth, your version of the tool to create the SCRIP file still has issues with missing values in the grid_imask field?

jonbob · 2020-11-04T17:52:40Z

@xylar - that metadata is not included in the SCRIP files, at least not automatically. So it would be lost for all mapping files

proteanplanet

This is approved.

Note: The resolution transition through the Archileago is barely noticable:

For interest only, with no meaningful impact on the applications of this particular mesh except for show and tell purposes, updates to geometric features since previous meshes now includes:

The Hawaiin island chain is reliably represented:

British Isles more accurately resolved:

And Italy reliably with a boot:

xylar · 2020-11-04T19:06:08Z

I'll check and modify my workflow if needed to get the metadata. For what it's worth, your version of the tool to create the SCRIP file still has issues with missing values in the grid_imask field?

Oh, no! I think that many simply be because we haven't had a release of MPAS-Tools since the bug fix but that's a good reminder that we need one! Sorry about that!

that metadata is not included in the SCRIP files, at least not automatically. So it would be lost for all mapping files

That's a good point. I believe I can add it but I hadn't been. But I don't think it's critical that it be included in the mapping files. What really matters is if it propagates to E3SM output files. And that may depend more on MPAS-Model than on anything we do. The main thing is to make sure it's in whatever initial condition files get put on the LCRC server. I'm not sure if that's you, @jonbob or @vanroekel or someone else...

jonbob · 2020-11-04T19:15:02Z

@xylar - OK, I'll use my version of the SCRIP file, so no problem. And I won't worry about metadata in the mapping files, which means I can get started right away. And we'll pay attention to the ocean initial file that gets committed. Thanks again for everything

testing_and_setup/compass/ocean/global_ocean/EC15to60/build_mesh/build_base_mesh.py

This version has a larger region of enhanced resolution around Greenland than EC15to60E2r01.

mark-petersen

I tested on grizzly. Thanks!

New Mesh: EC15to60E2r4 #740

Brings in a new mpas-source submodule with changes only to COMPASS in the ocean core. It adds mesh generation for EC15to60kmL60E3SMv2r04. Because COMPASS is not used in E3SM, it does not change simulation code. See MPAS-Dev/MPAS-Model#740 [BFB]

COMPASS only, New Mesh: EC15to60E2r4 Brings in a new mpas-source submodule with changes only to COMPASS in the ocean core. It adds mesh generation for EC15to60kmL60E3SMv2r04. Because COMPASS is not used in E3SM, it does not change simulation code. See MPAS-Dev/MPAS-Model#740 [BFB]

xylar changed the base branch from master to ocean/develop November 3, 2020 22:33

xylar requested review from mark-petersen, sbrus89 and vanroekel November 3, 2020 22:34

xylar assigned vanroekel Nov 3, 2020

xylar added COMPASS mesh Ocean labels Nov 3, 2020

xylar force-pushed the ocean/EC15to60E2r4 branch 2 times, most recently from 251bc4d to 78be72e Compare November 3, 2020 22:42

vanroekel requested review from lconlon, maltrud, milenaveneziani and proteanplanet November 3, 2020 23:08

xylar force-pushed the ocean/EC15to60E2r4 branch 3 times, most recently from 6c30b31 to 1a607cf Compare November 4, 2020 03:17

proteanplanet approved these changes Nov 4, 2020

View reviewed changes

maltrud approved these changes Nov 4, 2020

View reviewed changes

mark-petersen reviewed Nov 5, 2020

View reviewed changes

testing_and_setup/compass/ocean/global_ocean/EC15to60/build_mesh/build_base_mesh.py Outdated Show resolved Hide resolved

xylar added 2 commits November 5, 2020 19:51

Add a new version of the EC15to60 mesh

bc523be

This version has a larger region of enhanced resolution around Greenland than EC15to60E2r01.

Move high res 5 degrees further north

0ac2e6d

xylar force-pushed the ocean/EC15to60E2r4 branch from 1a607cf to 0ac2e6d Compare November 5, 2020 18:51

mark-petersen assigned mark-petersen and unassigned vanroekel Nov 5, 2020

mark-petersen approved these changes Nov 5, 2020

View reviewed changes

mark-petersen merged commit 30ca470 into MPAS-Dev:ocean/develop Nov 5, 2020

mark-petersen added a commit that referenced this pull request Nov 5, 2020

Merge 'ocean/develop' into e3sm/develop

677304d

New Mesh: EC15to60E2r4 #740

mark-petersen mentioned this pull request Nov 5, 2020

COMPASS only, New Mesh: EC15to60E2r4 E3SM-Project/E3SM#3933

Merged

This was referenced Nov 6, 2020

New Mesh: EC30to60kmL60E3SMv2r04 and add 1900 state estimate initial condition #671

Closed

Geojson files for EC15to60 are being linked in EC30to60 as well MPAS-Dev/compass#19

Closed

mark-petersen mentioned this pull request Nov 9, 2020

Merge COMPASS contents from MPAS-Dev/MPAS-Model/ocean/develop MPAS-Dev/compass#2

Merged

xylar mentioned this pull request Nov 12, 2020

New Mesh: ECwISC30to60kmL60E3SMv2r01 and add 1900 state estimate initial condition MPAS-Dev/compass#15

Merged

xylar deleted the ocean/EC15to60E2r4 branch November 25, 2020 13:11

matthewhoffman added COMPASS mesh Ocean labels Mar 17, 2021

New Mesh: EC15to60E2r4 #740

New Mesh: EC15to60E2r4 #740

Uh oh!

Conversation

xylar commented Nov 3, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

xylar commented Nov 3, 2020

Uh oh!

xylar commented Nov 3, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

First draft

Uh oh!

vanroekel commented Nov 3, 2020

Uh oh!

xylar commented Nov 3, 2020

Uh oh!

proteanplanet commented Nov 3, 2020

Uh oh!

xylar commented Nov 3, 2020

Uh oh!

xylar commented Nov 4, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Results on Anvil

Uh oh!

proteanplanet commented Nov 4, 2020

Uh oh!

xylar commented Nov 4, 2020

Uh oh!

proteanplanet commented Nov 4, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

xylar commented Nov 4, 2020

Uh oh!

xylar commented Nov 4, 2020

Uh oh!

proteanplanet commented Nov 4, 2020

Uh oh!

jonbob commented Nov 4, 2020

Uh oh!

xylar commented Nov 4, 2020

Uh oh!

jonbob commented Nov 4, 2020

Uh oh!

xylar commented Nov 4, 2020

Uh oh!

jonbob commented Nov 4, 2020

Uh oh!

jonbob commented Nov 4, 2020

Uh oh!

proteanplanet left a comment

Choose a reason for hiding this comment

Uh oh!

xylar commented Nov 4, 2020

Uh oh!

jonbob commented Nov 4, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

mark-petersen left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

10 participants

xylar commented Nov 3, 2020 •

edited

Loading

xylar commented Nov 3, 2020 •

edited

Loading

xylar commented Nov 4, 2020 •

edited

Loading

proteanplanet commented Nov 4, 2020 •

edited

Loading

jonbob commented Nov 4, 2020 •

edited

Loading