Building CESM 1.2.2.1 Tetralith

This page describes the building the CESM 1.2.2.1 on NSC's Tetralith system. Tetralith is NSC's largest HPC cluster. It replaced NSC's previous HPC cluster Triolith in 2018. Tetralith is funded by SNIC and uses for research by Swedish research groups. Access to Tetralith is granted by SNIC.

CESM1 unix group on triolith

If you are going to run one of the CESM models on Tetralith, you should join the 'cesm_input-data' storage project (snic2014-5-61). This is not absolutely necessary but being a member allows access to shared storage space (/proj/cesm_input-data) where pre-downloaded initial data and ancillary files are stored. To join the 'cesm_input-data' project, log into SUPR (https://supr.snic.se/) and click the 'View and Manage Projects' button, the request membership to the snic2014-5-61 project.

Getting the source code

CESM is a free, open source model, but a short registration is required before access to the subversion repository is granted. The source code, build and run scripts can be downloaded as described in the CESM1.2 user guide. To list the available model versions, after CESM registration (on Tetralith):

svn list https://svn-ccsm-models.cgd.ucar.edu/cesm1/release_tags

It is recommended that the source files be downloaded into the /home file system. This ensures that your code, and configuration files are backed up.

Copying the Tetralith configuration files

As an example, download the CESM distribution to some base folder <cesm-base>:

1. Change to the appropriate folder

cd <cesm-base>

2. Checkout version 1.2.2.1

svn co https://svn-ccsm-models.cgd.ucar.edu/cesm1/release_tags/cesm1_2_2_1 cesm1.2.2.1

3. Files containing Tetralith specific machine environment variables have been created and are stored in the folder /proj/cesm_input-data/tetralith_config_cesm1.2.2.1. These files should be copied to the appropriate location in your local distribution. Based on the steps above:

cd <cesm-base>/cesm1.2.2.1/scripts/

cp /proj/cesm_input-data/tetralith_config_cesm1.2.2.1/create_newcase .

cd <cesm-base>/cesm1.2.2.1/scripts/ccsm_utils/Case.template

cp /proj/cesm_input-data/tetralith_config_cesm1.2.2.1/config_definition.xml .

cp /proj/cesm_input-data/tetralith_config_cesm1.2.2.1/ConfigCase.pm .

cd <cesm-base>/cesm1.2.2.1/scripts/ccsm_utils/Machines

cp /proj/cesm_input-data/tetralith_config_cesm1.2.2.1/config_compilers.xml .

cp /proj/cesm_input-data/tetralith_config_cesm1.2.2.1/config_machines.xml .

cp /proj/cesm_input-data/tetralith_config_cesm1.2.2.1/env_mach_specific.tetralith .

cp /proj/cesm_input-data/tetralith_config_cesm1.2.2.1/mkbatch.tetralith .

Creating a new case and building the model

1. Load modules:

module load buildenv-intel/2018.u1-bare netCDF-HDF5/4.6.2-nsc1-parallel-private_hdf5-intel20018a-eb

2. Create a new case, for example, to create a new 'f19_g16' resolution case using the 'B' compset:

cd <cesm-base>/cesm1.2.2.1/scripts/

./create_newcase -case ../cases/lbtest-f19_g16-B -res f19_g16 -compset B -mach tetralith -ccsm_out /proj/bolinc/users/${USER}/cesm1.2.2.1

3. Configure and build your model:

cd <cesm-base>/cesm1.2.2.1/cases/lbtest-f19_g16-B

./cesm_setup

./lbtest-f19_g16-B.build

4. Edit the env_run.xml and the $CASE.run files for the particular simulation you wish to make, then submit your job:

sbatch ./lbtest-f19_g16-B.run