Building NorESM (CMIP5) on the NSC triolith system

This page describes the building of the CMIP5 configuration of the NorESM (NorESM) model on NSC's triolith system (triolith). Triolith consists of 1200 HP SL230s compute nodes is equipped with two Intel E5-2660 (2.2 GHz Sandybridge) processors with 8 cores each, i.e 16 cores per node. Triolith is equipped with a fast interconnect (Infiniband from Mellanox (FDR IB, 56 Gb/s) in a 2:1 blocking configuration) for high performance for parallel applications.

Triolith is a SNIC-funded system, and computing time on Triolith is exclusively allocated to SNIC projects (Large, Medium and Small). See [1] and [2] for information on how to apply for a SNIC project. Contact: support@nsc.liu.se.

CESM1 unix group on triolith

If you are going to run the NorESM model on triolith, you need to first the 'cesm_input-data' storage project on Triolith (snic2014-5-61). This allows access to the /proj/cesm_input-data project space where the source code and pre-downloaded initial data and ancillary files are stored. To join the 'cesm_input-data' project, log into SUPR (https://supr.snic.se/) and click the 'View and Manage Projects' button, the request membership to the snic2014-5-61 project.

NorESM - Getting the source code

NorESM is "free" software however contact with the Norwegian EarthClim consortium is required to use the model. Once an agreement is in place, the source code can be obtained from the shared directory on Triolith: /proj/cesm_input-data/NorESM.

1. Use Git ([3]) to clone a copy of the NorESM source to your own work space, e.g. create a source tree /home/${USER}/NorESM-CMIP5:

cd /home/${USER}

git clone /proj/cesm_input-data/NorESM/source/noresm-ver1-cmip5 NorESM-CMIP5

2. Create a case in the same way cases are created for the CCSM/CESM model. Note: when using the 'create_newcase' script, you have to add an additional variable 'ccsm_out'. This defines the folder where the model is compiled and where output is written, and must correspond to an appropriate Triolith project storage area (e.g. /proj/bolinc/users/x_abcde/NorESM)

cd /home/${USER}/noresm-ver1-cmip5/scripts

./create_newcase -case ../cases/triolithTest -mach triolith -res f19_g16 -compset N2000AERL -pecount S -ccsm_out /proj/bolinc/users/${USER}/NorESM

Note: Two pe patterns have been tested, one using 2 nodes (32 pes) and the other using 5 nodes (80 pes): see EC-Earth and CESM1 model timings for timings. the -pecount S keyword will set-up the model using the 2 node/32 pe configuration. For the 5 node/80 pe configuration use -pecount M.

3. Configure and build your model:

cd /home/$USER/noresm-ver1-cmip5/triolithTest

./configure -case

./triolithTest.build

4. Edit the env_run.xml file for the particular simulation you wish to make, then submit your job:

sbatch ./triolith.run

Notes

Initial data files can be found in the /proj/cesm_input-data/NorESM folder (rest*).

The Oslo aerosol source code can be found in the /home/${USER}/noresm-ver1-cmip5/models/atm/cam/src/physics/cam_oslo folder.