Skip to content

Commit 3e193bd

Browse files
manishvenumanishvenu
authored
Implement GitHub Actions CI testing (Fixes #138) (#189)
This PR fixes Issue #138, which was about implementing CI in MOM_Interface with four initial tasks. Please reference back to the issue for those requirements. Changes Introduced by this PR: A GitHub Action Workflow (please see below for details) under .github/workflows/general-ci-tests.yml Incidentally, Black formatting on two scripts in cime_config CI/Github Action Workflow Details: Workflow is structured into three independent jobs that are triggered on push and pull-request events onto the main branch: checking the black format of scripts in cime_config running the check_default_params test under the tests folder building the standalone MOM6 and running two lightweight MOM6 examples (double_gyre, single_column/KPP) Testing: Test Failures: The bottom two jobs (2 & 3) were tested by creating an error (See Commits starting with Test CI), and the first job with Black formatting was already failing (commit) and was corrected early on in the process. Test Events/Triggers: Every job in the commit history was triggered on a push. Pull-Requests were tested in Test CI: Pull Requests #188. Notes: The commits should (need?) be squashed. (My fault, hah) Checking out the correct MOM branch for the standalone build is a little trickier because we are starting with the CESM repo. To be safe, the triggering event is manually checked out instead of using the common Github action actions/checkout@v4 (See the steps Checkout initial event (Pull Request) and Checkout initial event (Push)). I think the workflow is pretty easy to read through, but would welcome any advice/help on the clarity.
1 parent 26b97e0 commit 3e193bd

10 files changed

Lines changed: 383 additions & 99 deletions

File tree

‎.github/workflows/general-ci-tests.yml‎

Lines changed: 110 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,110 @@
1+
name: General MOM_interface CI
2+
3+
# This CI workflow tests against the following questions:
4+
# 1. Does standalone mom build and run?
5+
# 2. Does it pass the tests/check_default_params.py test?
6+
# 3. Do the scripts in cime_config pass the black formatter?
7+
# Please see Issue #138 for more information
8+
9+
# Controls when the workflow will run
10+
on:
11+
# Triggers the workflow on push or pull request events but only for the listed branches
12+
push:
13+
branches: ["main" ]
14+
pull_request:
15+
branches: [ "main" ]
16+
17+
jobs:
18+
19+
check_standalone_mom_build_and_run_lightweight_examples:
20+
# The type of runner that the job will run on
21+
runs-on: ubuntu-latest
22+
23+
steps:
24+
25+
# Copied from NCAR/MOM6 - Install Basic Build Packages for MOM6
26+
- name: Install Ubuntu Linux packages
27+
shell: bash
28+
run: |
29+
echo "::group::Install linux packages"
30+
sudo apt-get update
31+
sudo apt-get install netcdf-bin
32+
sudo apt-get install libnetcdf-dev
33+
sudo apt-get install libnetcdff-dev
34+
sudo apt-get install openmpi-bin
35+
sudo apt-get install libopenmpi-dev
36+
sudo apt-get install linux-tools-common
37+
sudo apt-get install -y csh
38+
echo "::endgroup::"
39+
40+
# Checkout CESM (default branch) and externals
41+
- name: Checkout CESM and Externals
42+
run: |
43+
git clone https://github.com/ESCOMP/CESM.git
44+
cd CESM
45+
./bin/git-fleximod update
46+
47+
# Checkout the correct MOM Branch
48+
- name: Checkout initial event (Pull Request)
49+
if: ${{ github.event_name == 'pull_request' }}
50+
run: |
51+
echo "Handling pull request"
52+
cd $GITHUB_WORKSPACE/CESM/components/mom/
53+
git fetch origin pull/${{ github.event.pull_request.number }}/head:pr-${{ github.event.pull_request.number }}
54+
git checkout pr-${{ github.event.pull_request.number }}
55+
56+
- name: Checkout initial event (Push)
57+
if: ${{ github.event_name == 'push' }}
58+
run: |
59+
echo "Handling push"
60+
cd $GITHUB_WORKSPACE/CESM/components/mom/
61+
git checkout ${{ github.sha }}
62+
63+
# Build the standalone mom using the macos script. build_examples-ncar doesn't work.
64+
- name: Build Standalone MOM
65+
run: |
66+
cd $GITHUB_WORKSPACE/CESM/components/mom/standalone/build
67+
./build_examples.sh --compiler gnu --machine ubuntu
68+
69+
# CD into a couple MOM examples and run MOM in them. These are very light weight and quick.
70+
- name: Run Double Gyre Test
71+
run: |
72+
cd $GITHUB_WORKSPACE/CESM/components/mom/standalone/examples/double_gyre
73+
$GITHUB_WORKSPACE/CESM/components/mom/standalone/build/gnu/MOM6/MOM6
74+
- name: Run Single Column KPP Test
75+
run: |
76+
cd $GITHUB_WORKSPACE/CESM/components/mom/standalone/examples/single_column/KPP
77+
$GITHUB_WORKSPACE/CESM/components/mom/standalone/build/gnu/MOM6/MOM6
78+
79+
80+
# Job to run the check_default_params script, which is a test
81+
check_default_params:
82+
83+
runs-on: ubuntu-latest
84+
85+
steps:
86+
# Checkout the repo
87+
- uses: actions/checkout@v4
88+
89+
# Run the test
90+
- name: Run the check_default_params script
91+
run: python tests/check_default_params.py
92+
93+
# Job to run the black formatter for cime_config, see black documentation for more info
94+
check_black_format_for_cime_config:
95+
96+
runs-on: ubuntu-latest
97+
98+
steps:
99+
# Checkout the repo
100+
- uses: actions/checkout@v4
101+
102+
# Run black check
103+
- uses: psf/black@stable
104+
with:
105+
options: "--check --verbose"
106+
src: "./cime_config"
107+
108+
109+
110+

‎.travis.yml‎

Lines changed: 0 additions & 18 deletions
This file was deleted.

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
[![Build Status](https://travis-ci.org/ESCOMP/MOM_interface.svg?branch=master)](https://travis-ci.org/ESCOMP/MOM_interface)
1+
[![MOM_interface CI](https://github.com/ESCOMP/MOM_interface/actions/workflows/general-ci-tests.yml/badge.svg)](https://github.com/ESCOMP/MOM_interface/actions/workflows/general-ci-tests.yml)
22

33
# MOM_interface
44

‎cime_config/SystemTests/dimcs.py‎

Lines changed: 24 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -32,8 +32,8 @@
3232
"H_RESCALE_POWER = -11",
3333
"Z_RESCALE_POWER = -11",
3434
"R_RESCALE_POWER = -11",
35-
"Q_RESCALE_POWER = -11"
36-
]
35+
"Q_RESCALE_POWER = -11",
36+
]
3737

3838
run_suffixes = [
3939
"base",
@@ -67,27 +67,36 @@
6767
"scale the dimension Q by 2**-11",
6868
]
6969

70+
7071
class DIMCS(SystemTestsCompareN):
7172

7273
def __init__(self, case):
7374
self.comp = case.get_value("COMP_OCN")
74-
SystemTestsCompareN.__init__(self, case, N=len(nl_contents),
75-
separate_builds = False,
76-
run_suffixes = run_suffixes,
77-
run_descriptions = run_descriptions,
78-
ignore_fieldlist_diffs = True)
75+
SystemTestsCompareN.__init__(
76+
self,
77+
case,
78+
N=len(nl_contents),
79+
separate_builds=False,
80+
run_suffixes=run_suffixes,
81+
run_descriptions=run_descriptions,
82+
ignore_fieldlist_diffs=True,
83+
)
7984

8085
def _common_setup(self):
81-
nl_contents_common = '''
86+
nl_contents_common = """
8287
! DIMCS test changes
83-
'''
84-
append_to_user_nl_files(caseroot = self._case.get_value("CASEROOT"),
85-
component = self.comp,
86-
contents = nl_contents_common)
88+
"""
89+
append_to_user_nl_files(
90+
caseroot=self._case.get_value("CASEROOT"),
91+
component=self.comp,
92+
contents=nl_contents_common,
93+
)
8794

8895
def _case_setup(self, i):
8996

9097
# Second append user_nl change sepecific to case-i
91-
append_to_user_nl_files(caseroot = self._case.get_value("CASEROOT"),
92-
component = self.comp,
93-
contents = nl_contents[i])
98+
append_to_user_nl_files(
99+
caseroot=self._case.get_value("CASEROOT"),
100+
component=self.comp,
101+
contents=nl_contents[i],
102+
)

‎cime_config/SystemTests/dimcsl.py‎

Lines changed: 23 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@
2727
"Z_RESCALE_POWER = 11",
2828
"R_RESCALE_POWER = 11",
2929
"Q_RESCALE_POWER = 11",
30-
]
30+
]
3131

3232
run_suffixes = [
3333
"base",
@@ -49,27 +49,36 @@
4949
"scale the dimension Q by 2**11",
5050
]
5151

52+
5253
class DIMCSL(SystemTestsCompareN):
5354

5455
def __init__(self, case):
5556
self.comp = case.get_value("COMP_OCN")
56-
SystemTestsCompareN.__init__(self, case, N=len(nl_contents),
57-
separate_builds = False,
58-
run_suffixes = run_suffixes,
59-
run_descriptions = run_descriptions,
60-
ignore_fieldlist_diffs = True)
57+
SystemTestsCompareN.__init__(
58+
self,
59+
case,
60+
N=len(nl_contents),
61+
separate_builds=False,
62+
run_suffixes=run_suffixes,
63+
run_descriptions=run_descriptions,
64+
ignore_fieldlist_diffs=True,
65+
)
6166

6267
def _common_setup(self):
63-
nl_contents_common = '''
68+
nl_contents_common = """
6469
! DIMCSL test changes
65-
'''
66-
append_to_user_nl_files(caseroot = self._case.get_value("CASEROOT"),
67-
component = self.comp,
68-
contents = nl_contents_common)
70+
"""
71+
append_to_user_nl_files(
72+
caseroot=self._case.get_value("CASEROOT"),
73+
component=self.comp,
74+
contents=nl_contents_common,
75+
)
6976

7077
def _case_setup(self, i):
7178

7279
# Second append user_nl change sepecific to case-i
73-
append_to_user_nl_files(caseroot = self._case.get_value("CASEROOT"),
74-
component = self.comp,
75-
contents = nl_contents[i])
80+
append_to_user_nl_files(
81+
caseroot=self._case.get_value("CASEROOT"),
82+
component=self.comp,
83+
contents=nl_contents[i],
84+
)

‎standalone/build/build_examples-darwin.sh‎

Lines changed: 0 additions & 42 deletions
This file was deleted.

standalone/build/build_examples-ncar.sh renamed to standalone/build/build_examples.sh

Lines changed: 52 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -15,8 +15,56 @@ CESM_ROOT=`pwd -P`
1515
SHR_ROOT=${CESM_ROOT}/share
1616
FMS_ROOT=${CESM_ROOT}/libraries/FMS
1717

18-
COMPILER=intel
19-
TEMPLATE=${TEMPLATE_DIR}/ncar-${COMPILER}.mk
18+
# Default compiler
19+
COMPILER="intel"
20+
MACHINE="ncar"
21+
22+
# Parse command line arguments
23+
while [[ "$#" -gt 0 ]]; do
24+
case $1 in
25+
--compiler)
26+
COMPILER="$2"
27+
shift ;;
28+
--machine)
29+
MACHINE="$2"
30+
shift ;;
31+
*)
32+
echo "Unknown parameter passed: $1"
33+
echo "Usage: $0 [--compiler <compiler>] [--machine <machine>]"
34+
exit 1 ;;
35+
esac
36+
shift
37+
done
38+
echo "Using compiler: $COMPILER"
39+
echo "Using machine: $MACHINE"
40+
41+
TEMPLATE=${TEMPLATE_DIR}/${MACHINE}-${COMPILER}.mk
42+
43+
# Throw error if template does not exist:
44+
if [ ! -f $TEMPLATE ]; then
45+
echo "ERROR: Template file $TEMPLATE does not exist."
46+
echo "Templates are based on the machine and compiler arguments: machine-compiler.mk. Available templates are:"
47+
ls ${TEMPLATE_DIR}/*.mk
48+
echo "Exiting."
49+
exit 1
50+
fi
51+
52+
# Set -j option based on the MACHINE argument
53+
case $MACHINE in
54+
"homebrew" )
55+
JOBS=2
56+
;;
57+
"ubuntu" )
58+
JOBS=4
59+
;;
60+
"ncar")
61+
JOBS=36
62+
;;
63+
*)
64+
echo "Invalid machine type for make -j option: $MACHINE"
65+
exit 1
66+
;;
67+
esac
2068

2169
if [ -e $1 ]; then
2270
BLD_ROOT=${COMPILER}
@@ -25,8 +73,6 @@ else
2573
fi
2674

2775
# 1) Build FMS
28-
29-
# (b) build FMS
3076
cd ${INTERFACE_ROOT}/standalone/build
3177
mkdir -p ${BLD_ROOT}/FMS
3278
cd ${BLD_ROOT}/FMS
@@ -36,14 +82,14 @@ ${MKMF_ROOT}/list_paths ${FMS_ROOT}/src
3682
echo "${SHR_ROOT}/src/shr_kind_mod.F90" >> path_names
3783
echo "${SHR_ROOT}/src/shr_const_mod.F90" >> path_names
3884
${MKMF_ROOT}/mkmf -t ${TEMPLATE} -p libfms.a -c "-Duse_libMPI -Duse_netCDF -DSPMD" path_names
39-
make -j36 NETCDF=3 REPRO=1 libfms.a
85+
make -j${JOBS} NETCDF=3 REPRO=1 libfms.a
4086

4187
# 2) Build MOM6
4288
cd ${INTERFACE_ROOT}/standalone/build
4389
mkdir -p ${BLD_ROOT}/MOM6
4490
cd ${BLD_ROOT}/MOM6
4591
${MKMF_ROOT}/list_paths -l ${MOM_ROOT}/{config_src/infra/FMS2,config_src/memory/dynamic_symmetric,config_src/drivers/solo_driver,../externals/MARBL/src,config_src/external,src/{*,*/*}}/
4692
${MKMF_ROOT}/mkmf -t ${TEMPLATE} -o '-I../FMS' -p MOM6 -l '-L../FMS -lfms' -c '-Duse_libMPI -Duse_netCDF -DSPMD' path_names
47-
make -j36 NETCDF=3 REPRO=1 MOM6
93+
make -j${JOBS} NETCDF=3 REPRO=1 MOM6
4894

4995
echo "Finished build at `date`"

‎standalone/build/submit_casper_build‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,4 +19,4 @@
1919
#PBS -m abe
2020

2121
### Run the executable
22-
./build_examples-ncar.sh intel-casper
22+
./build_examples.sh --compiler intel --machine ncar

0 commit comments

Comments
 (0)