Error when running `model_covariance01`

[jrgant]

I'm trying to get the model_covariance01 experiment running, but I keep getting the following error when I run python make02_launch_sims.py:

cannot import name 'add_ep4_from_path' from 'emodpy.emod_task' (/home/jrgant/.venv/emodpy/lib/python3.12/site-packages/emodpy/emod_task.py)
NoneType: None
Traceback (most recent call last):
  File "/home/jrgant/scratch/EMOD-Generic-Scripts/model_covariance01/experiment_covariance01/make02_launch_sims.py", line 10, in <module>
    from emod_exp import start_exp
  File "/home/jrgant/scratch/EMOD-Generic-Scripts/local_python/emod_exp.py", line 16, in <module>
    from emodpy.emod_task import EMODTask, add_ep4_from_path
ImportError: cannot import name 'add_ep4_from_path' from 'emodpy.emod_task' (/home/jrgant/.venv/emodpy/lib/python3.12/site-packages/emodpy/emod_task.py)

Here's info about my virtual environment:

Package                     Version
--------------------------- --------------
allure-pytest               2.13.5
allure-python-commons       2.13.5
arrow                       1.3.0
attrs                       25.3.0
backoff                     2.2.1
binaryornot                 0.4.4
certifi                     2025.8.3
chardet                     5.2.0
charset-normalizer          3.4.3
click                       8.1.8
click-plugins               1.1.1.2
colorama                    0.4.6
coloredlogs                 15.0.1
contourpy                   1.3.3
cookiecutter                2.6.0
coverage                    7.10.6
cycler                      0.12.1
dataclasses-json            0.6.7
diskcache                   5.4.0
docker                      7.1.0
emod-api                    2.0.26
emodpy                      2.1.13
execnet                     2.1.1
filelock                    3.19.1
fonttools                   4.59.2
future                      1.0.0
geographiclib               2.1
graphviz                    0.21
humanfriendly               10.0
idmtools                    2.2.1
idmtools_cli                2.2.1
idmtools_models             2.2.1
idmtools_platform_comps     2.2.1
idmtools_platform_container 2.2.1
idmtools_platform_general   2.2.1
idmtools_platform_slurm     2.2.1
idmtools_test               2.2.1
idna                        3.10
iniconfig                   2.1.0
Jinja2                      3.1.6
joblib                      1.5.2
junitparser                 3.1.2
kiwisolver                  1.4.9
livereload                  2.6.3
lz4                         4.4.4
markdown-it-py              4.0.0
MarkupSafe                  3.0.2
marshmallow                 3.26.1
matplotlib                  3.10.6
mdurl                       0.1.2
more-itertools              10.3.0
mypy_extensions             1.1.0
natsort                     8.4.0
numpy                       2.3.3
packaging                   24.2
pandas                      2.3.2
parse                       1.20.2
pillow                      11.3.0
pip                         24.0
pluggy                      1.6.0
prodict                     0.8.20
py                          1.11.0
pyCOMPS                     2.11.0
Pygments                    2.19.2
pyparsing                   3.2.4
pyperclip                   1.9.0
pyproj                      3.7.2
pytest                      7.4.4
pytest-cache                1.0
pytest-cov                  7.0.0
pytest-html                 3.1.1
pytest-lazy-fixture         0.6.3
pytest-metadata             3.1.1
pytest-mock                 3.15.0
pytest-timeout              2.3.1
pytest-xdist                3.8.0
python-dateutil             2.9.0.post0
python-slugify              8.0.4
pytz                        2025.2
PyYAML                      6.0.2
requests                    2.32.5
rich                        14.1.0
scikit-learn                1.7.2
scipy                       1.16.2
setuptools                  80.9.0
shapely                     2.1.1
six                         1.17.0
tabulate                    0.9.0
termcolor                   2.3.0
text-unidecode              1.3
threadpoolctl               3.6.0
tornado                     6.5.2
tqdm                        4.67.1
types-python-dateutil       2.9.0.20250822
typing_extensions           4.15.0
typing-inspect              0.9.0
tzdata                      2025.2
urllib3                     2.5.0
xdg                         6.0.0
yaspin                      3.0.2

Looking at ~/.venv/emodpy/lib/python3.12/site-packages/emodpy/emod_task.py indeed I don't see a definition for add_ep4_from_path.

Thanks for any help you can provide!

[kfrey-idm]

The function add_ep4_from_path was removed from EMODTask in the 2.1.11 release of emodpy

A replacement is add_embedded_python_scripts_from_path (a much more intuitive name than ep4, which stood for Embedded Post Processing and Pre Processing).

The main branch of this repo is pinned to emodpy==2.1.0 because of the recent changes to emodpy.

I've got a working branch for (some of) the updates needed to use 2.1.13. It's still very much in-progress, but the covariance example should be working in that branch: https://github.com/EMOD-Hub/EMOD-Generic-Scripts/tree/kfrey_update

[jrgant]

Thank you, I no longer get that error when I check out the kfrey_update branch. Is there a way to bypass COMPS for these examples?

[kfrey-idm]

With Docker, it's possible to add run_local=True as an argument to the function start_exp and run the simulations locally. There's an Ubuntu build of the executable in the directory local_python/container_assets that gets used, but you'll still need to have an emod_env:latest container locally based on this docker file: docker/emod_env/Dockerfile

Local Docker use is mostly for verification and validation, though. Individual sims run pretty quickly, but finishing hundreds of them needs some kind of cluster compute resource. There are several examples of how to build the software for different platforms (e.g., env_Ubuntu24/EMOD_EXE_Ubuntu24.def) but no good out-of-the-box solutions.

[jrgant]

Eventually we'll be using SLURM, so I'll refer to the documentation on that platform. I'm just trying to get a minimal workflow together for someone so they can build on it. Thanks for the additional info.