Add a framework for modular model extensions

.pre-commit-config.yaml

@@ -1,7 +1,7 @@
 repos:
   - repo: https://github.com/astral-sh/uv-pre-commit
     # uv version.
-    rev: 0.11.19
+    rev: 0.11.23
     hooks:
       # Dependency management
       - id: uv-lock
@@ -47,7 +47,7 @@ repos:
     # Python Linting & Formatting with Ruff
   - repo: https://github.com/astral-sh/ruff-pre-commit
     # Ruff version.
-    rev: "v0.15.16"
+    rev: "v0.15.18"
     hooks:
       - id: ruff
         name: ruff (linter)

docs/source/computational_implementation.rst

@@ -320,6 +320,11 @@ dimensionality of 3, and (following the :ref:`naming scheme
 :code:`region`, the second an element of :code:`time_optimize`,
 and third a demand commodity.
 
+.. seealso::
+   The same component pattern (index set, parameter, and constraint declared in a
+   ``model.py`` and implemented in component modules) can be packaged as an
+   optional, self-contained add-on. See :ref:`extensions` for the extension
+   framework and a copy-from template.
 
 
 

docs/source/extensions.rst

@@ -0,0 +1,246 @@
+.. _extensions:
+
+Extension Framework
+===================
+
+Temoa includes a lightweight framework for adding **optional model components**
+without modifying the core model. An extension can contribute its own database
+tables, Pyomo components (sets, parameters, and constraints), and data-loading
+rules. Extensions are declared once and then enabled per run through
+configuration.
+
+This page describes how the framework works and how to author a new extension.
+A ready-to-copy scaffold lives at ``temoa/extensions/template``.
+
+Overview
+--------
+
+Use an extension when you want to add modeling capability that is:
+
+* **Optional** -- only active when explicitly enabled.
+* **Self-contained** -- owns its own tables and model components.
+* **Non-invasive** -- adds to the core model rather than editing it.
+
+If a feature is fundamental to every Temoa run, it belongs in
+``temoa/components`` (a core component) instead of an extension.
+
+Each extension is described by a single :class:`ExtensionSpec` (declarative
+metadata plus hook functions). The spec is registered with the framework, after
+which users can enable the extension by id.
+
+Lifecycle
+---------
+
+When a run is configured with ``extensions = ["..."]``, the framework threads the
+enabled specs through configuration, model construction, and data loading:
+
+.. code-block:: text
+
+   config: extensions = ["my_ext"]
+        |
+        v
+   resolve_extension_specs()          validate ids -> ExtensionSpec list
+        |
+        v
+   TemoaModel(extensions=[...])
+        |
+        +--> apply_model_extension_hooks()
+        |        calls spec.register_model_components(model)
+        |        -> attaches Params / Sets / Constraints to the model
+        |
+        v
+   HybridLoader
+        +--> ensure_enabled_extension_tables_exist()   (offer to append schema)
+        +--> assert_disabled_extension_tables_are_empty()
+        +--> merge_regional_group_tables()
+        +--> build_manifest() -> appends spec.build_manifest_items(model)
+                 -> loads each owned table into its component
+
+The relevant code lives in :mod:`temoa.extensions.framework`,
+:mod:`temoa.core.model`, and :mod:`temoa.data_io.hybrid_loader`.
+
+``ExtensionSpec`` reference
+---------------------------
+
+.. list-table::
+   :header-rows: 1
+   :widths: 30 70
+
+   * - Field
+     - Purpose
+   * - ``extension_id``
+     - Unique, lowercase id. This is what users put in ``extensions = [...]``.
+   * - ``owned_tables``
+     - Tuple of database tables owned exclusively by this extension. Used by the
+       disabled/enabled table guards.
+   * - ``regional_group_tables``
+     - Map of ``table -> column`` for tables whose region column may hold a
+       regional *group* name. Merged into the loader's regional-group handling.
+   * - ``register_model_components``
+     - Hook ``Callable[[TemoaModel], None]`` that attaches model components.
+   * - ``build_manifest_items``
+     - Hook ``Callable[[TemoaModel], list[LoadItem]]`` describing how to load the
+       extension's data.
+   * - ``schema_sql_path``
+     - Path to a ``.sql`` file applied (with consent) when the extension is
+       enabled but its tables are missing.
+   * - ``fail_if_tables_populated_when_disabled``
+     - When ``True``, loading fails if the extension is disabled but its owned
+       tables contain data, preventing silently-ignored inputs.
+
+Recommended package layout
+--------------------------
+
+Mirror the structure of the core model (``temoa/core`` + ``temoa/components``) so
+extension code is organized the same way as the rest of the codebase:
+
+.. code-block:: text
+
+   temoa/extensions/<your_extension>/
+       __init__.py             # re-export the ExtensionSpec
+       extension.py            # the ExtensionSpec definition
+       data_manifest.py        # build_manifest_items()
+       tables.sql              # CREATE TABLE IF NOT EXISTS for owned tables
+       core/
+           __init__.py
+           model.py            # typing subtype + register_model_components()
+       components/
+           __init__.py
+           <family>.py         # one module per constraint family
+
+Centralize the component *declarations* in ``core/model.py`` (just as
+``temoa/core/model.py`` does) and keep the index-set and constraint-rule logic in
+``components/`` modules (just as ``temoa/components`` does).
+
+.. _extensions-typing:
+
+The typing pattern
+------------------
+
+Core model components are declared as attribute assignments inside
+``TemoaModel.__init__`` (for example ``self.time_optimize = Set(...)``), which is
+why the type checker knows about them. An extension instead adds attributes from
+*outside* the class, so without help the type checker reports
+``"TemoaModel" has no attribute ...`` and provides no autocomplete.
+
+Three rules make typing carry over cleanly:
+
+1. **Declare a ``TYPE_CHECKING``-only subtype.** In ``core/model.py``, define a
+   subclass of ``TemoaModel`` that annotates every component the extension adds.
+   It inherits all core attributes, so component code sees both core and
+   extension members.
+
+   .. code-block:: python
+
+      if TYPE_CHECKING:
+          from temoa.core.model import TemoaModel
+
+          class ExampleModel(TemoaModel):
+              example_new_capacity_limit: Param
+              example_new_capacity_limit_constraint_rpt: Set
+              example_new_capacity_limit_constraint: Constraint
+
+2. **Annotate component functions with the subtype.** Index-set and
+   constraint-rule functions take ``model: ExampleModel``. This restores both
+   mypy coverage and editor autocomplete.
+
+3. **Keep spec hooks on the base type and ``cast`` internally.** Functions stored
+   on the ``ExtensionSpec`` (``register_model_components`` and
+   ``build_manifest_items``) must keep ``model: TemoaModel`` to match the hook
+   callable types. Narrowing the parameter is a contravariance error. ``cast``
+   once at the top:
+
+   .. code-block:: python
+
+      def register_model_components(model: TemoaModel) -> None:
+          m = cast('ExampleModel', model)
+          m.example_new_capacity_limit = Param(...)
+
+.. note::
+   Extension attribute names share the single ``TemoaModel`` namespace at
+   runtime. Keep them unique across extensions to avoid collisions.
+
+Adding a new extension
+----------------------
+
+#. **Copy the template.** Duplicate ``temoa/extensions/template`` to
+   ``temoa/extensions/<your_extension>/``.
+#. **Rename ids and tables.** Update ``extension_id``, ``owned_tables``,
+   ``regional_group_tables``, params, sets, and constraints to your domain.
+#. **Declare components.** Add annotations to the typing subtype in
+   ``core/model.py`` and create the components in ``register_model_components``.
+#. **Write the constraint logic.** Add index-set and rule functions under
+   ``components/`` and annotate them with your subtype.
+#. **Describe data loading.** Add one ``LoadItem`` per owned table in
+   ``data_manifest.py``.
+#. **Define the schema.** Add a ``CREATE TABLE IF NOT EXISTS`` per owned table to
+   ``tables.sql``.
+#. **Register the spec.** Import your spec in
+   :func:`temoa.extensions.framework.get_known_extension_specs` and add it to the
+   ``specs`` list. (Until you do this, the extension is inert -- enabling it
+   raises an "Unknown extension id" error.)
+#. **Enable it.** Add the id to your configuration TOML:
+
+   .. code-block:: toml
+
+      extensions = ["<your_extension>"]
+
+Data loading and ``LoadItem``
+-----------------------------
+
+``build_manifest_items`` returns one :class:`temoa.data_io.loader_manifest.LoadItem`
+per database table the extension reads. Common fields:
+
+.. list-table::
+   :header-rows: 1
+   :widths: 30 70
+
+   * - Field
+     - Purpose
+   * - ``component``
+     - The Pyomo ``Set`` or ``Param`` to populate.
+   * - ``table``
+     - Source table name in the database.
+   * - ``columns``
+     - Columns to select; for a ``Param`` the final column is the value.
+   * - ``index_length``
+     - Number of leading columns that form the index.
+   * - ``validator_name`` / ``validation_map``
+     - Source-trace validation: a viable-set name on the loader and which index
+       columns it applies to.
+   * - ``is_table_required``
+     - Set ``False`` for optional extension inputs so a missing table is not an
+       error.
+
+Verification
+------------
+
+After authoring an extension, confirm:
+
+#. **Types** -- ``mypy temoa/extensions/<your_extension>`` reports no issues.
+#. **Imports** -- ``python -c "import temoa.extensions.<your_extension>.extension"``.
+#. **Wiring** -- a model built with the extension enabled attaches the expected
+   components, and the test suite passes.
+
+The template extension
+-----------------------
+
+``temoa/extensions/template`` is a complete, type-checked, but deliberately
+**unregistered** scaffold. Because it is not listed in
+``get_known_extension_specs``, it cannot be enabled until you register it, so it
+never affects normal runs. Copy the folder as the starting point for a new
+extension; every file carries ``# TEMPLATE:`` comments explaining what to change.
+
+.. _extension-catalog:
+
+Available extensions
+--------------------
+
+The extensions that ship with Temoa are documented on their own pages below.
+Each page describes the extension's parameters and constraints. This list grows
+as new extensions are added.
+
+.. toctree::
+   :maxdepth: 1
+
+   extensions/growth_rates

docs/source/extensions/growth_rates.rst

@@ -0,0 +1,99 @@
+.. _extension-growth-rates:
+
+Growth Rate Limits
+==================================
+
+The **growth_rates** extension adds optional constraints that bound how quickly
+the capacity (or new capacity) of a technology or technology group may change
+between consecutive model periods. It is disabled by default and enabled per
+run through configuration:
+
+.. code-block:: toml
+
+   extensions = ["growth_rates"]
+
+Its parameters live in the extension-owned tables of the same name and are loaded
+only when the extension is enabled. See :ref:`extensions` for how the extension
+framework wires these components into the model.
+
+Parameters
+----------
+
+.. csv-table::
+   :header: "Parameter", "Database Table", "Model Element", "Notes"
+   :widths: 15, 20, 25, 40
+
+   ":math:`\text{LGC}_{r,t}`", ":code:`limit_growth_capacity`", ":code:`limit_growth_capacity`", "capacity growth rate limits; :code:`tech_or_group` column accepts a technology name or group name"
+   ":math:`\text{LDGC}_{r,t}`", ":code:`limit_degrowth_capacity`", ":code:`limit_degrowth_capacity`", "capacity degrowth rate limits; :code:`tech_or_group` column accepts a technology name or group name"
+   ":math:`\text{LGNC}_{r,t}`", ":code:`limit_growth_new_capacity`", ":code:`limit_growth_new_capacity`", "new capacity growth rate limits; :code:`tech_or_group` column accepts a technology name or group name"
+   ":math:`\text{LDGNC}_{r,t}`", ":code:`limit_degrowth_new_capacity`", ":code:`limit_degrowth_new_capacity`", "new capacity degrowth rate limits; :code:`tech_or_group` column accepts a technology name or group name"
+   ":math:`\mathrm{LGNC}_{\Delta,r,t}`", ":code:`limit_growth_new_capacity_delta`", ":code:`limit_growth_new_capacity_delta`", "new capacity growth acceleration limits; :code:`tech_or_group` column accepts a technology name or group name"
+   ":math:`\mathrm{LDGNC}_{\Delta,r,t}`", ":code:`limit_degrowth_new_capacity_delta`", ":code:`limit_degrowth_new_capacity_delta`", "new capacity degrowth deceleration limits; :code:`tech_or_group` column accepts a technology name or group name"
+
+
+limit_growth_capacity
+~~~~~~~~~~~~~~~~~~~~~~~
+
+:math:`{LGC}_{r \in R, t \in T}`
+
+The :code:`limit_growth_capacity` parameter defines the maximum annual rate at
+which the total capacity of a technology (or group) can grow between periods.
+The :code:`tech_or_group` column accepts a technology name or group name.
+
+
+limit_degrowth_capacity
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+:math:`{LDGC}_{r \in R, t \in T}`
+
+The :code:`limit_degrowth_capacity` parameter defines the maximum annual rate
+at which the total capacity of a technology (or group) can shrink between
+periods.  The :code:`tech_or_group` column accepts a technology name or group name.
+
+
+limit_growth_new_capacity
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+:math:`{LGNC}_{r \in R, t \in T}`
+
+The :code:`limit_growth_new_capacity` parameter constrains the rate of increase
+in new capacity deployment between consecutive periods.
+
+
+limit_degrowth_new_capacity
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+:math:`{LDGNC}_{r \in R, t \in T}`
+
+The :code:`limit_degrowth_new_capacity` parameter constrains the rate of decrease
+in new capacity deployment between consecutive periods.
+
+
+limit_growth_new_capacity_delta
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+:math:`\mathrm{LGNC}_{\Delta,r \in R, t \in T}`
+
+The :code:`limit_growth_new_capacity_delta` parameter constrains the acceleration
+of new capacity growth between periods. This essentially adds "inertia" to the
+growth of new capacity deployment.
+
+
+limit_degrowth_new_capacity_delta
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+:math:`\mathrm{LDGNC}_{\Delta,r \in R, t \in T}`
+
+The :code:`limit_degrowth_new_capacity_delta` parameter constrains the
+deceleration of new capacity degrowth between periods, essentially adding
+"intertia" to the degrowth of new capacity deployment.
+
+
+Constraints
+-----------
+
+.. autofunction:: temoa.extensions.growth_rates.components.growth_capacity.limit_growth_capacity
+
+.. autofunction:: temoa.extensions.growth_rates.components.growth_new_capacity.limit_growth_new_capacity
+
+.. autofunction:: temoa.extensions.growth_rates.components.growth_new_capacity_delta.limit_growth_new_capacity_delta

docs/source/index.rst

@@ -27,6 +27,7 @@ Temoa Project Documentation
    myopic
    stochastics
    unit_checking
+   extensions
 
 .. toctree::
    :maxdepth: 1