From 1029ee29fd74adec9f6cb1de084a87d1b54a3864 Mon Sep 17 00:00:00 2001 From: Anton Krivoborodov <63401640+antonkri@users.noreply.github.com> Date: Wed, 17 Jun 2026 13:31:03 +0000 Subject: [PATCH 1/2] WIP: Add integration process documentation Add step-by-step integration process documentation including module addition, documentation, unit tests, platforms, component tests, integration tests, reporting, and code quality steps, plus a checklist and reference. --- docs/integration_process/checklist.rst | 48 ++++ .../integration_process.rst | 65 ++++++ docs/integration_process/reference.rst | 148 ++++++++++++ .../integration_process/step_1_add_module.rst | 169 ++++++++++++++ .../step_2_documentation.rst | 132 +++++++++++ .../integration_process/step_3_unit_tests.rst | 81 +++++++ docs/integration_process/step_4_platforms.rst | 215 ++++++++++++++++++ .../step_5_component_tests.rst | 29 +++ .../step_6_integration_tests.rst | 78 +++++++ docs/integration_process/step_7_reporting.rst | 118 ++++++++++ .../step_8_code_quality.rst | 64 ++++++ 11 files changed, 1147 insertions(+) create mode 100644 docs/integration_process/checklist.rst create mode 100644 docs/integration_process/reference.rst create mode 100644 docs/integration_process/step_1_add_module.rst create mode 100644 docs/integration_process/step_2_documentation.rst create mode 100644 docs/integration_process/step_3_unit_tests.rst create mode 100644 docs/integration_process/step_4_platforms.rst create mode 100644 docs/integration_process/step_5_component_tests.rst create mode 100644 docs/integration_process/step_6_integration_tests.rst create mode 100644 docs/integration_process/step_7_reporting.rst create mode 100644 docs/integration_process/step_8_code_quality.rst diff --git a/docs/integration_process/checklist.rst b/docs/integration_process/checklist.rst new file mode 100644 index 00000000000..3c2df6c50ba --- /dev/null +++ b/docs/integration_process/checklist.rst @@ -0,0 +1,48 @@ +.. + # ******************************************************************************* + # Copyright (c) 2026 Contributors to the Eclipse Foundation + # + # See the NOTICE file(s) distributed with this work for additional + # information regarding copyright ownership. + # + # This program and the accompanying materials are made available under the + # terms of the Apache License Version 2.0 which is available at + # https://www.apache.org/licenses/LICENSE-2.0 + # + # SPDX-License-Identifier: Apache-2.0 + # ******************************************************************************* + +Checklist +========= + +Use this as a final review before opening your pull request: + +.. list-table:: + :header-rows: 1 + :widths: 40 60 + + * - Step + - Done when… + * - 1 · Add module (register, regenerate, patches) + - module entry added under ``modules.target_sw`` with ``repo`` + ``hash``; + ``update_module_from_known_good.py`` ran and generated files committed; + patches referenced if needed; ``known_good_correct`` and ``bzlmod-lock`` + checks green. + * - 2 · Documentation + - ``needs_json`` added to ``//:docs`` data and toctree entry in + ``sw_components.rst``; ``//:docs_combo`` shows the module. + * - 3 · Unit tests (default platform) + - module compiles and unit tests pass on ``--config=linux-x86_64`` via + ``quality_runners.py``; ``metadata`` tuned if needed. + * - 4 · Platforms (build & test) + - ``score_pkg_bundle`` + ``*.score.json`` added and registered in + ``showcases/BUILD`` (ships into every image); ``bazel test + @score_my_module//...`` passes. + * - 5 · Component tests + - *not available yet — to be introduced.* + * - 6 · Integration tests + - tests added under ``feature_integration_tests/itf``. + * - 7 · Reporting + - Rust coverage generated; verification/status tables updated. + * - 8 · Code quality + - formatting, Bazel Clippy (Rust) and the CodeQL/MISRA C++ scan pass. diff --git a/docs/integration_process/integration_process.rst b/docs/integration_process/integration_process.rst index 2468fba0860..fb0e92706b1 100644 --- a/docs/integration_process/integration_process.rst +++ b/docs/integration_process/integration_process.rst @@ -12,5 +12,70 @@ # SPDX-License-Identifier: Apache-2.0 # ******************************************************************************* +################### Integration Process ################### + +This document is a step-by-step *how-to* for module owners who want to integrate +their own S-CORE module into the **Reference Integration** for the first time. + +It walks you through everything from registering the module in +``known_good.json`` over wiring it into the Bazel build, the showcases, the +images and the CLI, up to adding it to the CI/CD pipelines and the consolidated +reporting (documentation, coverage and verification reports). + + +What "integration" means here +============================= + +The Reference Integration is a single `Bazel (bzlmod) `_ +build that pulls every S-CORE module together at a defined commit, builds them +against each other and runs their tests, showcases and reports. + +Integrating a module is **incremental**: each step below builds on the previous +one and *extends the reach* of your module a little further into the +integration. You climb the ladder only as far as your module needs — a pure +library may stop after it builds in-tree, while a module with a runnable example +goes all the way to running on every target platform and being exercised by +integration tests. + +The chapters that follow walk this ladder in order. Stop at whatever rung makes +sense for your module. + + +Prerequisites +============= + +Before you start, make sure that: + +* Your module lives in its own GitHub repository under the ``eclipse-score`` + organization and is a **bzlmod module**, i.e. it has a top-level + ``MODULE.bazel`` with a ``module(name = "score_")`` declaration. +* The module name follows the ``score_`` convention (e.g. + ``score_communication``). The same name is used as the Bazel + ``bazel_dep`` / repository name (``@score_//...``). +* The module builds and tests pass standalone on Linux ``x86_64``. +* You know the commit hash you want to pin (the integration always pins an exact + commit, never a floating branch). + + +The integration steps +====================== + +Work through the chapters in order — each one extends the reach of your module a +little further into the integration: + +.. toctree:: + :maxdepth: 2 + :numbered: + + step_1_add_module + step_2_documentation + step_3_unit_tests + step_4_platforms + step_5_component_tests + step_6_integration_tests + step_7_reporting + step_8_code_quality + checklist + reference diff --git a/docs/integration_process/reference.rst b/docs/integration_process/reference.rst new file mode 100644 index 00000000000..56193ea6bdf --- /dev/null +++ b/docs/integration_process/reference.rst @@ -0,0 +1,148 @@ +.. + # ******************************************************************************* + # Copyright (c) 2026 Contributors to the Eclipse Foundation + # + # See the NOTICE file(s) distributed with this work for additional + # information regarding copyright ownership. + # + # This program and the accompanying materials are made available under the + # terms of the Apache License Version 2.0 which is available at + # https://www.apache.org/licenses/LICENSE-2.0 + # + # SPDX-License-Identifier: Apache-2.0 + # ******************************************************************************* + +Reference +========= + +This section is a catalogue of *what already exists* in the integration — the +CI checks that run on every change and the consolidated reports — together with +the touch-point a new module has in each. Use it to understand where your module +shows up once it is wired in via the steps above. The supported target platforms +and their images are listed in :ref:`supported_platforms` (Step 4). + +.. _ci_checks: + +CI checks +--------- + +The pipelines live under `.github/workflows <../../.github/workflows>`_. Most +operate on ``//...`` and pick up your module without changes; the table says +what each check verifies and whether you typically need to touch anything. + +**Gating** marks the checks that run in the merge queue +(``merge_group``) and must therefore be green before a pull request can be +merged. The remaining checks are informational, run only on ``push``/schedule, +or gate a different event (release approvals). The definitive list of *required* +status checks is configured in the repository's GitHub branch-protection / +merge-queue settings; the column below reflects merge-queue participation +declared in the workflows. + +.. list-table:: + :header-rows: 1 + :widths: 30 36 12 22 + + * - Check (workflow) + - What it verifies + - Gating + - Your action + * - `build_and_test_qnx.yml <../../.github/workflows/build_and_test_qnx.yml>`_ + - Builds the QNX IFS and runs integration tests on QEMU. + - **yes** + - none (graph-wide) + * - `build_and_test_autosd.yml <../../.github/workflows/build_and_test_autosd.yml>`_ + - Builds the AutoSD OCI image with the Automotive Image Builder. + - **yes** + - none (graph-wide) + * - `build_and_test_ebclfsa.yml <../../.github/workflows/build_and_test_ebclfsa.yml>`_ + - Builds the EBcLfSA image and runs the high-integrity safety tests. + - **yes** + - none (graph-wide) + * - `known_good_correct.yml <../../.github/workflows/known_good_correct.yml>`_ + - Fails if the generated Bazel fragments drift from ``known_good.json``. + - **yes** + - commit step 1.2 output + * - `bzlmod-lock.yml <../../.github/workflows/bzlmod-lock.yml>`_ + - Verifies ``MODULE.bazel.lock`` is consistent with the module graph. + - **yes** + - update lockfile if it fails + * - `format.yml <../../.github/workflows/format.yml>`_ + - Runs the code-formatting checks. + - **yes** + - run the format targets + * - `codeql-multiple-repo-scan.yml <../../.github/workflows/codeql-multiple-repo-scan.yml>`_ + - Multi-repository CodeQL security scan across the integrated modules. + - **yes** + - none + * - `test_and_docs.yml <../../.github/workflows/test_and_docs.yml>`_ + - Code-quality checks, builds the docs and reports, publishes to Pages. + - **yes** + - wire docs in Step 2, reports in Step 7 + * - `check_release_approvals.yml <../../.github/workflows/check_release_approvals.yml>`_ + - Enforces required approvals on PRs targeting ``releases/*`` branches. + - release branches only + - none + * - `build_and_test_linux.yml <../../.github/workflows/build_and_test_linux.yml>`_ + - Builds the Linux x86_64 image and runs feature integration tests on Docker. + - no + - none (graph-wide) + * - `internal_tests.yml <../../.github/workflows/internal_tests.yml>`_ + - Runs the integration tooling tests (``//scripts/tooling:tooling_tests``). + - no + - only if you change scripts + * - `docs_cleanup.yml <../../.github/workflows/docs_cleanup.yml>`_ + - Scheduled cleanup of published documentation versions. + - no + - none + * - `test_integration.yml <../../.github/workflows/test_integration.yml>`_ / + `reusable_smoke-test.yml <../../.github/workflows/reusable_smoke-test.yml>`_ / + `reusable_integration-build.yml <../../.github/workflows/reusable_integration-build.yml>`_ + - Smoke-test / reusable build of the latest module ``main`` branches. + - no + - add runtime targets to + `ci/showcase_targets_run.txt <../../ci/showcase_targets_run.txt>`_ + +.. _reports: + +Reports +------- + +The consolidated outputs published by the integration. They are built by +``test_and_docs.yml`` and rendered into the documentation site. + +.. list-table:: + :header-rows: 1 + :widths: 30 40 30 + + * - Report + - Contents + - Source / target + * - Consolidated documentation + - All integrated module docs merged into one Sphinx site. + - `BUILD <../../BUILD>`_ ``docs(...)`` → ``bazel run //:docs_combo`` + * - Platform verification report + - Per-release requirements/architecture verification, safety analyses and + per-test-case results. + - `docs/verification_report/platform_verification_report.rst <../../docs/verification_report/platform_verification_report.rst>`_ + * - Unit test summary + - Per-module unit-test execution table (generated at build time by + ``quality_runners.py``). + - `docs/verification_report/unit_test_summary.md <../../docs/verification_report/unit_test_summary.md>`_ + * - Coverage summary + - Per-module C++ and Rust coverage table (generated at build time by + ``quality_runners.py``). + - `docs/verification_report/coverage_summary.md <../../docs/verification_report/coverage_summary.md>`_ + * - C++ coverage (per module) + - lcov/genhtml line/function/branch report for modules declaring ``cpp``. + - `scripts/quality_runners.py <../../scripts/quality_runners.py>`_ → + ``python3 scripts/quality_runners.py --modules-to-test `` + * - Rust coverage reports + - Per-module Rust line coverage (C0/C1) for modules declaring ``rust``. + - `rust_coverage/BUILD <../../rust_coverage/BUILD>`_ → + ``bazel run //rust_coverage:rust_coverage_`` + * - Overall feature & process status + - Feature/process completion dashboard derived from the pinned module repos. + - `docs/s_core_v_1/roadmap/overall_status.rst <../../docs/s_core_v_1/roadmap/overall_status.rst>`_ + * - Integration status dashboard + - Live build/health overview of the integration. + - `status_dashboard.html `_ diff --git a/docs/integration_process/step_1_add_module.rst b/docs/integration_process/step_1_add_module.rst new file mode 100644 index 00000000000..54723beacd2 --- /dev/null +++ b/docs/integration_process/step_1_add_module.rst @@ -0,0 +1,169 @@ +.. + # ******************************************************************************* + # Copyright (c) 2026 Contributors to the Eclipse Foundation + # + # See the NOTICE file(s) distributed with this work for additional + # information regarding copyright ownership. + # + # This program and the accompanying materials are made available under the + # terms of the Apache License Version 2.0 which is available at + # https://www.apache.org/licenses/LICENSE-2.0 + # + # SPDX-License-Identifier: Apache-2.0 + # ******************************************************************************* + +Step 1 — Adding module to reference integration +================================================= + + **What this unlocks:** your module becomes a *known* part of the integration, + pinned at an exact commit, enters the **Bazel module graph** and builds + cleanly against every other module. This is the foundation every later step + builds on — afterwards ``@score_my_module//...`` resolves and any other + target can depend on your module. + +1.1 Register the module in ``known_good.json`` +---------------------------------------------- + +``known_good.json`` is the single source of truth for which modules — and at +which commit — are part of the integration. Every other generated file is +derived from it, so this is always the first change. + +Open `known_good.json <../../known_good.json>`_. It has two top-level groups +under ``modules``: + +* ``target_sw`` — the S-CORE software modules that are built and shipped + (baselibs, communication, persistency, logging, …). **Your module goes here.** +* ``tooling`` — supporting tooling repos (docs-as-code, process, platform, …). + +Add an entry for your module under ``modules.target_sw``: + +.. code-block:: json + + "score_my_module": { + "repo": "https://github.com/eclipse-score/my_module.git", + "hash": "0000000000000000000000000000000000000000", + "metadata": { + "code_root_path": "//src/...", + "langs": ["rust"], + "extra_test_config": [], + "exclude_test_targets": [], + "rust_coverage_config": "ferrocene-coverage" + } + } + +Field reference +~~~~~~~~~~~~~~~ + +``repo`` (required) + HTTPS clone URL of the module repository. + +``hash`` (required, mutually exclusive with ``version``) + Exact commit to pin. Use ``version`` instead if the module is published to a + Bazel registry (this emits a ``single_version_override`` rather than a + ``git_override``). + +``bazel_patches`` (optional) + List of patch labels applied on top of the checked-out commit (see + :ref:`patches`). + +``metadata.code_root_path`` (default ``//score/...``) + Bazel target pattern that points at the module's source root. Used to scope + test discovery and Rust coverage queries (e.g. ``//src/...``). + +``metadata.langs`` (default ``["cpp", "rust"]``) + Languages used by the module. ``rust`` enables a generated Rust coverage + report (see :ref:`reporting`). + +``metadata.extra_test_config`` + Extra ``--//flag=value`` build settings injected when building/testing this + module (feature flags, config selections, …). + +``metadata.exclude_test_targets`` + Test targets that must **not** run in the integration (flaky, environment + specific, or not meaningful out-of-tree). Supports wildcards such as + ``//score/json/examples:*``. + +``metadata.rust_coverage_config`` (default ``ferrocene-coverage``) + The Bazel coverage config to use for the generated Rust coverage report. + +``pin_version`` (optional, default ``false``) + When ``true``, the automatic "update to latest HEAD" scripts will leave this + module's hash untouched. + +.. note:: + + To bump an existing entry to the latest commit on a branch you can use + ``scripts/known_good/update_module_latest.py`` instead of editing the hash by + hand. The first integration, however, is added manually. + + +1.2 Regenerate the Bazel module fragments +----------------------------------------- + + Your module now enters the **Bazel module graph**. From now on + ``@score_my_module//...`` resolves and any other target — showcases, docs, + other modules — can depend on it. (If it declares ``rust``, its Rust coverage + target is generated here too.) + +The Bazel files are **generated** from ``known_good.json`` — never edit them by +hand. After editing the JSON, regenerate them: + +.. code-block:: bash + + python3 scripts/known_good/update_module_from_known_good.py + +This (re)writes: + +* `bazel_common/score_modules_target_sw.MODULE.bazel <../../bazel_common/score_modules_target_sw.MODULE.bazel>`_ + — a ``bazel_dep`` + ``git_override`` block for your module. This file is + pulled into the build via ``include(...)`` in + `MODULE.bazel <../../MODULE.bazel>`_. +* `rust_coverage/BUILD <../../rust_coverage/BUILD>`_ — a + ``rust_coverage_report`` target for every module that declares ``rust`` in its + ``langs`` (skipped for C++-only modules). + +Commit the regenerated files together with the ``known_good.json`` change. CI +verifies that the generated files are in sync with the JSON (see the +``known_good_correct`` workflow), so a manual edit that drifts from the JSON +will fail. + +.. _patches: + +1.3 (optional) Add Bazel patches +-------------------------------- + +If the module does not build cleanly out-of-tree (e.g. label visibility issues), +add patch files under ``patches//`` and reference them from the +``bazel_patches`` list of the module entry in ``known_good.json``. See +`patches/communication <../../patches/communication>`_ for a worked example. The +generator emits ``patches = [...]`` with ``patch_strip = 1`` automatically. + +1.4 CI checks that the module was added correctly +------------------------------------------------- + +Once the changes above are pushed, two **gating** CI jobs verify that the module +was wired in consistently. They do not build your module's code — they only +check that the generated metadata is in sync with ``known_good.json``: + +.. list-table:: + :header-rows: 1 + :widths: 28 48 24 + + * - Check (workflow) + - What it verifies + - Fix if it fails + * - Known Good Matches Bazel + (`known_good_correct.yml <../../.github/workflows/known_good_correct.yml>`_) + - Re-runs ``update_module_from_known_good.py`` and fails if the generated + Bazel fragments differ from what you committed — i.e. ``known_good.json`` + and the generated ``*.MODULE.bazel`` / ``rust_coverage/BUILD`` are out of + sync. + - re-run step 1.2 and commit the regenerated files + * - Bzlmod Lockfile Check + (`bzlmod-lock.yml <../../.github/workflows/bzlmod-lock.yml>`_) + - Verifies ``MODULE.bazel.lock`` is consistent with the updated module + graph (your new ``bazel_dep`` / ``git_override``). + - refresh and commit ``MODULE.bazel.lock`` + +Both jobs run in the merge queue, so they must be green before the pull request +can be merged. The full catalogue of checks is under :ref:`ci_checks`. diff --git a/docs/integration_process/step_2_documentation.rst b/docs/integration_process/step_2_documentation.rst new file mode 100644 index 00000000000..12849471dee --- /dev/null +++ b/docs/integration_process/step_2_documentation.rst @@ -0,0 +1,132 @@ +.. + # ******************************************************************************* + # Copyright (c) 2026 Contributors to the Eclipse Foundation + # + # See the NOTICE file(s) distributed with this work for additional + # information regarding copyright ownership. + # + # This program and the accompanying materials are made available under the + # terms of the Apache License Version 2.0 which is available at + # https://www.apache.org/licenses/LICENSE-2.0 + # + # SPDX-License-Identifier: Apache-2.0 + # ******************************************************************************* + +Step 2 — Join the documentation and process-compliance checks +============================================================= + + **What this unlocks:** your module's documentation appears in the single, + combined docs site that is built and published for the whole integration — + and, just as importantly, your module's **requirements, architecture and + other process artifacts are validated against the S-CORE process** as part of + the same build. This is usually the first thing you *extend the integration + with* after the module is in the graph. + +Building the docs is not only about rendering pages. The integration uses the +docs-as-code toolchain (`score_docs_as_code +`_, pulled in via +``score_sphinx_bundle`` in `docs/conf.py <../../docs/conf.py>`_), which runs the +**S-CORE metamodel checks** on every ``needs`` object: requirements, +architecture elements, safety artifacts and their links must conform to the +process model defined in `score_process +`_. If your module's +requirements/architecture are malformed, unlinked or violate the metamodel, the +docs build (and therefore CI) fails — so wiring your module in here is what gets +its process artifacts continuously checked, not just published. + +The integration builds one Sphinx site that merges the docs of every integrated +module. To pull yours in: + +#. Add your module's ``needs_json`` to the ``docs(...)`` rule's ``data`` list in + the top-level `BUILD <../../BUILD>`_ file: + + .. code-block:: python + + docs( + data = [ + "@score_persistency//:needs_json", + # ... + "@score_my_module//:needs_json", + ], + known_good = "known_good.json", + source_dir = "docs", + ) + + This requires your module to expose a ``//:needs_json`` target (i.e. it is a + docs-as-code module). + +#. Add a toctree entry so the module shows up in the navigation. **Which page + you edit depends on what kind of module it is** — the site groups modules + into two top-level sections, each backed by its own ``.rst`` page: + + .. list-table:: + :header-rows: 1 + :widths: 22 30 48 + + * - Section + - Page to edit + - Put your module here if it is… + * - **Modules** + - `docs/sw_components.rst <../../docs/sw_components.rst>`_ + - an S-CORE **software module** that ships in the integration — i.e. it + lives under ``modules.target_sw`` in ``known_good.json`` (communication, + persistency, logging, orchestrator, baselibs, …). This is the common + case. + * - **Process, Methods & Tools** + - `docs/process_methods_tools.rst <../../docs/process_methods_tools.rst>`_ + - a **tooling / process** repo from ``modules.tooling`` (platform, process, + docs-as-code) rather than a shipped software module. + + For a normal software module, add the entry to the ``Modules`` toctree in + `docs/sw_components.rst <../../docs/sw_components.rst>`_, next to the existing + modules: + + .. code-block:: rst + + Modules + ======= + + .. toctree:: + :titlesonly: + :maxdepth: 1 + + _collections/score_persistency/docs/index + _collections/score_logging/docs/index + # ... + _collections/score_my_module/docs/index + + The ``_collections/score_my_module/docs/index`` path is **not** a file in this + repository — the docs build mounts every module's documentation under + ``_collections//`` at build time. So the path must be: + + * ``score_my_module`` — the **module/repository name** as registered in + ``known_good.json`` (the ``@score_my_module`` repo), and + * ``docs/index`` — the path of the **documentation root** *inside that + module's repository* (most modules use ``docs/index``; match whatever your + module actually exposes via its ``needs_json`` docs target). + +Build the full docs locally to verify your module shows up — or use the +live-preview server which rebuilds on every change: + +.. code-block:: bash + + # one-shot full build incl. all modules + bazel run //:docs_combo + + # live preview in the browser (auto-rebuild) + bazel run //:live_preview_combo_experimental + +The docs are built and published by the ``test_and_docs`` workflow (see +:ref:`ci_checks`). + +.. note:: + + **Planned refactoring.** Today documentation generation is entangled with + unit tests, coverage and feature integration tests in the single + ``test_and_docs`` workflow + (`test_and_docs.yml <../../.github/workflows/test_and_docs.yml>`_). This + should be refactored: the documentation build (including the metamodel / + process-compliance checks) belongs in a **dedicated, reusable workflow shared + across all S-CORE repositories**, hosted centrally in the + `cicd-workflows `_ + repository, instead of being duplicated and maintained per repo. diff --git a/docs/integration_process/step_3_unit_tests.rst b/docs/integration_process/step_3_unit_tests.rst new file mode 100644 index 00000000000..820428c0927 --- /dev/null +++ b/docs/integration_process/step_3_unit_tests.rst @@ -0,0 +1,81 @@ +.. + # ******************************************************************************* + # Copyright (c) 2026 Contributors to the Eclipse Foundation + # + # See the NOTICE file(s) distributed with this work for additional + # information regarding copyright ownership. + # + # This program and the accompanying materials are made available under the + # terms of the Apache License Version 2.0 which is available at + # https://www.apache.org/licenses/LICENSE-2.0 + # + # SPDX-License-Identifier: Apache-2.0 + # ******************************************************************************* + +Step 3 — Get compiled and unit-tested on the default platform +============================================================= + + **What this unlocks:** every module registered in ``known_good.json`` is + automatically **compiled and run through its unit tests** on the default + platform (Linux ``x86_64``, ``--config=linux-x86_64``) — no per-module wiring + is needed. As soon as your module is in the graph (Step 1), its unit tests + are part of the integration's quality gate. + +Unit tests are driven by +`scripts/quality_runners.py <../../scripts/quality_runners.py>`_, which loads +``known_good.json`` and iterates over **every** module under +``modules.target_sw``. For each module it runs ``bazel coverage`` with the +``unit-tests`` config, which builds and tests the module's source tree on the +default Linux ``x86_64`` platform: + +.. code-block:: bash + + # what the runner effectively executes per module + bazel coverage --config=unit-tests \ + --instrumentation_filter=@score_my_module \ + @score_my_module + + # reproduce locally for a single module + python3 scripts/quality_runners.py --modules-to-test score_my_module + +``--config=unit-tests`` expands to ``--config=linux-x86_64`` (plus +``--build_tests_only`` and a ``-manual`` tag filter), so this is the **default +platform** baseline that every module must pass — independent of the +target-platform images in Step 4. + +What you can tune from ``known_good.json`` +------------------------------------------ + +You do not edit ``quality_runners.py`` to control what is tested — the per-module +``metadata`` in ``known_good.json`` (Step 1) drives it: + +* ``metadata.code_root_path`` — the target pattern that is built and tested + (``@score_my_module``), e.g. ``//src/...``. +* ``metadata.extra_test_config`` — extra ``--flag=value`` build/test settings + injected for this module (feature flags, config selections). +* ``metadata.exclude_test_targets`` — test targets that must **not** run + (flaky, environment-specific or not meaningful out-of-tree); they are passed as + ``-@score_my_module`` exclusions. Supports wildcards such as + ``//score/json/examples:*``. + +So enrolling your module into unit testing requires no extra step beyond Step 1 +— you only adjust these fields when the defaults do not fit. + +The unit-test results and the code-coverage numbers produced here are exported +into the consolidated reports (markdown summaries on the docs site, HTML +coverage reports and the CI job summary). How and where exactly is described in +Step 7, see :ref:`reporting`. + +This runs in the ``test_and_docs`` workflow +(`test_and_docs.yml <../../.github/workflows/test_and_docs.yml>`_), in the same +job as the docs build (see :ref:`ci_checks`). + +.. note:: + + **Planned refactoring.** As with the documentation build, unit testing is + currently bundled into the monolithic ``test_and_docs`` workflow together with + coverage and feature integration tests. This should be split out into a + **dedicated, reusable unit-test workflow shared across all S-CORE + repositories**, hosted centrally in the + `cicd-workflows `_ + repository, instead of being duplicated per repo. diff --git a/docs/integration_process/step_4_platforms.rst b/docs/integration_process/step_4_platforms.rst new file mode 100644 index 00000000000..6054a6e35ec --- /dev/null +++ b/docs/integration_process/step_4_platforms.rst @@ -0,0 +1,215 @@ +.. + # ******************************************************************************* + # Copyright (c) 2026 Contributors to the Eclipse Foundation + # + # See the NOTICE file(s) distributed with this work for additional + # information regarding copyright ownership. + # + # This program and the accompanying materials are made available under the + # terms of the Apache License Version 2.0 which is available at + # https://www.apache.org/licenses/LICENSE-2.0 + # + # SPDX-License-Identifier: Apache-2.0 + # ******************************************************************************* + +Step 4 — Get built and tested on the target platforms +===================================================== + + **What this unlocks:** your module is actually **compiled and tested by the + pipelines** — built for and deployed to every target platform (Linux x86_64, + QNX, AutoSD, EBcLfSA aarch64), offered in the interactive CLI, and exercised + by CI. This is the rung that turns a library into something that *runs* inside + the integration images. + +The integration ships four target images, each built and tested by its own +gating CI job. The table lists the platforms, the image target, the runner used +to execute it, and the per-platform CI check. + +.. _supported_platforms: + +.. list-table:: Supported target platforms + :header-rows: 1 + :widths: 22 14 22 22 20 + + * - Platform + - Arch + - Image + - Runner + - Build & test workflow + * - Linux + - x86_64 + - `images/linux_x86_64 <../../images/linux_x86_64>`_ + - Docker (`runners/docker_x86_64 <../../runners/docker_x86_64>`_) + - `build_and_test_linux.yml <../../.github/workflows/build_and_test_linux.yml>`_ + * - QNX 8 + - x86_64 + - `images/qnx_x86_64 <../../images/qnx_x86_64>`_ + - QEMU (`runners/qemu_x86_64 <../../runners/qemu_x86_64>`_) + - `build_and_test_qnx.yml <../../.github/workflows/build_and_test_qnx.yml>`_ + * - Red Hat AutoSD + - x86_64 + - `images/autosd <../../images/autosd>`_ + - Docker (`runners/docker_x86_64 <../../runners/docker_x86_64>`_) + - `build_and_test_autosd.yml <../../.github/workflows/build_and_test_autosd.yml>`_ + * - EB corbos Linux (Safety Apps) + - aarch64 + - `images/ebclfsa_aarch64 <../../images/ebclfsa_aarch64>`_ + - QEMU (`runners/qemu_aarch64 <../../runners/qemu_aarch64>`_) + - `build_and_test_ebclfsa.yml <../../.github/workflows/build_and_test_ebclfsa.yml>`_ + +.. note:: + + The EB corbos *Safety Applications* image is special: it runs a + high-integrity application demo under additional constraints. Read + `images/ebclfsa_aarch64/README.md <../../images/ebclfsa_aarch64/README.md>`_ + before targeting it. + +Run any image locally in QEMU +----------------------------- + +Every image exposes a ``:run`` target that boots it in an emulator, so you can +try your module on the real target without hardware: + +.. code-block:: bash + + bazel run //images/linux_x86_64:run + bazel run //images/qnx_x86_64:run + bazel run //images/autosd:run + bazel run //images/ebclfsa_aarch64:run + +The QNX and aarch64 images boot under QEMU, which needs a one-time host setup +(TUN device, bridge helper, libvirt network). Follow the QEMU runner how-to in +`runners/qemu_x86_64/README.md <../../runners/qemu_x86_64/README.md>`_ before the +first run. + +Two ways to get your module onto the images +------------------------------------------- + +There are two ways to make your binary part of the images: + +* **As a showcase (recommended).** Bundle your binary once into the aggregate + ``//showcases`` package; it is then deployed into **every** image + automatically — one change covers all four platforms, and the binary is also + offered in the interactive CLI. See :ref:`add_showcase` below. +* **Directly into each image.** Wire the binary into each image's build + description by hand. This is **per-image work that you repeat four times** + (once per image) and is only needed for components that are not run as a CLI + showcase — typically a daemon. The ``datarouter`` is integrated this way; see + how the QNX image does it in + `images/qnx_x86_64/build/BUILD <../../images/qnx_x86_64/build/BUILD>`_ (the + ``datarouter`` source and the ``DATAROUTER_PATH`` mapping). See + :ref:`add_directly` below. Prefer the showcase route unless you have a reason + not to. + +.. _add_showcase: + +4.1 Add a runtime showcase +-------------------------- +A *showcase* is the vehicle that gets your module onto the platforms: you bundle +a runnable binary, and the aggregate ``//showcases`` package is deployed into +**every** image automatically. You do this once — there is no per-platform work. + +There are two patterns under `showcases/ <../../showcases>`_: + +* **Standalone** (`showcases/standalone <../../showcases/standalone>`_) — bundle + a binary that already exists in your module's repo. +* **Composed** (e.g. + `showcases/orchestration_persistency <../../showcases/orchestration_persistency>`_) + — a small binary defined *here* that combines several modules. + +**1. Define the bundle.** Add a ``score_pkg_bundle`` target. To re-use a binary +that lives in your module: + +.. code-block:: python + + load("//bazel_common:bundlers.bzl", "score_pkg_bundle") + + score_pkg_bundle( + name = "my_module", + bins = ["@score_my_module//examples:my_example"], + config_data = ["//showcases/standalone:my_module.score.json"], + package_dir = "standalone", + ) + +**2. Add a CLI descriptor.** Create a ``*.score.json`` next to the bundle. The +CLI auto-discovers every ``*.score.json`` deployed under ``/showcases`` and +offers it in the interactive menu, so **no code change to the CLI is required**: + +.. code-block:: json + + { + "name": "My Module Example", + "description": "Demonstrates my_module at runtime", + "apps": [ + { + "path": "/showcases/bin/my_example", + "args": [], + "env": {}, + "dir": "/showcases/data/my_module" + } + ] + } + +**3. Register the bundle in the top-level showcase package.** Add your bundle to +the aggregate ``score_pkg_bundle`` in +`showcases/BUILD <../../showcases/BUILD>`_ so it is packaged into the images: + +.. code-block:: python + + score_pkg_bundle( + name = "showcases", + bins = ["//showcases/cli"], + other_package_files = [ + "//showcases/standalone:comm_pkg_files", + # ... + "//showcases/standalone:my_module_pkg_files", + ], + package_dir = "showcases", + ) + +Once your bundle is part of ``//showcases``, it is reachable from +``//images/:image`` and is therefore **built by every platform CI +job** automatically. + +.. _add_directly: + +4.2 Add a binary directly to an image +------------------------------------- + +If your component is not a CLI showcase — for example a background daemon that +must always be present — you wire it straight into each image's build +description instead. Unlike the showcase route, **this is per-image work: you +repeat it for every image you target** (Linux, QNX, AutoSD, EBcLfSA). + +The reference example is the ``datarouter``. In the QNX image it is added as a +source of the image target and exposed to the image's build description via a +location mapping, then placed into the filesystem by ``system.build``: + +.. code-block:: python + + # images/qnx_x86_64/build/BUILD + qnx_ifs( + name = "init", + srcs = [ + # ... + "//showcases", + "@score_logging//score/datarouter", + "//feature_integration_tests/configs/datarouter:etc_configs", + ], + ext_repo_maping = { + "BUNDLE_PATH": "$(location //showcases:showcases)", + "DATAROUTER_PATH": "$(location @score_logging//score/datarouter:datarouter)", + }, + ) + +See `images/qnx_x86_64/build/BUILD <../../images/qnx_x86_64/build/BUILD>`_ and the +matching deployment lines in +`images/qnx_x86_64/build/system.build <../../images/qnx_x86_64/build/system.build>`_ +for the full picture, and repeat the equivalent wiring in the other images you +need (`images/linux_x86_64 <../../images/linux_x86_64>`_, +`images/autosd <../../images/autosd>`_, +`images/ebclfsa_aarch64 <../../images/ebclfsa_aarch64>`_). + +If your module simply cannot run on a given platform, exclude its targets via +``metadata.exclude_test_targets`` / ``metadata.extra_test_config`` in +``known_good.json`` (Step 1) rather than editing the workflow. diff --git a/docs/integration_process/step_5_component_tests.rst b/docs/integration_process/step_5_component_tests.rst new file mode 100644 index 00000000000..b2b0915c8a2 --- /dev/null +++ b/docs/integration_process/step_5_component_tests.rst @@ -0,0 +1,29 @@ +.. + # ******************************************************************************* + # Copyright (c) 2026 Contributors to the Eclipse Foundation + # + # See the NOTICE file(s) distributed with this work for additional + # information regarding copyright ownership. + # + # This program and the accompanying materials are made available under the + # terms of the Apache License Version 2.0 which is available at + # https://www.apache.org/licenses/LICENSE-2.0 + # + # SPDX-License-Identifier: Apache-2.0 + # ******************************************************************************* + +Step 5 — Add component tests +============================ + + **What this unlocks:** your module is verified **in isolation as a + component** — its public interfaces are tested against their specification, + independently of the other modules and of the full integration images. + +.. warning:: + + **Currently missing — needs to be introduced.** There is no component-test + stage in the Reference Integration yet. Component tests (testing a single + module's components/interfaces in isolation, between unit tests in Step 3 and + the runtime feature integration tests in Step 6) still need to be designed and + wired into the integration and its CI. This step is a placeholder so the + intended test pyramid is documented; update it once the mechanism exists. diff --git a/docs/integration_process/step_6_integration_tests.rst b/docs/integration_process/step_6_integration_tests.rst new file mode 100644 index 00000000000..fb038ce7a2b --- /dev/null +++ b/docs/integration_process/step_6_integration_tests.rst @@ -0,0 +1,78 @@ +.. + # ******************************************************************************* + # Copyright (c) 2026 Contributors to the Eclipse Foundation + # + # See the NOTICE file(s) distributed with this work for additional + # information regarding copyright ownership. + # + # This program and the accompanying materials are made available under the + # terms of the Apache License Version 2.0 which is available at + # https://www.apache.org/licenses/LICENSE-2.0 + # + # SPDX-License-Identifier: Apache-2.0 + # ******************************************************************************* + +Step 6 — Add feature integration tests +====================================== + + **What this unlocks:** your showcase is **exercised at runtime** on the + deployed images, not just built. This is the difference between "it ships" and + "it provably works on the target". + +Two test frameworks +------------------- + +The integration currently uses **two** separate frameworks for integration +tests, each driven by its own Bazel target and run in a different CI workflow: + +* **FIT — scenario-based feature integration tests** + (`feature_integration_tests/test_cases <../../feature_integration_tests/test_cases>`_, + target ``//feature_integration_tests/test_cases:fit``). Python/Pytest test + cases that orchestrate Rust and C++ scenarios. They are executed as part of the + ``test_and_docs`` workflow on the default Linux platform: + + .. code-block:: bash + + bazel test --config=linux-x86_64 //feature_integration_tests/test_cases:fit + +* **ITF — Integration Test Framework** + (`feature_integration_tests/itf <../../feature_integration_tests/itf>`_, + target ``//feature_integration_tests/itf``). Black-box tests that run against a + booted image. They are executed for **two** images only — QNX (on QEMU) by + `build_and_test_qnx.yml <../../.github/workflows/build_and_test_qnx.yml>`_ and + Linux x86_64 by + `build_and_test_linux.yml <../../.github/workflows/build_and_test_linux.yml>`_: + + .. code-block:: bash + + # QNX image, booted under QEMU + bazel test --config=itf-qnx-x86_64 //feature_integration_tests/itf + # Linux x86_64 image + bazel test --config=linux-x86_64 //feature_integration_tests/itf + +Add black-box tests that exercise your showcase on a running target under +`feature_integration_tests/itf <../../feature_integration_tests/itf>`_. They run +against the deployed image and assert that the binary is present and behaves as +expected: + +.. code-block:: python + + def test_my_example_is_deployed(target): + exit_code, _ = target.execute("test -f /showcases/bin/my_example") + assert exit_code == 0 + + def test_my_example_runs(target): + exit_code, out = target.execute("/showcases/bin/my_example") + assert exit_code == 0 + +The existing ``test_run_all_showcases`` test invokes ``cli --examples=all``, so +any showcase you registered in Step 4 is already covered by the end-to-end run. + +.. note:: + + **Planned consolidation.** Having two integration-test frameworks (FIT and + ITF) executed by different workflows on different platforms is a known gap. + The execution should be consolidated so that **all integration tests — both + FIT and ITF — run against all four target images** (Linux x86_64, QNX, AutoSD, + EBcLfSA aarch64), instead of FIT running only in ``test_and_docs`` on Linux + and ITF running only for the QNX and Linux images. diff --git a/docs/integration_process/step_7_reporting.rst b/docs/integration_process/step_7_reporting.rst new file mode 100644 index 00000000000..97ea0db8ce4 --- /dev/null +++ b/docs/integration_process/step_7_reporting.rst @@ -0,0 +1,118 @@ +.. + # ******************************************************************************* + # Copyright (c) 2026 Contributors to the Eclipse Foundation + # + # See the NOTICE file(s) distributed with this work for additional + # information regarding copyright ownership. + # + # This program and the accompanying materials are made available under the + # terms of the Apache License Version 2.0 which is available at + # https://www.apache.org/licenses/LICENSE-2.0 + # + # SPDX-License-Identifier: Apache-2.0 + # ******************************************************************************* + +.. _reporting: + +Step 7 — Add coverage and verification reports +============================================== + + **What this unlocks:** your module shows up in the **consolidated coverage and + verification / status reports** that are published alongside the docs — the + evidence side of the integration. + +How coverage is collected +------------------------- + +The ``test_and_docs`` workflow runs +`scripts/quality_runners.py <../../scripts/quality_runners.py>`_, which iterates +over **every** module under ``modules.target_sw`` in ``known_good.json`` and, per +module, runs its unit tests with coverage (``bazel coverage``) and then extracts +a per-language summary. The driver is the module's ``metadata.langs``: + +* ``"cpp"`` → a C++ (lcov/genhtml) coverage run. +* ``"rust"`` → a Rust coverage run. + +Both write into the two consolidated reports +`docs/verification_report/unit_test_summary.md <../../docs/verification_report/unit_test_summary.md>`_ +and +`docs/verification_report/coverage_summary.md <../../docs/verification_report/coverage_summary.md>`_. +The same scoping metadata applies to both languages: + +* ``metadata.code_root_path`` — the target pattern that is instrumented and + tested (``@score_my_module``). +* ``metadata.extra_test_config`` — extra ``--flag`` settings for the coverage run. +* ``metadata.exclude_test_targets`` — targets excluded from the run. + +So for most modules **no extra wiring is needed** — declaring the language in +``metadata.langs`` (Step 1) is what enrolls the module into coverage. + +Where the results are exported +------------------------------ + +``quality_runners.py`` does not only run the tests and coverage — it parses the +output per module and **exports the results** in three forms: + +* **Markdown summaries committed into the docs.** A unit-test execution table + and a coverage table are written to + `docs/verification_report/unit_test_summary.md <../../docs/verification_report/unit_test_summary.md>`_ + (columns: module, passed, failed, skipped, total) and + `docs/verification_report/coverage_summary.md <../../docs/verification_report/coverage_summary.md>`_ + (columns: module, lines, functions, branches). These two files are pulled into + the published documentation by + `docs/verification_report/platform_verification_report.rst <../../docs/verification_report/platform_verification_report.rst>`_, + so every module's unit-test and coverage numbers appear in the + **Platform Verification Report** on the docs site. +* **Detailed HTML coverage reports.** The per-module ``genhtml`` / Rust coverage + HTML is written under the coverage output directory + (``artifacts/coverage/cpp/`` and ``artifacts/coverage/rust/`` + by default; override with ``--coverage-output-dir``). +* **CI job summary.** In CI the ``test_and_docs`` workflow additionally appends + the two markdown summaries to the GitHub Actions **job summary** + (``$GITHUB_STEP_SUMMARY``), so the pass/fail and coverage tables are visible + directly on the workflow run. + +C++ coverage +------------ + +If your module declares ``cpp`` in ``metadata.langs``, ``quality_runners.py`` +runs ``bazel coverage`` over ``@score_my_module`` and produces an +HTML/lcov report via ``genhtml`` (the ``lcov`` package must be present — CI +installs it; locally ``sudo apt-get install -y lcov``). The line/function/branch +numbers land in ``coverage_summary.md``. To make your module's C++ coverage +meaningful: + +#. Ensure ``metadata.langs`` contains ``"cpp"`` and ``metadata.code_root_path`` + points at the instrumented source root. +#. Use ``metadata.exclude_test_targets`` to drop tests that should not count + (e.g. examples, flaky or environment-specific targets). +#. Reproduce a single module's run locally with: + + .. code-block:: bash + + python3 scripts/quality_runners.py --modules-to-test score_my_module + +Rust coverage +------------- + +If your module declares ``rust`` in ``metadata.langs``, the +``rust_coverage_report`` target is generated automatically into +`rust_coverage/BUILD <../../rust_coverage/BUILD>`_ by step 1.2 — no manual action +is needed. Tune ``metadata.exclude_test_targets`` and +``metadata.rust_coverage_config`` in ``known_good.json`` to control what is +measured. Build a single module's report with: + +.. code-block:: bash + + bazel run //rust_coverage:rust_coverage_score_my_module + +Verification & status reports +----------------------------- + +The consolidated reports live under +`docs/verification_report <../../docs/verification_report>`_ and the roadmap / +status trackers under +`docs/s_core_v_1/roadmap <../../docs/s_core_v_1/roadmap>`_ (e.g. +``overall_status.rst``). Add your module to the relevant tables/rows so it shows +up in the platform verification report and the feature/process status overview. +The full catalogue of reports is under :ref:`reports`. diff --git a/docs/integration_process/step_8_code_quality.rst b/docs/integration_process/step_8_code_quality.rst new file mode 100644 index 00000000000..d52acbb69ac --- /dev/null +++ b/docs/integration_process/step_8_code_quality.rst @@ -0,0 +1,64 @@ +.. + # ******************************************************************************* + # Copyright (c) 2026 Contributors to the Eclipse Foundation + # + # See the NOTICE file(s) distributed with this work for additional + # information regarding copyright ownership. + # + # This program and the accompanying materials are made available under the + # terms of the Apache License Version 2.0 which is available at + # https://www.apache.org/licenses/LICENSE-2.0 + # + # SPDX-License-Identifier: Apache-2.0 + # ******************************************************************************* + +Step 8 — Code quality +===================== + + **What this unlocks:** your module's source is **statically analysed and + style-checked** on every pull request, on top of the unit, component and + integration tests of the previous steps. + +The integration runs several language-specific code-quality checks. They operate +on ``//...`` (or on the pinned module checkouts), so they pick up your module +automatically once it is in the graph — there is nothing module-specific to wire +in. + +What runs today +--------------- + +* **Formatting checks** (all languages) — + `format.yml <../../.github/workflows/format.yml>`_ runs the shared S-CORE + formatter (a reusable workflow from + `cicd-workflows `_) and fails + if any file is not formatted. Reproduce / fix locally by running the format + targets before pushing. +* **Bazel Clippy** (Rust) — Rust static analysis runs through the ``rules_rust`` + Clippy aspect over the Rust targets, flagging lint violations as build errors. +* **CodeQL multi-repo scan** (C++) — + `codeql-multiple-repo-scan.yml <../../.github/workflows/codeql-multiple-repo-scan.yml>`_ + checks out every module pinned in ``known_good.json`` and runs CodeQL with the + **MISRA C++ coding standards** pack across all of them. + +.. note:: + + **Missing — to be introduced.** There is no **clang-tidy** stage for C++ yet. + clang-tidy is planned to complement the CodeQL / MISRA scan for C++ static + analysis, but it is currently not wired into the integration or its CI. Until + then, C++ static analysis relies on the CodeQL scan only. + +Generated reports +----------------- + +The CodeQL scan exports its findings as build artifacts on the workflow run: + +* a **SARIF** result file (``sarif-results/cpp.sarif``), uploaded as the + ``codeql-sarif-results`` artifact, and +* a human-readable **HTML report** (``codeql-report.html``), generated from the + SARIF with ``sarif-tools`` and uploaded as the ``codeql-html-report`` + artifact. + +Download these from the *Artifacts* section of the CodeQL workflow run to review +the C++ findings. The formatting and Clippy checks do not produce a separate +report — they pass or fail directly on the workflow run, with the offending +files / lints shown in the job log. From d34c639792aacc5cc98fc00a6ac0e91a606ce5e0 Mon Sep 17 00:00:00 2001 From: Anton Krivoborodov <63401640+antonkri@users.noreply.github.com> Date: Fri, 19 Jun 2026 06:24:58 +0000 Subject: [PATCH 2/2] fix review findings --- .../integration_process.rst | 11 ++- docs/integration_process/reference.rst | 48 ++++++------ .../integration_process/step_1_add_module.rst | 47 ++++++++--- .../step_2_documentation.rst | 27 ++++--- .../integration_process/step_3_unit_tests.rst | 27 ++++--- docs/integration_process/step_4_platforms.rst | 77 ++++++++++--------- .../step_5_component_tests.rst | 7 +- .../step_6_integration_tests.rst | 19 +++-- docs/integration_process/step_7_reporting.rst | 36 ++++++--- .../step_8_code_quality.rst | 13 ++-- 10 files changed, 190 insertions(+), 122 deletions(-) diff --git a/docs/integration_process/integration_process.rst b/docs/integration_process/integration_process.rst index fb0e92706b1..55a21d828af 100644 --- a/docs/integration_process/integration_process.rst +++ b/docs/integration_process/integration_process.rst @@ -17,7 +17,9 @@ Integration Process ################### This document is a step-by-step *how-to* for module owners who want to integrate -their own S-CORE module into the **Reference Integration** for the first time. +their own S-CORE module into the **Reference Integration** — whether for the +first time, when bringing it back after an exclusion, or to make it part of the +S-CORE releases in general. It walks you through everything from registering the module in ``known_good.json`` over wiring it into the Bazel build, the showcases, the @@ -54,7 +56,12 @@ Before you start, make sure that: * The module name follows the ``score_`` convention (e.g. ``score_communication``). The same name is used as the Bazel ``bazel_dep`` / repository name (``@score_//...``). -* The module builds and tests pass standalone on Linux ``x86_64``. +* Your module is registered in the + `S-CORE Bazel registry `_ so + that it can be resolved as a ``bazel_dep`` during the integration. +* The module builds and tests pass standalone on all four currently supported + target platforms: Linux ``x86_64``, QNX 8 ``x86_64``, Red Hat AutoSD + ``x86_64`` and EB corbos Linux (Safety Apps) ``aarch64``. * You know the commit hash you want to pin (the integration always pins an exact commit, never a floating branch). diff --git a/docs/integration_process/reference.rst b/docs/integration_process/reference.rst index 56193ea6bdf..9b01c4653c6 100644 --- a/docs/integration_process/reference.rst +++ b/docs/integration_process/reference.rst @@ -26,7 +26,7 @@ and their images are listed in :ref:`supported_platforms` (Step 4). CI checks --------- -The pipelines live under `.github/workflows <../../.github/workflows>`_. Most +The pipelines live under `.github/workflows `_. Most operate on ``//...`` and pick up your module without changes; the table says what each check verifies and whether you typically need to touch anything. @@ -46,61 +46,61 @@ declared in the workflows. - What it verifies - Gating - Your action - * - `build_and_test_qnx.yml <../../.github/workflows/build_and_test_qnx.yml>`_ + * - `build_and_test_qnx.yml `_ - Builds the QNX IFS and runs integration tests on QEMU. - **yes** - none (graph-wide) - * - `build_and_test_autosd.yml <../../.github/workflows/build_and_test_autosd.yml>`_ + * - `build_and_test_autosd.yml `_ - Builds the AutoSD OCI image with the Automotive Image Builder. - **yes** - none (graph-wide) - * - `build_and_test_ebclfsa.yml <../../.github/workflows/build_and_test_ebclfsa.yml>`_ + * - `build_and_test_ebclfsa.yml `_ - Builds the EBcLfSA image and runs the high-integrity safety tests. - **yes** - none (graph-wide) - * - `known_good_correct.yml <../../.github/workflows/known_good_correct.yml>`_ + * - `known_good_correct.yml `_ - Fails if the generated Bazel fragments drift from ``known_good.json``. - **yes** - commit step 1.2 output - * - `bzlmod-lock.yml <../../.github/workflows/bzlmod-lock.yml>`_ + * - `bzlmod-lock.yml `_ - Verifies ``MODULE.bazel.lock`` is consistent with the module graph. - **yes** - update lockfile if it fails - * - `format.yml <../../.github/workflows/format.yml>`_ + * - `format.yml `_ - Runs the code-formatting checks. - **yes** - run the format targets - * - `codeql-multiple-repo-scan.yml <../../.github/workflows/codeql-multiple-repo-scan.yml>`_ + * - `codeql-multiple-repo-scan.yml `_ - Multi-repository CodeQL security scan across the integrated modules. - **yes** - none - * - `test_and_docs.yml <../../.github/workflows/test_and_docs.yml>`_ + * - `test_and_docs.yml `_ - Code-quality checks, builds the docs and reports, publishes to Pages. - **yes** - wire docs in Step 2, reports in Step 7 - * - `check_release_approvals.yml <../../.github/workflows/check_release_approvals.yml>`_ + * - `check_release_approvals.yml `_ - Enforces required approvals on PRs targeting ``releases/*`` branches. - release branches only - none - * - `build_and_test_linux.yml <../../.github/workflows/build_and_test_linux.yml>`_ + * - `build_and_test_linux.yml `_ - Builds the Linux x86_64 image and runs feature integration tests on Docker. - no - none (graph-wide) - * - `internal_tests.yml <../../.github/workflows/internal_tests.yml>`_ + * - `internal_tests.yml `_ - Runs the integration tooling tests (``//scripts/tooling:tooling_tests``). - no - only if you change scripts - * - `docs_cleanup.yml <../../.github/workflows/docs_cleanup.yml>`_ + * - `docs_cleanup.yml `_ - Scheduled cleanup of published documentation versions. - no - none - * - `test_integration.yml <../../.github/workflows/test_integration.yml>`_ / - `reusable_smoke-test.yml <../../.github/workflows/reusable_smoke-test.yml>`_ / - `reusable_integration-build.yml <../../.github/workflows/reusable_integration-build.yml>`_ + * - `test_integration.yml `_ / + `reusable_smoke-test.yml `_ / + `reusable_integration-build.yml `_ - Smoke-test / reusable build of the latest module ``main`` branches. - no - add runtime targets to - `ci/showcase_targets_run.txt <../../ci/showcase_targets_run.txt>`_ + `ci/showcase_targets_run.txt `_ .. _reports: @@ -119,30 +119,30 @@ The consolidated outputs published by the integration. They are built by - Source / target * - Consolidated documentation - All integrated module docs merged into one Sphinx site. - - `BUILD <../../BUILD>`_ ``docs(...)`` → ``bazel run //:docs_combo`` + - `BUILD `_ ``docs(...)`` → ``bazel run //:docs_combo`` * - Platform verification report - Per-release requirements/architecture verification, safety analyses and per-test-case results. - - `docs/verification_report/platform_verification_report.rst <../../docs/verification_report/platform_verification_report.rst>`_ + - `docs/verification_report/platform_verification_report.rst `_ * - Unit test summary - Per-module unit-test execution table (generated at build time by ``quality_runners.py``). - - `docs/verification_report/unit_test_summary.md <../../docs/verification_report/unit_test_summary.md>`_ + - `docs/verification_report/unit_test_summary.md `_ * - Coverage summary - Per-module C++ and Rust coverage table (generated at build time by ``quality_runners.py``). - - `docs/verification_report/coverage_summary.md <../../docs/verification_report/coverage_summary.md>`_ + - `docs/verification_report/coverage_summary.md `_ * - C++ coverage (per module) - lcov/genhtml line/function/branch report for modules declaring ``cpp``. - - `scripts/quality_runners.py <../../scripts/quality_runners.py>`_ → + - `scripts/quality_runners.py `_ → ``python3 scripts/quality_runners.py --modules-to-test `` * - Rust coverage reports - Per-module Rust line coverage (C0/C1) for modules declaring ``rust``. - - `rust_coverage/BUILD <../../rust_coverage/BUILD>`_ → + - `rust_coverage/BUILD `_ → ``bazel run //rust_coverage:rust_coverage_`` * - Overall feature & process status - Feature/process completion dashboard derived from the pinned module repos. - - `docs/s_core_v_1/roadmap/overall_status.rst <../../docs/s_core_v_1/roadmap/overall_status.rst>`_ + - `docs/s_core_v_1/roadmap/overall_status.rst `_ * - Integration status dashboard - Live build/health overview of the integration. - `status_dashboard.html `_ diff --git a/docs/integration_process/step_1_add_module.rst b/docs/integration_process/step_1_add_module.rst index 54723beacd2..21f415b2659 100644 --- a/docs/integration_process/step_1_add_module.rst +++ b/docs/integration_process/step_1_add_module.rst @@ -15,10 +15,13 @@ Step 1 — Adding module to reference integration ================================================= - **What this unlocks:** your module becomes a *known* part of the integration, - pinned at an exact commit, enters the **Bazel module graph** and builds - cleanly against every other module. This is the foundation every later step - builds on — afterwards ``@score_my_module//...`` resolves and any other +.. admonition:: What it unlocks + :class: tip + + **Module in the graph** — Your module becomes a *known* part of the + integration, pinned at an exact commit, enters the **Bazel module graph** and + builds cleanly against every other module. This is the foundation every later + step builds on — afterwards ``@score_my_module//...`` resolves and any other target can depend on your module. 1.1 Register the module in ``known_good.json`` @@ -28,7 +31,7 @@ Step 1 — Adding module to reference integration which commit — are part of the integration. Every other generated file is derived from it, so this is always the first change. -Open `known_good.json <../../known_good.json>`_. It has two top-level groups +Open `known_good.json `_. It has two top-level groups under ``modules``: * ``target_sw`` — the S-CORE software modules that are built and shipped @@ -114,14 +117,28 @@ hand. After editing the JSON, regenerate them: This (re)writes: -* `bazel_common/score_modules_target_sw.MODULE.bazel <../../bazel_common/score_modules_target_sw.MODULE.bazel>`_ +* `bazel_common/score_modules_target_sw.MODULE.bazel `_ — a ``bazel_dep`` + ``git_override`` block for your module. This file is pulled into the build via ``include(...)`` in - `MODULE.bazel <../../MODULE.bazel>`_. -* `rust_coverage/BUILD <../../rust_coverage/BUILD>`_ — a + `MODULE.bazel `_. +* `rust_coverage/BUILD `_ — a ``rust_coverage_report`` target for every module that declares ``rust`` in its ``langs`` (skipped for C++-only modules). +Next, refresh the Bazel lockfile so it reflects the updated module graph: + +.. code-block:: bash + + bazel mod deps --lockfile_mode=update + +Adding your module introduces a new ``bazel_dep`` / ``git_override`` into the +module graph, which changes the set of resolved dependencies. Running ``bazel +mod deps`` with ``--lockfile_mode=update`` re-resolves the graph and writes the +new dependencies into `MODULE.bazel.lock `_. Committing +an up-to-date lockfile keeps the dependency resolution reproducible for everyone +and is required by the ``Bzlmod Lockfile Check`` CI job (see section 1.4), which +fails if the committed lockfile is out of sync with the module graph. + Commit the regenerated files together with the ``known_good.json`` change. CI verifies that the generated files are in sync with the JSON (see the ``known_good_correct`` workflow), so a manual edit that drifts from the JSON @@ -135,9 +152,17 @@ will fail. If the module does not build cleanly out-of-tree (e.g. label visibility issues), add patch files under ``patches//`` and reference them from the ``bazel_patches`` list of the module entry in ``known_good.json``. See -`patches/communication <../../patches/communication>`_ for a worked example. The +`patches/communication `_ for a worked example. The generator emits ``patches = [...]`` with ``patch_strip = 1`` automatically. +.. note:: + + Patching a module should be the **exception, not the rule**. A patch keeps a + local divergence from the upstream module that has to be maintained and + re-based on every update. Prefer fixing the underlying issue in the module's + own repository so it builds cleanly out-of-tree, and only reach for a patch + as a temporary, last-resort workaround. + 1.4 CI checks that the module was added correctly ------------------------------------------------- @@ -153,14 +178,14 @@ check that the generated metadata is in sync with ``known_good.json``: - What it verifies - Fix if it fails * - Known Good Matches Bazel - (`known_good_correct.yml <../../.github/workflows/known_good_correct.yml>`_) + (`known_good_correct.yml `_) - Re-runs ``update_module_from_known_good.py`` and fails if the generated Bazel fragments differ from what you committed — i.e. ``known_good.json`` and the generated ``*.MODULE.bazel`` / ``rust_coverage/BUILD`` are out of sync. - re-run step 1.2 and commit the regenerated files * - Bzlmod Lockfile Check - (`bzlmod-lock.yml <../../.github/workflows/bzlmod-lock.yml>`_) + (`bzlmod-lock.yml `_) - Verifies ``MODULE.bazel.lock`` is consistent with the updated module graph (your new ``bazel_dep`` / ``git_override``). - refresh and commit ``MODULE.bazel.lock`` diff --git a/docs/integration_process/step_2_documentation.rst b/docs/integration_process/step_2_documentation.rst index 12849471dee..3584547f1fc 100644 --- a/docs/integration_process/step_2_documentation.rst +++ b/docs/integration_process/step_2_documentation.rst @@ -15,17 +15,20 @@ Step 2 — Join the documentation and process-compliance checks ============================================================= - **What this unlocks:** your module's documentation appears in the single, - combined docs site that is built and published for the whole integration — - and, just as importantly, your module's **requirements, architecture and - other process artifacts are validated against the S-CORE process** as part of - the same build. This is usually the first thing you *extend the integration - with* after the module is in the graph. +.. admonition:: What it unlocks + :class: tip + + **Docs & process checks** — Your module's documentation appears in the + single, combined docs site that is built and published for the whole + integration — and, just as importantly, your module's **requirements, + architecture and other process artifacts are validated against the S-CORE + process** as part of the same build. This is usually the first thing you + *extend the integration with* after the module is in the graph. Building the docs is not only about rendering pages. The integration uses the docs-as-code toolchain (`score_docs_as_code `_, pulled in via -``score_sphinx_bundle`` in `docs/conf.py <../../docs/conf.py>`_), which runs the +``score_sphinx_bundle`` in `docs/conf.py `_), which runs the **S-CORE metamodel checks** on every ``needs`` object: requirements, architecture elements, safety artifacts and their links must conform to the process model defined in `score_process @@ -38,7 +41,7 @@ The integration builds one Sphinx site that merges the docs of every integrated module. To pull yours in: #. Add your module's ``needs_json`` to the ``docs(...)`` rule's ``data`` list in - the top-level `BUILD <../../BUILD>`_ file: + the top-level `BUILD `_ file: .. code-block:: python @@ -67,18 +70,18 @@ module. To pull yours in: - Page to edit - Put your module here if it is… * - **Modules** - - `docs/sw_components.rst <../../docs/sw_components.rst>`_ + - `docs/sw_components.rst `_ - an S-CORE **software module** that ships in the integration — i.e. it lives under ``modules.target_sw`` in ``known_good.json`` (communication, persistency, logging, orchestrator, baselibs, …). This is the common case. * - **Process, Methods & Tools** - - `docs/process_methods_tools.rst <../../docs/process_methods_tools.rst>`_ + - `docs/process_methods_tools.rst `_ - a **tooling / process** repo from ``modules.tooling`` (platform, process, docs-as-code) rather than a shipped software module. For a normal software module, add the entry to the ``Modules`` toctree in - `docs/sw_components.rst <../../docs/sw_components.rst>`_, next to the existing + `docs/sw_components.rst `_, next to the existing modules: .. code-block:: rst @@ -124,7 +127,7 @@ The docs are built and published by the ``test_and_docs`` workflow (see **Planned refactoring.** Today documentation generation is entangled with unit tests, coverage and feature integration tests in the single ``test_and_docs`` workflow - (`test_and_docs.yml <../../.github/workflows/test_and_docs.yml>`_). This + (`test_and_docs.yml `_). This should be refactored: the documentation build (including the metamodel / process-compliance checks) belongs in a **dedicated, reusable workflow shared across all S-CORE repositories**, hosted centrally in the diff --git a/docs/integration_process/step_3_unit_tests.rst b/docs/integration_process/step_3_unit_tests.rst index 820428c0927..149fcc31db5 100644 --- a/docs/integration_process/step_3_unit_tests.rst +++ b/docs/integration_process/step_3_unit_tests.rst @@ -15,14 +15,17 @@ Step 3 — Get compiled and unit-tested on the default platform ============================================================= - **What this unlocks:** every module registered in ``known_good.json`` is - automatically **compiled and run through its unit tests** on the default - platform (Linux ``x86_64``, ``--config=linux-x86_64``) — no per-module wiring - is needed. As soon as your module is in the graph (Step 1), its unit tests - are part of the integration's quality gate. +.. admonition:: What it unlocks + :class: tip + + **Default-platform unit tests** — Every module registered in + ``known_good.json`` is automatically **compiled and run through its unit + tests** on the default platform (Linux ``x86_64``, ``--config=linux-x86_64``) + — no per-module wiring is needed. As soon as your module is in the graph + (Step 1), its unit tests are part of the integration's quality gate. Unit tests are driven by -`scripts/quality_runners.py <../../scripts/quality_runners.py>`_, which loads +`scripts/quality_runners.py `_, which loads ``known_good.json`` and iterates over **every** module under ``modules.target_sw``. For each module it runs ``bazel coverage`` with the ``unit-tests`` config, which builds and tests the module's source tree on the @@ -54,9 +57,13 @@ You do not edit ``quality_runners.py`` to control what is tested — the per-mod * ``metadata.extra_test_config`` — extra ``--flag=value`` build/test settings injected for this module (feature flags, config selections). * ``metadata.exclude_test_targets`` — test targets that must **not** run - (flaky, environment-specific or not meaningful out-of-tree); they are passed as - ``-@score_my_module`` exclusions. Supports wildcards such as - ``//score/json/examples:*``. + (flaky, environment-specific or not meaningful out-of-tree). You list plain + module-relative labels such as ``//src/cpp/tests:bm_kvs_cpp``; the runner + automatically turns each into a ``-@score_my_module`` exclusion. + Supports wildcards such as ``//score/json/examples:*``. +* ``metadata.rust_coverage_config`` (default ``ferrocene-coverage``) — selects + the Rust coverage configuration applied when running ``bazel coverage`` for + this module (e.g. ``ferrocene-coverage-per``). So enrolling your module into unit testing requires no extra step beyond Step 1 — you only adjust these fields when the defaults do not fit. @@ -67,7 +74,7 @@ coverage reports and the CI job summary). How and where exactly is described in Step 7, see :ref:`reporting`. This runs in the ``test_and_docs`` workflow -(`test_and_docs.yml <../../.github/workflows/test_and_docs.yml>`_), in the same +(`test_and_docs.yml `_), in the same job as the docs build (see :ref:`ci_checks`). .. note:: diff --git a/docs/integration_process/step_4_platforms.rst b/docs/integration_process/step_4_platforms.rst index 6054a6e35ec..6fa7d4f598f 100644 --- a/docs/integration_process/step_4_platforms.rst +++ b/docs/integration_process/step_4_platforms.rst @@ -15,15 +15,20 @@ Step 4 — Get built and tested on the target platforms ===================================================== - **What this unlocks:** your module is actually **compiled and tested by the - pipelines** — built for and deployed to every target platform (Linux x86_64, - QNX, AutoSD, EBcLfSA aarch64), offered in the interactive CLI, and exercised - by CI. This is the rung that turns a library into something that *runs* inside - the integration images. +.. admonition:: What it unlocks + :class: tip -The integration ships four target images, each built and tested by its own -gating CI job. The table lists the platforms, the image target, the runner used -to execute it, and the per-platform CI check. + **Target-platform builds & tests** — Your module is actually **compiled and + tested by the pipelines** — built for and deployed to every target platform + (Linux x86_64, QNX, AutoSD, EBcLfSA aarch64), offered in the interactive CLI, + and exercised by CI. This is the rung that turns a library into something + that *runs* inside the integration images. + +The integration provides build definitions for four target images, each built +and tested by its own gating CI job — it does not distribute ready-to-flash +images; you build them yourself from these definitions. The table lists the +platforms, the image target, the runner used to execute it, and the per-platform +CI check. .. _supported_platforms: @@ -38,30 +43,30 @@ to execute it, and the per-platform CI check. - Build & test workflow * - Linux - x86_64 - - `images/linux_x86_64 <../../images/linux_x86_64>`_ - - Docker (`runners/docker_x86_64 <../../runners/docker_x86_64>`_) - - `build_and_test_linux.yml <../../.github/workflows/build_and_test_linux.yml>`_ + - `images/linux_x86_64 `_ + - Docker (`runners/docker_x86_64 `_) + - `build_and_test_linux.yml `_ * - QNX 8 - x86_64 - - `images/qnx_x86_64 <../../images/qnx_x86_64>`_ - - QEMU (`runners/qemu_x86_64 <../../runners/qemu_x86_64>`_) - - `build_and_test_qnx.yml <../../.github/workflows/build_and_test_qnx.yml>`_ + - `images/qnx_x86_64 `_ + - QEMU (`runners/qemu_x86_64 `_) + - `build_and_test_qnx.yml `_ * - Red Hat AutoSD - x86_64 - - `images/autosd <../../images/autosd>`_ - - Docker (`runners/docker_x86_64 <../../runners/docker_x86_64>`_) - - `build_and_test_autosd.yml <../../.github/workflows/build_and_test_autosd.yml>`_ + - `images/autosd `_ + - Docker (`runners/docker_x86_64 `_) + - `build_and_test_autosd.yml `_ * - EB corbos Linux (Safety Apps) - aarch64 - - `images/ebclfsa_aarch64 <../../images/ebclfsa_aarch64>`_ - - QEMU (`runners/qemu_aarch64 <../../runners/qemu_aarch64>`_) - - `build_and_test_ebclfsa.yml <../../.github/workflows/build_and_test_ebclfsa.yml>`_ + - `images/ebclfsa_aarch64 `_ + - QEMU (`runners/qemu_aarch64 `_) + - `build_and_test_ebclfsa.yml `_ .. note:: The EB corbos *Safety Applications* image is special: it runs a high-integrity application demo under additional constraints. Read - `images/ebclfsa_aarch64/README.md <../../images/ebclfsa_aarch64/README.md>`_ + `images/ebclfsa_aarch64/README.md `_ before targeting it. Run any image locally in QEMU @@ -79,7 +84,7 @@ try your module on the real target without hardware: The QNX and aarch64 images boot under QEMU, which needs a one-time host setup (TUN device, bridge helper, libvirt network). Follow the QEMU runner how-to in -`runners/qemu_x86_64/README.md <../../runners/qemu_x86_64/README.md>`_ before the +`runners/qemu_x86_64/README.md `_ before the first run. Two ways to get your module onto the images @@ -96,25 +101,25 @@ There are two ways to make your binary part of the images: (once per image) and is only needed for components that are not run as a CLI showcase — typically a daemon. The ``datarouter`` is integrated this way; see how the QNX image does it in - `images/qnx_x86_64/build/BUILD <../../images/qnx_x86_64/build/BUILD>`_ (the + `images/qnx_x86_64/build/BUILD `_ (the ``datarouter`` source and the ``DATAROUTER_PATH`` mapping). See :ref:`add_directly` below. Prefer the showcase route unless you have a reason not to. .. _add_showcase: -4.1 Add a runtime showcase --------------------------- +Add a runtime showcase +---------------------- A *showcase* is the vehicle that gets your module onto the platforms: you bundle a runnable binary, and the aggregate ``//showcases`` package is deployed into **every** image automatically. You do this once — there is no per-platform work. -There are two patterns under `showcases/ <../../showcases>`_: +There are two patterns under `showcases/ `_: -* **Standalone** (`showcases/standalone <../../showcases/standalone>`_) — bundle +* **Standalone** (`showcases/standalone `_) — bundle a binary that already exists in your module's repo. * **Composed** (e.g. - `showcases/orchestration_persistency <../../showcases/orchestration_persistency>`_) + `showcases/orchestration_persistency `_) — a small binary defined *here* that combines several modules. **1. Define the bundle.** Add a ``score_pkg_bundle`` target. To re-use a binary @@ -152,7 +157,7 @@ offers it in the interactive menu, so **no code change to the CLI is required**: **3. Register the bundle in the top-level showcase package.** Add your bundle to the aggregate ``score_pkg_bundle`` in -`showcases/BUILD <../../showcases/BUILD>`_ so it is packaged into the images: +`showcases/BUILD `_ so it is packaged into the images: .. code-block:: python @@ -173,8 +178,8 @@ job** automatically. .. _add_directly: -4.2 Add a binary directly to an image -------------------------------------- +Add a binary directly to an image +--------------------------------- If your component is not a CLI showcase — for example a background daemon that must always be present — you wire it straight into each image's build @@ -202,13 +207,13 @@ location mapping, then placed into the filesystem by ``system.build``: }, ) -See `images/qnx_x86_64/build/BUILD <../../images/qnx_x86_64/build/BUILD>`_ and the +See `images/qnx_x86_64/build/BUILD `_ and the matching deployment lines in -`images/qnx_x86_64/build/system.build <../../images/qnx_x86_64/build/system.build>`_ +`images/qnx_x86_64/build/system.build `_ for the full picture, and repeat the equivalent wiring in the other images you -need (`images/linux_x86_64 <../../images/linux_x86_64>`_, -`images/autosd <../../images/autosd>`_, -`images/ebclfsa_aarch64 <../../images/ebclfsa_aarch64>`_). +need (`images/linux_x86_64 `_, +`images/autosd `_, +`images/ebclfsa_aarch64 `_). If your module simply cannot run on a given platform, exclude its targets via ``metadata.exclude_test_targets`` / ``metadata.extra_test_config`` in diff --git a/docs/integration_process/step_5_component_tests.rst b/docs/integration_process/step_5_component_tests.rst index b2b0915c8a2..24a8ef8e845 100644 --- a/docs/integration_process/step_5_component_tests.rst +++ b/docs/integration_process/step_5_component_tests.rst @@ -15,8 +15,11 @@ Step 5 — Add component tests ============================ - **What this unlocks:** your module is verified **in isolation as a - component** — its public interfaces are tested against their specification, +.. admonition:: What it unlocks + :class: tip + + **Component-level verification** — Your module is verified **in isolation as + a component** — its public interfaces are tested against their specification, independently of the other modules and of the full integration images. .. warning:: diff --git a/docs/integration_process/step_6_integration_tests.rst b/docs/integration_process/step_6_integration_tests.rst index fb038ce7a2b..01f9835486c 100644 --- a/docs/integration_process/step_6_integration_tests.rst +++ b/docs/integration_process/step_6_integration_tests.rst @@ -15,9 +15,12 @@ Step 6 — Add feature integration tests ====================================== - **What this unlocks:** your showcase is **exercised at runtime** on the - deployed images, not just built. This is the difference between "it ships" and - "it provably works on the target". +.. admonition:: What it unlocks + :class: tip + + **Runtime feature tests** — Your showcase is **exercised at runtime** on the + deployed images, not just built. This is the difference between "it ships" + and "it provably works on the target". Two test frameworks ------------------- @@ -26,7 +29,7 @@ The integration currently uses **two** separate frameworks for integration tests, each driven by its own Bazel target and run in a different CI workflow: * **FIT — scenario-based feature integration tests** - (`feature_integration_tests/test_cases <../../feature_integration_tests/test_cases>`_, + (`feature_integration_tests/test_cases `_, target ``//feature_integration_tests/test_cases:fit``). Python/Pytest test cases that orchestrate Rust and C++ scenarios. They are executed as part of the ``test_and_docs`` workflow on the default Linux platform: @@ -36,12 +39,12 @@ tests, each driven by its own Bazel target and run in a different CI workflow: bazel test --config=linux-x86_64 //feature_integration_tests/test_cases:fit * **ITF — Integration Test Framework** - (`feature_integration_tests/itf <../../feature_integration_tests/itf>`_, + (`feature_integration_tests/itf `_, target ``//feature_integration_tests/itf``). Black-box tests that run against a booted image. They are executed for **two** images only — QNX (on QEMU) by - `build_and_test_qnx.yml <../../.github/workflows/build_and_test_qnx.yml>`_ and + `build_and_test_qnx.yml `_ and Linux x86_64 by - `build_and_test_linux.yml <../../.github/workflows/build_and_test_linux.yml>`_: + `build_and_test_linux.yml `_: .. code-block:: bash @@ -51,7 +54,7 @@ tests, each driven by its own Bazel target and run in a different CI workflow: bazel test --config=linux-x86_64 //feature_integration_tests/itf Add black-box tests that exercise your showcase on a running target under -`feature_integration_tests/itf <../../feature_integration_tests/itf>`_. They run +`feature_integration_tests/itf `_. They run against the deployed image and assert that the binary is present and behaves as expected: diff --git a/docs/integration_process/step_7_reporting.rst b/docs/integration_process/step_7_reporting.rst index 97ea0db8ce4..f41ca81aee1 100644 --- a/docs/integration_process/step_7_reporting.rst +++ b/docs/integration_process/step_7_reporting.rst @@ -17,15 +17,27 @@ Step 7 — Add coverage and verification reports ============================================== - **What this unlocks:** your module shows up in the **consolidated coverage and - verification / status reports** that are published alongside the docs — the - evidence side of the integration. +.. admonition:: What it unlocks + :class: tip + + **Consolidated reports** — Your module shows up in the **consolidated + coverage and verification / status reports** that are published alongside the + docs — the evidence side of the integration. + +.. note:: + + **No extra action is required here.** As already explained in + :doc:`Step 3 `, the ``test_and_docs`` workflow already + executes your module's unit tests and gathers and publishes the code-coverage + data. If your module passes Step 3, it is automatically part of the reports + described below — this chapter only explains *how* that collection, + aggregation and publishing happens. How coverage is collected ------------------------- The ``test_and_docs`` workflow runs -`scripts/quality_runners.py <../../scripts/quality_runners.py>`_, which iterates +`scripts/quality_runners.py `_, which iterates over **every** module under ``modules.target_sw`` in ``known_good.json`` and, per module, runs its unit tests with coverage (``bazel coverage``) and then extracts a per-language summary. The driver is the module's ``metadata.langs``: @@ -34,9 +46,9 @@ a per-language summary. The driver is the module's ``metadata.langs``: * ``"rust"`` → a Rust coverage run. Both write into the two consolidated reports -`docs/verification_report/unit_test_summary.md <../../docs/verification_report/unit_test_summary.md>`_ +`docs/verification_report/unit_test_summary.md `_ and -`docs/verification_report/coverage_summary.md <../../docs/verification_report/coverage_summary.md>`_. +`docs/verification_report/coverage_summary.md `_. The same scoping metadata applies to both languages: * ``metadata.code_root_path`` — the target pattern that is instrumented and @@ -55,12 +67,12 @@ output per module and **exports the results** in three forms: * **Markdown summaries committed into the docs.** A unit-test execution table and a coverage table are written to - `docs/verification_report/unit_test_summary.md <../../docs/verification_report/unit_test_summary.md>`_ + `docs/verification_report/unit_test_summary.md `_ (columns: module, passed, failed, skipped, total) and - `docs/verification_report/coverage_summary.md <../../docs/verification_report/coverage_summary.md>`_ + `docs/verification_report/coverage_summary.md `_ (columns: module, lines, functions, branches). These two files are pulled into the published documentation by - `docs/verification_report/platform_verification_report.rst <../../docs/verification_report/platform_verification_report.rst>`_, + `docs/verification_report/platform_verification_report.rst `_, so every module's unit-test and coverage numbers appear in the **Platform Verification Report** on the docs site. * **Detailed HTML coverage reports.** The per-module ``genhtml`` / Rust coverage @@ -97,7 +109,7 @@ Rust coverage If your module declares ``rust`` in ``metadata.langs``, the ``rust_coverage_report`` target is generated automatically into -`rust_coverage/BUILD <../../rust_coverage/BUILD>`_ by step 1.2 — no manual action +`rust_coverage/BUILD `_ by step 1.2 — no manual action is needed. Tune ``metadata.exclude_test_targets`` and ``metadata.rust_coverage_config`` in ``known_good.json`` to control what is measured. Build a single module's report with: @@ -110,9 +122,9 @@ Verification & status reports ----------------------------- The consolidated reports live under -`docs/verification_report <../../docs/verification_report>`_ and the roadmap / +`docs/verification_report `_ and the roadmap / status trackers under -`docs/s_core_v_1/roadmap <../../docs/s_core_v_1/roadmap>`_ (e.g. +`docs/s_core_v_1/roadmap `_ (e.g. ``overall_status.rst``). Add your module to the relevant tables/rows so it shows up in the platform verification report and the feature/process status overview. The full catalogue of reports is under :ref:`reports`. diff --git a/docs/integration_process/step_8_code_quality.rst b/docs/integration_process/step_8_code_quality.rst index d52acbb69ac..667b01205fb 100644 --- a/docs/integration_process/step_8_code_quality.rst +++ b/docs/integration_process/step_8_code_quality.rst @@ -15,9 +15,12 @@ Step 8 — Code quality ===================== - **What this unlocks:** your module's source is **statically analysed and - style-checked** on every pull request, on top of the unit, component and - integration tests of the previous steps. +.. admonition:: What it unlocks + :class: tip + + **Static analysis & style checks** — Your module's source is **statically + analysed and style-checked** on every pull request, on top of the unit, + component and integration tests of the previous steps. The integration runs several language-specific code-quality checks. They operate on ``//...`` (or on the pinned module checkouts), so they pick up your module @@ -28,7 +31,7 @@ What runs today --------------- * **Formatting checks** (all languages) — - `format.yml <../../.github/workflows/format.yml>`_ runs the shared S-CORE + `format.yml `_ runs the shared S-CORE formatter (a reusable workflow from `cicd-workflows `_) and fails if any file is not formatted. Reproduce / fix locally by running the format @@ -36,7 +39,7 @@ What runs today * **Bazel Clippy** (Rust) — Rust static analysis runs through the ``rules_rust`` Clippy aspect over the Rust targets, flagging lint violations as build errors. * **CodeQL multi-repo scan** (C++) — - `codeql-multiple-repo-scan.yml <../../.github/workflows/codeql-multiple-repo-scan.yml>`_ + `codeql-multiple-repo-scan.yml `_ checks out every module pinned in ``known_good.json`` and runs CodeQL with the **MISRA C++ coding standards** pack across all of them.