Skip to content

Fix issue DPTP-4756 Add CF template#5263

Open
bear-redhat wants to merge 1 commit into
openshift:mainfrom
bear-redhat:issue/DPTP-4756
Open

Fix issue DPTP-4756 Add CF template#5263
bear-redhat wants to merge 1 commit into
openshift:mainfrom
bear-redhat:issue/DPTP-4756

Conversation

@bear-redhat

@bear-redhat bear-redhat commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

AWS STS Role Chaining Infrastructure

This PR adds three AWS CloudFormation templates to establish a secure, cross-account role chaining mechanism for CI test execution using OIDC-based authentication instead of static AWS credentials.

The Role Chain Architecture

The templates implement a three-hop assume-role chain that enables per-test pods to authenticate to AWS without embedding credentials in the cluster:

  1. Home Role (home-role.cf): Deployed once in each build cluster's AWS account, this role trusts the cluster's OIDC provider. It allows per-test ServiceAccount tokens to be exchanged for temporary credentials via sts:AssumeRoleWithWebIdentity, restricted to test namespaces matching the pattern ci-op-* with the audience sts.amazonaws.com.

  2. Hub Role (hub-role.cf): Deployed centrally in a shared hub account, this role trusts all home roles from build clusters and permits assuming target roles. The hub role serves as a central trust anchor that allows the architecture to scale to multiple target accounts without requiring each to maintain a list of all build cluster home roles.

  3. Target Role (target-role.cf): Deployed in each AWS account that tests need to access (e.g., aws-5 account), this role trusts only the hub role and provides comprehensive AWS permissions matching the legacy origin-ci-robot-provision user. It grants full access to CloudFormation, EC2, S3, IAM, Route53, ECR, EFS, Lambda, Secrets Manager, Elastic Load Balancing, and related services—all required for CI infrastructure provisioning and testing.

Authentication Flow

When a CI test pod requires AWS access:

  1. The pod uses a projected Kubernetes ServiceAccount token with audience sts.amazonaws.com
  2. The AWS SDK calls sts:AssumeRoleWithWebIdentity against the home role using the token
  3. The home role assumes the hub role to get temporary credentials
  4. The hub role assumes the appropriate target role for the required AWS account
  5. The test executes with the target role's permissions

Key Benefits

  • No Static Credentials: Eliminates need to store AWS API keys in cluster secrets; uses Kubernetes' projected ServiceAccount tokens instead
  • Scalable: Adding new build clusters or target accounts only requires parameter updates to the hub role stack
  • Auditable: All AWS API calls are traceable back to specific test namespaces and ServiceAccounts
  • Least Privilege: The trust relationships enforce specific conditions (namespace patterns, audience constraints) preventing unauthorized assume-role operations

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: automatic mode

@coderabbitai

coderabbitai Bot commented Jun 19, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@bear-redhat, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 15 minutes and 42 seconds. Learn how PR review limits work.

Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file).

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits.

🚦 How do rate limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan refill rate.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, the refill rate gradually slows as usage increases. The highest same-day bursts are limited more strictly.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 3bd0992f-bbec-4704-afa7-bfe8a496d9aa

📥 Commits

Reviewing files that changed from the base of the PR and between 38d8b96 and d3fca01.

📒 Files selected for processing (5)
  • hack/aws-sts-roles/home-role.cf
  • hack/aws-sts-roles/hub-role.cf
  • hack/aws-sts-roles/target-role.cf
  • pkg/steps/multi_stage/gen_test.go
  • pkg/steps/multi_stage/init_test.go
📝 Walkthrough

Walkthrough

Three new CloudFormation templates establish a three-tier STS role-chaining model for CI: a home role (OIDC trust in build cluster accounts), a hub role (trusted by home roles, assumes target roles), and a target role (ci-step-runner) granting broad CI permissions in target AWS accounts.

Changes

AWS STS Role Chaining CloudFormation Templates

Layer / File(s) Summary
Home Role (OIDC trust, build cluster accounts)
hack/aws-sts-roles/home-role.cf
Defines CIHomeRole with OIDC AssumeRoleWithWebIdentity trust gated on ci-op-* service accounts and sts.amazonaws.com audience. Inline policy permits sts:AssumeRole on arn:aws:iam::*:role/ci-step-runner. Outputs RoleArn for hub role deployment.
Hub Role (cross-account aggregator)
hack/aws-sts-roles/hub-role.cf
Defines CIHubRole trusted by a comma-delimited TrustedHomeRoleArns parameter. Inline policy allows assuming any ci-step-runner role across all target accounts. Outputs RoleArn for target role deployment and cluster profile secrets.
Target Role (ci-step-runner, CI permissions)
hack/aws-sts-roles/target-role.cf
Defines CITargetRole (named ci-step-runner) trusted by the hub role ARN. Attaches AWS-managed policies for CloudFormation, ECR Public, EFS, Secrets Manager, and Lambda, plus inline policies covering EC2/ELB/AutoScaling, S3, IAM/STS, Route53, Resource Groups, and read-only Service Quotas. Outputs RoleArn as target_role_arn.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Suggested reviewers

  • danilo-gemoli
  • droslean
🚥 Pre-merge checks | ✅ 16 | ❌ 1

❌ Failed checks (1 inconclusive)

Check name Status Explanation Resolution
Title check ❓ Inconclusive The title refers to adding CloudFormation templates but is vague about what they do and includes an issue reference that doesn't convey the actual change content. Consider a more descriptive title like 'Add CloudFormation templates for CI STS role chaining' that clarifies the purpose and scope of the changes.
✅ Passed checks (16 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Go Error Handling ✅ Passed The PR adds only CloudFormation templates (YAML format) for AWS IAM roles, not Go code. The Go error handling check is not applicable.
Test Coverage For New Features ✅ Passed The PR adds three CloudFormation templates (.cf files) for AWS infrastructure configuration. These are configuration-only changes that fall under the stated exception, not code implementations requ...
Stable And Deterministic Test Names ✅ Passed PR contains only CloudFormation templates (.cf files), not Ginkgo test files. The custom check for stable test names is not applicable.
Test Structure And Quality ✅ Passed PR contains only CloudFormation templates (IAM role definitions), not Ginkgo test code. The check is not applicable to this PR.
Microshift Test Compatibility ✅ Passed No Ginkgo e2e tests were added in this PR—only CloudFormation templates for AWS STS role infrastructure. Check is not applicable.
Single Node Openshift (Sno) Test Compatibility ✅ Passed This PR adds CloudFormation templates for AWS IAM roles, not Ginkgo e2e tests. The SNO compatibility check applies only to e2e tests and is not applicable here.
Topology-Aware Scheduling Compatibility ✅ Passed PR adds only AWS CloudFormation IAM role templates with no Kubernetes deployment manifests, operator code, or scheduling constraints. Check does not apply to AWS infrastructure-as-code.
Ote Binary Stdout Contract ✅ Passed PR adds only CloudFormation templates (.cf files) to hack/aws-sts-roles/; no Go code, test code, or stdout-writing logic. The custom check for OTE binary stdout contract is not applicable to infras...
Ipv6 And Disconnected Network Test Compatibility ✅ Passed No Ginkgo e2e tests added in this PR. Custom check applies only to e2e tests; PR adds CloudFormation templates for IAM role provisioning.
No-Weak-Crypto ✅ Passed Three new CloudFormation templates added contain no weak cryptography algorithms (MD5, SHA1, DES, RC4, 3DES, Blowfish, ECB), custom crypto implementations, or insecure token comparisons. Files are...
Container-Privileges ✅ Passed PR contains only AWS CloudFormation templates defining IAM roles, not container/Kubernetes manifests. The container-privileges check is not applicable.
No-Sensitive-Data-In-Logs ✅ Passed The CloudFormation templates contain no logging configurations that expose sensitive data (passwords, tokens, API keys, PII, session IDs, hostnames, or customer data).

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@openshift-ci openshift-ci Bot requested review from jmguzik and psalajova June 19, 2026 15:11
@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 19, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (2)
hack/aws-sts-roles/home-role.cf (1)

34-50: 💤 Low value

Consider using native YAML syntax for consistency with other templates.

The hub-role.cf and target-role.cf templates use native YAML for AssumeRolePolicyDocument, but this template embeds a JSON string via !Sub |. Functionally equivalent, but YAML syntax would be more maintainable and consistent across the PR.

♻️ Suggested YAML syntax
       RoleName: ci-step-runner
-      AssumeRolePolicyDocument: !Sub |
-        {
-          "Version": "2012-10-17",
-          "Statement": [{
-            "Effect": "Allow",
-            "Principal": { "Federated": "${OIDCProviderArn}" },
-            "Action": "sts:AssumeRoleWithWebIdentity",
-            "Condition": {
-              "StringLike": {
-                "${OIDCProviderURL}:sub": "system:serviceaccount:ci-op-*:*"
-              },
-              "StringEquals": {
-                "${OIDCProviderURL}:aud": "sts.amazonaws.com"
-              }
-            }
-          }]
-        }
+      AssumeRolePolicyDocument:
+        Version: "2012-10-17"
+        Statement:
+          - Effect: Allow
+            Principal:
+              Federated: !Ref OIDCProviderArn
+            Action: sts:AssumeRoleWithWebIdentity
+            Condition:
+              StringLike:
+                !Sub "${OIDCProviderURL}:sub": "system:serviceaccount:ci-op-*:*"
+              StringEquals:
+                !Sub "${OIDCProviderURL}:aud": "sts.amazonaws.com"
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@hack/aws-sts-roles/home-role.cf` around lines 34 - 50, The
AssumeRolePolicyDocument in home-role.cf uses embedded JSON string syntax with
!Sub |, while other templates (hub-role.cf and target-role.cf) use native YAML
syntax. Convert the AssumeRolePolicyDocument from the embedded JSON string
format to native YAML structure by removing the !Sub | and the quoted JSON, and
instead represent the Version, Statement, Effect, Principal, Action, and
Condition as direct YAML nested objects and mappings. This will maintain the
same functionality while improving consistency and maintainability across all
three role templates.
hack/aws-sts-roles/target-role.cf (1)

76-83: ⚖️ Poor tradeoff

Security note: iam:* and sts:* are very broad permissions.

The iam:* action allows modifying any IAM entity in the target account, including the ci-step-runner role itself. Combined with sts:*, this permits lateral movement to any role in the account.

The description states this mirrors origin-ci-robot-provision, so this is likely intentional parity. If you want to tighten this in the future, consider:

  • Scoping iam:* to specific resource patterns (e.g., roles/policies for cluster provisioning)
  • Restricting sts:AssumeRole to known installer roles
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@hack/aws-sts-roles/target-role.cf` around lines 76 - 83, The Sid IAMFull and
STSAssumeRole statements in the policy have overly broad permissions that could
enable lateral movement. To improve security, replace the wildcard actions
"iam:*" and "sts:*" with more specific permissions. For the IAMFull statement,
restrict the Action to only necessary IAM operations for cluster provisioning
(such as iam:CreateRole, iam:PutRolePolicy, iam:GetRole, etc.) and scope the
Resource to specific resource patterns like arn:aws:iam::account:role/cluster-*
or similar relevant patterns. For the STSAssumeRole statement, change the Action
from "sts:*" to specifically "sts:AssumeRole" and restrict the Resource to only
known installer role ARNs instead of using "*". Document why these specific
permissions are necessary for your provisioning workflow.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@hack/aws-sts-roles/target-role.cf`:
- Around line 7-9: The documentation comment describing the assume chain in the
target-role.cf file is incomplete and omits the home role hop. Update the assume
chain comment that currently describes the flow from per-test SA token through
ci-step-runner in the hub account to also include the intermediate home role
step from the build cluster account. The corrected chain should include all
three hops: per-test SA token (OIDC) to ci-step-runner (home-role.cf, build
cluster account) to ci-step-runner (hub-role.cf, hub account) to ci-step-runner
(this, target account). Ensure the documentation in the comments reflects the
complete chain as defined in both home-role.cf and hub-role.cf files.

---

Nitpick comments:
In `@hack/aws-sts-roles/home-role.cf`:
- Around line 34-50: The AssumeRolePolicyDocument in home-role.cf uses embedded
JSON string syntax with !Sub |, while other templates (hub-role.cf and
target-role.cf) use native YAML syntax. Convert the AssumeRolePolicyDocument
from the embedded JSON string format to native YAML structure by removing the
!Sub | and the quoted JSON, and instead represent the Version, Statement,
Effect, Principal, Action, and Condition as direct YAML nested objects and
mappings. This will maintain the same functionality while improving consistency
and maintainability across all three role templates.

In `@hack/aws-sts-roles/target-role.cf`:
- Around line 76-83: The Sid IAMFull and STSAssumeRole statements in the policy
have overly broad permissions that could enable lateral movement. To improve
security, replace the wildcard actions "iam:*" and "sts:*" with more specific
permissions. For the IAMFull statement, restrict the Action to only necessary
IAM operations for cluster provisioning (such as iam:CreateRole,
iam:PutRolePolicy, iam:GetRole, etc.) and scope the Resource to specific
resource patterns like arn:aws:iam::account:role/cluster-* or similar relevant
patterns. For the STSAssumeRole statement, change the Action from "sts:*" to
specifically "sts:AssumeRole" and restrict the Resource to only known installer
role ARNs instead of using "*". Document why these specific permissions are
necessary for your provisioning workflow.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 846fd2a8-149e-4239-965e-568f25e50a07

📥 Commits

Reviewing files that changed from the base of the PR and between 2bfca56 and 38d8b96.

📒 Files selected for processing (3)
  • hack/aws-sts-roles/home-role.cf
  • hack/aws-sts-roles/hub-role.cf
  • hack/aws-sts-roles/target-role.cf
🔗 Linked repositories identified

CodeRabbit considers these linked repositories for cross-repo context during reviews:

  • openshift/release (manual)
  • openshift/ci-docs (manual)
  • openshift/release-controller (manual)
  • openshift/ci-chat-bot (manual)

Comment thread hack/aws-sts-roles/target-role.cf Outdated
Comment on lines +7 to +9
Trusts the ci-step-runner hub role. The assume chain is:
per-test SA token (OIDC) -> ci-step-runner (hub-role.cf, hub account)
-> ci-step-runner (this, target account)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Documentation error: missing home role hop in the assume chain.

The chain description omits the home role step. Per home-role.cf and hub-role.cf, the full chain is:

per-test SA token (OIDC) -> ci-step-runner (home-role.cf, build cluster acct)
  -> ci-step-runner (hub-role.cf, hub account)
  -> ci-step-runner (this, target account)
📝 Suggested fix
   Trusts the ci-step-runner hub role. The assume chain is:
-    per-test SA token (OIDC) -> ci-step-runner (hub-role.cf, hub account)
+    per-test SA token (OIDC) -> ci-step-runner (home-role.cf, build cluster acct)
+      -> ci-step-runner (hub-role.cf, hub account)
       -> ci-step-runner (this, target account)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@hack/aws-sts-roles/target-role.cf` around lines 7 - 9, The documentation
comment describing the assume chain in the target-role.cf file is incomplete and
omits the home role hop. Update the assume chain comment that currently
describes the flow from per-test SA token through ci-step-runner in the hub
account to also include the intermediate home role step from the build cluster
account. The corrected chain should include all three hops: per-test SA token
(OIDC) to ci-step-runner (home-role.cf, build cluster account) to ci-step-runner
(hub-role.cf, hub account) to ci-step-runner (this, target account). Ensure the
documentation in the comments reflects the complete chain as defined in both
home-role.cf and hub-role.cf files.

@hector-vido

Copy link
Copy Markdown
Contributor

/lgtm

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Jun 19, 2026
@openshift-ci openshift-ci Bot removed the lgtm Indicates that a PR is ready to be merged. label Jun 19, 2026
@hector-vido

Copy link
Copy Markdown
Contributor

/lgtm

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Jun 19, 2026
@openshift-ci

openshift-ci Bot commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bear-redhat, hector-vido

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:
  • OWNERS [bear-redhat,hector-vido]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@bear-redhat

Copy link
Copy Markdown
Contributor Author

/override ci/prow/integration
ci/prow/integration

@bear-redhat

Copy link
Copy Markdown
Contributor Author

/override ci/prow/integration

@openshift-ci

openshift-ci Bot commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

@bear-redhat: /override requires failed status contexts, check run or a prowjob name to operate on.
The following unknown contexts/checkruns were given:

  • [ci/prow/integration](https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_ci-tools/5263/pull-ci-openshift-ci-tools-main-integration/2067999468573167616)

Only the following failed contexts/checkruns were expected:

  • CodeRabbit
  • ci/prow/breaking-changes
  • ci/prow/checkconfig
  • ci/prow/codegen
  • ci/prow/e2e
  • ci/prow/format
  • ci/prow/frontend-checks
  • ci/prow/images
  • ci/prow/integration
  • ci/prow/lint
  • ci/prow/secret-bootstrapper-validation
  • ci/prow/secret-generator-validation
  • ci/prow/security
  • ci/prow/unit
  • ci/prow/validate-prow
  • ci/prow/validate-vendor
  • pull-ci-openshift-ci-tools-main-breaking-changes
  • pull-ci-openshift-ci-tools-main-checkconfig
  • pull-ci-openshift-ci-tools-main-codegen
  • pull-ci-openshift-ci-tools-main-e2e
  • pull-ci-openshift-ci-tools-main-format
  • pull-ci-openshift-ci-tools-main-frontend-checks
  • pull-ci-openshift-ci-tools-main-images
  • pull-ci-openshift-ci-tools-main-integration
  • pull-ci-openshift-ci-tools-main-lint
  • pull-ci-openshift-ci-tools-main-secret-bootstrapper-validation
  • pull-ci-openshift-ci-tools-main-secret-generator-validation
  • pull-ci-openshift-ci-tools-main-security
  • pull-ci-openshift-ci-tools-main-unit
  • pull-ci-openshift-ci-tools-main-validate-prow
  • pull-ci-openshift-ci-tools-main-validate-vendor
  • tide

If you are trying to override a checkrun that has a space in it, you must put a double quote on the context.

Details

In response to this:

/override ci/prow/integration
ci/prow/integration

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Scheduling tests matching the pipeline_run_if_changed or not excluded by pipeline_skip_if_only_changed parameters:
/test e2e

@openshift-ci

openshift-ci Bot commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

@bear-redhat: Overrode contexts on behalf of bear-redhat: ci/prow/integration

Details

In response to this:

/override ci/prow/integration

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

/retest-required

Remaining retests: 0 against base HEAD 2bfca56 and 2 for PR HEAD d3fca01 in total

@openshift-ci

openshift-ci Bot commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

@bear-redhat: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants