Skip to content

fix: pin CatalogSource pods to restricted-v2 SCC on OpenShift HCP#1332

Open
TGPSKI wants to merge 2 commits into
openshift:mainfrom
TGPSKI:fix/catalog-pod-required-scc-annotation
Open

fix: pin CatalogSource pods to restricted-v2 SCC on OpenShift HCP#1332
TGPSKI wants to merge 2 commits into
openshift:mainfrom
TGPSKI:fix/catalog-pod-required-scc-annotation

Conversation

@TGPSKI

@TGPSKI TGPSKI commented Jun 25, 2026

Copy link
Copy Markdown

Summary

On ROSA HCP clusters, CatalogSource pods can be assigned a customer-installed SCC instead of restricted-v2, causing the pod to run with an unexpected UID and crash with permission denied errors on pre-baked cache directories.

This PR adds the openshift.io/required-scc: restricted-v2 annotation to CatalogSource pods when securityContextConfig is Restricted, pinning SCC selection and preventing custom SCCs from preempting admission.

Problem

OpenShift's SCC admission selects the "most restrictive" SCC from the candidate set of the creating user. On HCP, the entity that creates catalog pods on the data plane has broader SCC access than the local catalog-operator ServiceAccount on classic OpenShift — evidenced by the security.openshift.io/validated-scc-subject-type: user annotation on affected pods.

A custom SCC with runAsUser.type: MustRunAs (single fixed UID) scores as more restrictive than restricted-v2's MustRunAsRange. When such an SCC wins, it mutates the pod to a UID that differs from the image's expected UID (1001 per USER directive), resulting in:

open /tmp/cache/pogreb.v1/db/lock: permission denied

This does not reproduce on classic OpenShift — the same custom SCC installed on classic causes restricted-v2 to still win, consistent with the local catalog-operator SA not having access to customer-installed SCCs.

Fix

Uses the openshift.io/required-scc pod annotation (GA since OCP 4.14, openshift/enhancements#1391) to pin SCC selection to restricted-v2 within the existing Restricted security context branch.

Key properties:

  • No-op on non-OpenShift: The annotation is silently ignored where no SCC admission controller exists
  • Respects user overrides: !alreadySet check preserves any annotation passed through CatalogSource annotations
  • Minimal blast radius: Only fires in the Restricted branch (PSA-labeled namespaces or explicit grpcPodConfig.securityContextConfig: restricted)

Test Plan

  • Existing TestPodExtractContent tests updated to expect the new annotation in Restricted mode
  • New TestPodRequiredSCCAnnotation test validates:
    • Restricted config adds the annotation
    • Legacy config does not add the annotation
    • User-provided annotation value is preserved (not overwritten)
  • Full pkg/controller/registry/reconciler/ test suite passes

References

On HCP clusters, SCC admission evaluates against the creating user
(the hosted control plane) rather than the pod's ServiceAccount. This
broader access set allows customer-installed SCCs with MustRunAs to
score as more restrictive than restricted-v2's MustRunAsRange, winning
admission and mutating the pod to an unexpected UID.

The UID mismatch causes permission denied errors on pre-baked cache
directories (e.g. /tmp/cache/pogreb.v1/db/lock), crashing catalog pods.

Add the openshift.io/required-scc annotation (GA since OCP 4.14,
openshift/enhancements#1391) to pin SCC selection to restricted-v2 when
securityContextConfig is Restricted. The annotation is a no-op on
non-OpenShift clusters and respects user-provided overrides.

Fixes: openshift#1331
@openshift-ci openshift-ci Bot requested review from bentito and grokspawn June 25, 2026 22:50
@openshift-ci

openshift-ci Bot commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: TGPSKI
Once this PR has been reviewed and has the lgtm label, please assign perdasilva for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@coderabbitai

coderabbitai Bot commented Jun 25, 2026

Copy link
Copy Markdown

Important

Review skipped

Review was skipped due to path filters

⛔ Files ignored due to path filters (1)
  • vendor/github.com/operator-framework/operator-lifecycle-manager/pkg/controller/registry/reconciler/reconciler.go is excluded by !**/vendor/**, !vendor/**

CodeRabbit blocks several paths by default. You can override this behavior by explicitly including those paths in the path filters. For example, including **/dist/** will override the default block on the dist directory, by removing the pattern from both the lists.

⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 6806f5c9-d64c-45bf-8ede-8462323f2018

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Walkthrough

The Pod reconciler now adds openshift.io/required-scc: restricted-v2 for restricted security context pods when the annotation is missing. Tests were updated to expect that annotation and to verify restricted, legacy, and user-provided annotation cases.

Changes

Restricted SCC annotation pinning

Layer / File(s) Summary
Pod annotation logic
staging/operator-lifecycle-manager/pkg/controller/registry/reconciler/reconciler.go
The restricted security context branch now sets openshift.io/required-scc to restricted-v2 when it is not already present.
Pod annotation tests
staging/operator-lifecycle-manager/pkg/controller/registry/reconciler/reconciler_test.go
TestPodExtractContent expected annotations include openshift.io/required-scc: "restricted-v2" for restricted cases, and a new table-driven test checks restricted, legacy, and preexisting annotation behavior.

Sequence Diagram(s)

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related issues

🚥 Pre-merge checks | ✅ 14 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (14 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main change: pinning CatalogSource pods to restricted-v2 SCC on OpenShift HCP.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed The added/updated test titles are static strings; no dynamic pod, node, namespace, date, UUID, or other generated values appear in titles.
Test Structure And Quality ✅ Passed Pure unit tests only; each subtest targets one behavior, no cluster resources/Eventually/cleanup issues, and assertion style matches package conventions.
Microshift Test Compatibility ✅ Passed Only Go unit tests were added/updated; no new Ginkgo e2e tests or MicroShift-unsupported APIs/resources are referenced. The new SCC annotation uses a MicroShift-supported API.
Single Node Openshift (Sno) Test Compatibility ✅ Passed No new Ginkgo e2e tests were added; the new test is plain testing.T unit code and has no multi-node/SNO assumptions.
Topology-Aware Scheduling Compatibility ✅ Passed Change only adds an SCC annotation; no new replicas, affinity, nodeSelector, tolerations, spread, or PDB logic was introduced.
Ote Binary Stdout Contract ✅ Passed No stdout-printing APIs or suite-level setup were added; the changed Pod logic only mutates annotations, and tests stay within normal Test funcs.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed The added coverage is plain Go unit tests (testing.T), not Ginkgo e2e; it adds no IPv4 assumptions or external network dependencies.
No-Weak-Crypto ✅ Passed Touched code only adds an SCC annotation and tests; no MD5/SHA1/DES/RC4/3DES, custom crypto, or secret/token comparisons appear.
Container-Privileges ✅ Passed The PR only adds an SCC annotation; no privileged/host namespace/SYS_ADMIN/allowPrivilegeEscalation=true settings were introduced, and existing securityContext hardens containers.
No-Sensitive-Data-In-Logs ✅ Passed The PR only adds an SCC annotation and tests; no new logging calls or sensitive-data output appears in the changed code.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@openshift-ci openshift-ci Bot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Jun 25, 2026
@openshift-ci

openshift-ci Bot commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Hi @TGPSKI. Thanks for your PR.

I'm waiting for a openshift member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work.

Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@TGPSKI

TGPSKI commented Jun 25, 2026

Copy link
Copy Markdown
Author

/test

@openshift-ci

openshift-ci Bot commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

@TGPSKI: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

Details

In response to this:

/test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@perdasilva

Copy link
Copy Markdown
Contributor

/ok-to-test

@openshift-ci openshift-ci Bot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jun 26, 2026
Run `go mod tidy && go mod vendor` to update the vendored copy
of reconciler.go to match the staging change from the previous
commit.
@TGPSKI

TGPSKI commented Jun 26, 2026

Copy link
Copy Markdown
Author

/test verify-deps

@openshift-ci

openshift-ci Bot commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

@TGPSKI: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-aws-upgrade-ovn-single-node 134b429 link false /test e2e-aws-upgrade-ovn-single-node

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ok-to-test Indicates a non-member PR verified by an org member that is safe to test.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants