docs(networking): Cilium Gateway API — architecture, security, migration#509
Conversation
✅ Deploy Preview for cozystack ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds documentation and platform configuration for an opt-in Cilium-backed Gateway API: Helm-rendered per-tenant Gateway resources, controller materialization of Gateways/Issuers/Certificates, HTTPRoute/TLSRoute handling (redirects, ACME, passthrough), LoadBalancer IP pool usage, admission policies, migration notes, and troubleshooting. Changes
Sequence Diagram(s)sequenceDiagram
participant Admin as Platform Helm/Values
participant K8s as Kubernetes API
participant Controller as cozystack-controller
participant TenantNS as Tenant Namespace
participant CertManager
participant Envoy as Envoy DaemonSet
participant LBPool as CiliumLoadBalancerIPPool
Admin->>K8s: enable platform Gateway (gateway.enabled, attachedNamespaces)
Admin->>K8s: install GatewayClass, ValidatingAdmissionPolicies
TenantNS->>K8s: tenant with spec.gateway: true
K8s->>Controller: render TenantGateway CRs
Controller->>K8s: materialize Gateway, Issuer, Certificate, HTTPRoute/TLSRoute
K8s->>CertManager: ACME certificate request (HTTP-01 or DNS-01)
CertManager-->>K8s: certificate issued
Controller->>LBPool: allocate tenant LoadBalancer IP
Envoy->>K8s: program listeners (HTTPS, redirects, optional passthrough)
Client->>Envoy: TLS or HTTP request
Envoy->>TenantNS: route to backend per HTTPRoute/TLSRoute
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Aleksei Sviridkin (lexfrei)
requested review from
Andrei Kvapil (kvaps) and
Timofei Larkin (lllamnyp)
as code owners
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces comprehensive documentation for the Gateway API support in Cozystack, detailing its architecture, security model, and migration path from ingress-nginx. The review feedback identifies opportunities to improve technical accuracy and consistency, specifically by clarifying that namespace whitelisting applies to both HTTPRoute and TLSRoute resources and resolving a naming inconsistency for the Kubernetes API route.
| - The exposed-service templates (dashboard, keycloak) stop rendering their `Ingress` and start rendering their `HTTPRoute`. | ||
| - TLS-passthrough services (cozystack-api, vm-exportproxy, cdi-uploadproxy) stop rendering their `Ingress` and start rendering a `TLSRoute` attached to a dedicated Passthrough listener. | ||
|
|
||
| The `attachedNamespaces` list restricts which namespaces may attach `HTTPRoute`s to tenant Gateways through the listener `allowedRoutes` whitelist (see [Security](#security)). It is also guarded by a runtime `ValidatingAdmissionPolicy` that rejects any `tenant-*` entry. |
There was a problem hiding this comment.
The documentation mentions that attachedNamespaces restricts HTTPRoute attachments. However, the architecture also utilizes TLSRoute for services like the Kubernetes API and KubeVirt proxies (as shown in the routing tables). It would be more accurate to state that this list applies to both HTTPRoute and TLSRoute (or Gateway API routes in general).
| The `attachedNamespaces` list restricts which namespaces may attach `HTTPRoute`s to tenant Gateways through the listener `allowedRoutes` whitelist (see [Security](#security)). It is also guarded by a runtime `ValidatingAdmissionPolicy` that rejects any `tenant-*` entry. | |
| The attachedNamespaces list restricts which namespaces may attach HTTPRoutes and TLSRoutes to tenant Gateways through the listener allowedRoutes whitelist (see [Security](#security)). It is also guarded by a runtime ValidatingAdmissionPolicy that rejects any tenant-* entry. |
There was a problem hiding this comment.
Fixed in 1ea0093: the paragraph now says HTTPRoute or TLSRoute. The allowedRoutes whitelist on the listener is route-kind-agnostic, so in practice it restricts every route type that attaches to the Gateway — including the TLSRoutes used for the Kubernetes API, vm-exportproxy, and cdi-uploadproxy.
|
|
||
| | Service | Namespace | `TLSRoute` name | Backend | Listener | | ||
| |---|---|---|---|---| | ||
| | Kubernetes API | `default` | `kubernetes-api` | `kubernetes:443` | `tls-api` | |
There was a problem hiding this comment.
There is an inconsistency in naming the Kubernetes API route. It is referred to as cozystack-api in the Mermaid diagram (line 27) and the migration section (line 265), but as kubernetes-api in this table. Using a consistent name throughout the document would improve clarity.
| | Kubernetes API | `default` | `kubernetes-api` | `kubernetes:443` | `tls-api` | | |
| | Kubernetes API | default | cozystack-api | kubernetes:443 | tls-api | |
There was a problem hiding this comment.
Fixed in dcb805c by aligning the Mermaid diagram to the real resource name. The TLSRoute is literally named kubernetes-api (see packages/system/cozystack-api/templates/api-tlsroute.yaml), so the table at line 144 is the source of truth. cozystack-api in the diagram referred to the cozystack package that ships this route, which was misleading. The diagram now says kubernetes-api and the migration prose clarifies the relationship (cozystack-api (Kubernetes API)).
|
|
||
| Every listener on a tenant Gateway pins `allowedRoutes.namespaces.from: Selector` to a `matchExpressions` whitelist against the built-in `kubernetes.io/metadata.name` label. That label is written by kube-apiserver on every namespace and cannot be spoofed. | ||
|
|
||
| The whitelist is the publishing tenant's namespace (always, implicit) plus `publishing.gateway.attachedNamespaces`. A namespace outside the list literally cannot attach any `HTTPRoute` to the Gateway. |
There was a problem hiding this comment.
Similar to the comment on line 101, this section should clarify that the whitelist applies to both HTTPRoute and TLSRoute, as both are used in the described architecture.
| The whitelist is the publishing tenant's namespace (always, implicit) plus `publishing.gateway.attachedNamespaces`. A namespace outside the list literally cannot attach any `HTTPRoute` to the Gateway. | |
| The whitelist is the publishing tenant's namespace (always, implicit) plus publishing.gateway.attachedNamespaces. A namespace outside the list literally cannot attach any HTTPRoute or TLSRoute to the Gateway. |
There was a problem hiding this comment.
Fixed in e15d865: the Layer 1 description now explicitly says HTTPRoute or TLSRoute. Same root cause as the line 101 comment — the listener-level whitelist applies to every route kind attaching to that listener.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@content/en/docs/next/networking/gateway-api.md`:
- Line 56: The in-page anchor "#tls-passthrough" in the sentence "Plus one extra
listener per TLS-passthrough service (see [TLS passthrough](`#tls-passthrough`)
below)" doesn't match the actual heading ID; locate the "TLS passthrough"
section heading in this document and either rename that heading (or add an
explicit HTML anchor/id) to produce the ID tls-passthrough, or update the link
fragment to the existing heading ID (for example whatever the generated slug
is); ensure the link target and the heading ID for the TLS passthrough section
are identical so the anchor works.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 1a003edc-54d7-4122-a90c-e40d9592e1c7
📒 Files selected for processing (1)
content/en/docs/next/networking/gateway-api.md
myasnikovdaniil
left a comment
myasnikovdaniil
left a comment
There was a problem hiding this comment.
No explanation of publishing.exposure flag in platform package, needs to be added
|
myasnikovdaniil Added a It covers what the flag does (ingress-nginx Service shape: ClusterIP+externalIPs vs LoadBalancer), why a Gateway API rollout is the natural moment to flip it (so ingress-nginx and the per-tenant Gateway draw from the same Cilium-managed pool), the KEP-5707 deprecation timeline that forces the move before Kubernetes v1.40, and the loadBalancer-mode caveats lifted from the platform values.yaml: non-empty |
|
Platform parameters must also land into platform package reference |
Aleksei Sviridkin (lexfrei)
force-pushed
the
docs/gateway-api-cilium
branch
from
10b1e7c to
6888f84
Compare
|
myasnikovdaniil Done — added the platform parameters to the Platform Package Reference in 6888f84:
Schema was verified against The branch was rebased onto |
There was a problem hiding this comment.
🧹 Nitpick comments (2)
content/en/docs/next/operations/configuration/platform-package.md (2)
108-108: Minor: Consider consistent spelling variant.Line 108 uses "Materialising" (British English). While both variants are correct, using consistent spelling throughout the documentation improves polish. Consider "Materializing" if the project prefers American English, or keep the current form if British English is the standard.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@content/en/docs/next/operations/configuration/platform-package.md` at line 108, The documentation uses the British English spelling "Materialising" in the description for `gateway.enabled`; update that word to the project's chosen variant (e.g., change "Materialising" to "Materializing") for consistency with the rest of the docs—edit the text in the `gateway.enabled` description to the preferred spelling.
66-66: Consider breaking up the dense table description for better scannability.The
publishing.exposuredescription packs mode definitions, deprecation timeline, validation behavior, and a caveat link into a single paragraph. Users scanning the table may miss the critical deprecation warning or the fail-fast validation note.♻️ Suggested restructure for improved readability
-| `publishing.exposure` | `"externalIPs"` | Mode for the ingress-nginx Service. `externalIPs` creates a `ClusterIP` Service with `Service.spec.externalIPs` populated from `publishing.externalIPs`. `loadBalancer` creates a `type: LoadBalancer` Service backed by a `CiliumLoadBalancerIPPool` populated with the same addresses. `Service.spec.externalIPs` is deprecated upstream in Kubernetes v1.36 ([KEP-5707][kep-5707]) — switch to `loadBalancer` before upgrading past v1.40. The chart fails fast if `loadBalancer` is set with an empty `publishing.externalIPs`. See [Gateway API → ingress-nginx Service mode]({{% ref "/docs/next/networking/gateway-api#publishingexposure--ingress-nginx-service-mode" %}}) for the full caveat list. | +| `publishing.exposure` | `"externalIPs"` | Mode for the ingress-nginx Service.<br/><br/>`externalIPs`: Creates a `ClusterIP` Service with `Service.spec.externalIPs` populated from `publishing.externalIPs`.<br/><br/>`loadBalancer`: Creates a `type: LoadBalancer` Service backed by a `CiliumLoadBalancerIPPool` using the same addresses.<br/><br/>**Deprecation notice:** `Service.spec.externalIPs` is deprecated in Kubernetes v1.36 ([KEP-5707][kep-5707]). Switch to `loadBalancer` before upgrading to v1.40.<br/><br/>**Validation:** The chart returns an error if `loadBalancer` is set with an empty `publishing.externalIPs`.<br/><br/>See [Gateway API → ingress-nginx Service mode]({{% ref "/docs/next/networking/gateway-api#publishingexposure--ingress-nginx-service-mode" %}}) for additional caveats. |This uses
<br/>tags (permitted byunsafe: trueGoldmark config) to create visual breaks within the table cell, making each concept easier to locate.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@content/en/docs/next/operations/configuration/platform-package.md` at line 66, The table cell for publishing.exposure is too dense; split its single paragraph into separate sentences or lines (using permitted <br/> tags) that each cover: the two modes and what they do (externalIPs vs loadBalancer and that loadBalancer uses CiliumLoadBalancerIPPool), the deprecation of Service.spec.externalIPs (KEP-5707) with the upgrade advice to switch before v1.40, the validation/fail-fast behavior when loadBalancer is set but publishing.externalIPs is empty, and the link to Gateway API → ingress-nginx Service mode, so readers can scan and find the deprecation and validation notes quickly.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@content/en/docs/next/operations/configuration/platform-package.md`:
- Line 108: The documentation uses the British English spelling "Materialising"
in the description for `gateway.enabled`; update that word to the project's
chosen variant (e.g., change "Materialising" to "Materializing") for consistency
with the rest of the docs—edit the text in the `gateway.enabled` description to
the preferred spelling.
- Line 66: The table cell for publishing.exposure is too dense; split its single
paragraph into separate sentences or lines (using permitted <br/> tags) that
each cover: the two modes and what they do (externalIPs vs loadBalancer and that
loadBalancer uses CiliumLoadBalancerIPPool), the deprecation of
Service.spec.externalIPs (KEP-5707) with the upgrade advice to switch before
v1.40, the validation/fail-fast behavior when loadBalancer is set but
publishing.externalIPs is empty, and the link to Gateway API → ingress-nginx
Service mode, so readers can scan and find the deprecation and validation notes
quickly.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 144efd7e-3b34-413f-890f-38c874ba5c84
📒 Files selected for processing (2)
content/en/docs/next/networking/gateway-api.mdcontent/en/docs/next/operations/configuration/platform-package.md
🚧 Files skipped from review as they are similar to previous changes (1)
- content/en/docs/next/networking/gateway-api.md
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
content/en/docs/next/operations/configuration/platform-package.md (1)
59-67:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winAddress markdownlint MD052 for
kep-5707by switching to an inline link in the table cell.markdownlint-cli2 reports
Missing link or image reference definition: "kep-5707"on thepublishing.exposurerow, even though the definition appears later in the file. This is typically a parsing limitation around table cells / reference-style links. Converting that specific mention to an inline link avoids the ambiguity and should clear the warning.🔧 Proposed fix
-| `publishing.exposure` | `"externalIPs"` | Mode for the ingress-nginx Service. `externalIPs` creates a `ClusterIP` Service with `Service.spec.externalIPs` populated from `publishing.externalIPs`. `loadBalancer` creates a `type: LoadBalancer` Service backed by a `CiliumLoadBalancerIPPool` populated with the same addresses. `Service.spec.externalIPs` is deprecated upstream in Kubernetes v1.36 ([KEP-5707][kep-5707]) — switch to `loadBalancer` before upgrading past v1.40. The chart fails fast if `loadBalancer` is set with an empty `publishing.externalIPs`. See [Gateway API → ingress-nginx Service mode]({{% ref "/docs/next/networking/gateway-api#publishingexposure--ingress-nginx-service-mode" %}}) for the full caveat list. | +| `publishing.exposure` | `"externalIPs"` | Mode for the ingress-nginx Service. `externalIPs` creates a `ClusterIP` Service with `Service.spec.externalIPs` populated from `publishing.externalIPs`. `loadBalancer` creates a `type: LoadBalancer` Service backed by a `CiliumLoadBalancerIPPool` populated with the same addresses. `Service.spec.externalIPs` is deprecated upstream in Kubernetes v1.36 ([KEP-5707](https://github.com/kubernetes/enhancements/issues/5707)) — switch to `loadBalancer` before upgrading past v1.40. The chart fails fast if `loadBalancer` is set with an empty `publishing.externalIPs`. See [Gateway API → ingress-nginx Service mode]({{% ref "/docs/next/networking/gateway-api#publishingexposure--ingress-nginx-service-mode" %}}) for the full caveat list. |🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@content/en/docs/next/operations/configuration/platform-package.md` around lines 59 - 67, The markdownlint MD052 warning is caused by the reference-style link `[kep-5707]` inside the `publishing.exposure` table cell; update that table row (the `publishing.exposure` entry in the diff) to use an inline link with the full URL for KEP-5707 instead of the reference-style link so the linter can resolve it (leave the later definition in place or remove it if you prefer, but the table must use the inline URL).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@content/en/docs/next/networking/gateway-api.md`:
- Around line 100-127: Update the platform Package example for
cozystack.cozystack-platform so gateway.attachedNamespaces includes the default
namespace; modify the values under
spec.components.platform.values.gateway.attachedNamespaces to add "default"
alongside the listed cozy-* namespaces (ensure you edit the snippet showing
gateway.enabled: true for cozystack.cozystack-platform and update
gateway.attachedNamespaces accordingly).
---
Outside diff comments:
In `@content/en/docs/next/operations/configuration/platform-package.md`:
- Around line 59-67: The markdownlint MD052 warning is caused by the
reference-style link `[kep-5707]` inside the `publishing.exposure` table cell;
update that table row (the `publishing.exposure` entry in the diff) to use an
inline link with the full URL for KEP-5707 instead of the reference-style link
so the linter can resolve it (leave the later definition in place or remove it
if you prefer, but the table must use the inline URL).
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 25ca4fc4-0ed7-4315-bd1a-4ae23063592b
📒 Files selected for processing (2)
content/en/docs/next/networking/gateway-api.mdcontent/en/docs/next/operations/configuration/platform-package.md
Andrei Kvapil (kvaps)
left a comment
Andrei Kvapil (kvaps)
left a comment
There was a problem hiding this comment.
Reviewed alongside cozystack/cozystack#2470 — see parent review for the full context. Three asks specific to this docs PR; they all stem from the same restructure of the security framing that the PR review requests in the implementation repo.
1. Restructure the ## Security section in content/en/docs/next/networking/gateway-api.md
The current "guarded at seven independent layers" framing overstates the role of layers that aren't protecting against tenant users — under Cozystack's API surface model, tenants only write apps.cozystack.io/* resources through cozystack-api and don't hold RBAC on gateway.networking.k8s.io/*, core Namespaces, or cozystack.io/Package. So most of the seven layers are defense-in-depth (against chart bugs, controller bugs, supply-chain compromise, admin mistakes), not first-line tenant defenses.
Asks:
- Replace the section intro with a split-by-purpose framing: tenant-user-input gates (Layer 4 + the cozystack-api admission-chain fix), defense-in-depth (Layers 1, 2, 5, 6, 7), and admin-against-themselves (Layer 3).
- Update the mermaid diagram (lines 266–292) so the attacker arrow lands on Layer 4 / cozystack-api admission as the user-input boundary, with the other layers branching off as covering chart / controller / supply-chain failure modes rather than all converging from the same
ATKnode. - In the Layer 7 description (lines 328–334) replace
a tenant user with HTTPRoute RBAC could otherwise exploit— that RBAC isn't granted in Cozystack by design. Reframe Layer 7 as defense-in-depth against an app-chart bug or supply-chain compromise emitting HTTPRoutes outside the apex.
2. Document tenant-user API surface explicitly somewhere in the page
A short paragraph (probably in ## Overview or just before ## Security) stating that tenants interact with the platform exclusively through apps.cozystack.io/* resources (Tenant, Bucket, Kubernetes, etc.) and that the security model is built around that constraint. This makes the rest of the Security section read correctly — without that anchor, readers can mistakenly assume tenants write Gateways or HTTPRoutes directly.
3. Rewrite the migration / pinning sections after the implementation PR drops tenant.spec.gatewayIP
The parent PR review asks for tenant.spec.gatewayIP and the CiliumLoadBalancerIPPool branch to be removed (they don't fit Cozystack's MetalLB-default LB stack and the node-public-IP semantics of publishing.externalIPs). When that lands:
### 3. Pinning a tenant Gateway to a specific external IP(lines 177–197) needs to be removed or replaced with a shorter note that per-tenant IP pinning, when needed, is a cluster-admin-sidemetallb.universe.tf/loadBalancerIPsannotation, not a tenant API field.- The
### Gateway Service <pending> LoadBalancer IPtroubleshooting entry (line 509) should be updated to point at MetalLB pool configuration as the resolution, since that's where IPs come from in default Cozystack.
Hold this third point until the implementation PR settles — the docs change is mechanical once gatewayIP is gone.
Happy to re-review once these are in.
|
Updated in lockstep with the implementation PR Three docs-side asks from the previous review are addressed: 1. 2. Pinning section rewritten under MetalLB. 3. Troubleshooting Plus a fourth piece: new "LB allocator prerequisites" subsection in the migration block with a worked |
Aleksei Sviridkin (lexfrei)
force-pushed
the
docs/gateway-api-cilium
branch
from
b48b79d to
830357b
Compare
|
Mirrored the revert in Net change here:
What stays from the previous revision:
|
Aleksei Sviridkin (lexfrei)
force-pushed
the
docs/gateway-api-cilium
branch
3 times, most recently
from
7f83582 to
1e73f98
Compare
Andrei Kvapil (kvaps)
left a comment
Andrei Kvapil (kvaps)
left a comment
There was a problem hiding this comment.
LGTM — the three asks from the May 6 review (mirroring cozystack/cozystack#2470) are all addressed:
-
Security section restructured. The "seven independent layers" framing is gone, replaced by the three-group split (tenant-user-input gate / defense-in-depth / admin-against-themselves), and the mermaid now separates the
USERsource (Tenantspec.host→ L3) from theCHARTsource (app-chart bug / supply-chain → L1, L2, L4, L5) instead of converging everything from one attacker node. Layer 5 (cozystack-route-hostname-policy, formerly Layer 7) is reframed as defense-in-depth against a chart bug / supply-chain compromise, and the "tenant with HTTPRoute RBAC" wording is gone. -
Tenant API surface documented. The new
### Tenant API surfaceparagraph in Overview (and the Security intro) state explicitly that tenants write onlyapps.cozystack.io/*throughcozystack-apiand hold no RBAC ongateway.networking.k8s.io/*/ Namespaces / Package — which anchors the rest of the Security section correctly. -
gatewayIP/ IP-pinning reworked. NogatewayIPfield anywhere; the traffic-path section and troubleshooting now describe IP allocation as mechanism-agnostic (MetalLB / Cilium LB-IPAM / robotlb / externalIPs), with pinning done admin-side vialoadBalancerIPon the Service.platform-package.mdis consistent too (tenant-*entries inattachedNamespacesdocumented as allowed).
Nice touch: the Inheritance section correctly states there is no per-child ReferenceGrant — the label selector is the cross-namespace gate.
One cross-PR consistency note (against the implementation repo, not this page): this page renumbered the model to 5 layers, but packages/extra/gateway/README.md in cozystack/cozystack#2470 still uses the 7-layer numbering and still documents cozystack-gateway-attached-namespaces-policy (Layer 3) and the render-time tenant-* ban (Layer 6) — both removed by the inheritance rework (the VAP no longer exists in the tree). The controller comments there also reference "Layer 7" for what this page calls Layer 5. Worth syncing the README/comments down to this page's 5-layer model so the two don't drift. Flagging that on the #2470 side too.
Approving.
Aleksei Sviridkin (lexfrei)
force-pushed
the
docs/gateway-api-cilium
branch
4 times, most recently
from
dab94b4 to
2801a7d
Compare
Aleksei Sviridkin (lexfrei)
force-pushed
the
docs/gateway-api-cilium
branch
3 times, most recently
from
b7ba196 to
0a4c992
Compare
Documents the per-tenant Gateway API ingress that landed in cozystack/cozystack#2470: - TenantGateway CRD reconciled by cozystack-controller: chart renders one CR per opted-in tenant, controller materialises the Gateway, per-tenant Issuer, redirect HTTPRoute, and Certificate(s). - Inheritance through the namespace.cozystack.io/gateway label: a tenant only owns a dedicated Gateway when it explicitly sets tenant.spec.gateway=true; every other tenant in the tree attaches through the nearest ancestor that owns one. The apps/tenant chart writes the label; the controller patches the same label onto cozy-* namespaces listed in attachedNamespaces. - HTTP-01 (default) renders per-listener Certificates; DNS-01 (opt-in) renders one wildcard cert per owning Gateway and extends it with per-child-apex SANs + one *.<child-apex> listener per inheriting tenant. Four DNS providers supported (cloudflare, route53, digitalocean, rfc2136). - Mechanism-agnostic LoadBalancer IP allocation: the auto-created Service draws its IP from whatever LB allocator the cluster admin has configured (MetalLB / Cilium LB-IPAM / robotlb / externalIPs). Cozystack ships MetalLB installed but does not auto-render any IPAddressPool / L2Advertisement / CiliumLoadBalancerIPPool. - Per-service routing tables: HTTPRoute for dashboard, keycloak, harbor, bucket; TLSRoute for kubernetes-api, vm-exportproxy, cdi-uploadproxy. - Five-layer security model grouped by who it defends against: Layer 4 gates tenant-user input on Tenant.spec.host; Layers 1, 2, 5, 7 are defense-in-depth against chart bugs / controller bugs / supply-chain compromise / admin mistakes; tenants do not hold gateway.networking.k8s.io/* RBAC by design. Layer 1 selector mechanics differ by listener role (HTTPS: inheritance label; HTTP-80: built-in metadata.name whitelist). - Foreign-takeover guards on six paths (Gateway, redirect HTTPRoute, Issuer, wildcard Certificate, per-listener Certificate, and the namespace label patching via cozystack.io/gateway-attached-by annotation) so hand-pinned operator state is never silently overwritten. - Migration story from ingress-nginx (per-cluster and per-tenant), troubleshooting recipes for the most likely stuck states. Also adds the Gateway and DNS-01 provider value rows to the platform-package reference. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la>
Aleksei Sviridkin (lexfrei)
force-pushed
the
docs/gateway-api-cilium
branch
from
0a4c992 to
475a505
Compare
3 participants
What this PR does
Adds a new
networking/gateway-api.mdpage to thenext/docs trunk describing the Cilium-backed Gateway API feature that landed in cozystack/cozystack#2470. Also extends the platform-package reference with thegateway.*andpublishing.certificates.dns01.*value rows the same upstream PR introduces.The page covers, as the feature shipped:
gateway.enabled) and a per-tenant toggle (tenant.spec.gateway) that opts a tenant into its own dedicated Gateway (own LB IP, own per-tenant Issuer, own Certificate);namespace.cozystack.io/gatewaylabel: tenants withoutspec.gateway: trueattach their routes to the nearest ancestor's Gateway via a label-keyedallowedRoutes.namespaces.selector(same shape as_namespace.ingressinheritance);Service.spec.externalIPs). Cozystack ships MetalLB installed but does not auto-render anyIPAddressPool/L2Advertisement/BGPAdvertisement/CiliumLoadBalancerIPPool. There is nogatewayIPfield on the Tenant CR — to pin an address, the operator pre-creates the Service withloadBalancerIPset or hands the tenant a reference to a named admin-managed pool;Ingressfor every cozystack-native exposed service (dashboard, keycloak, grafana, alerta via HTTPRoute; kubernetes-api, vm-exportproxy, cdi-uploadproxy via TLSRoute passthrough; harbor and bucket attached to the owning tenant's Gateway);Issuer(own ACME account) plus per-listener Certificates (HTTP-01) or a single wildcard Certificate extended with per-child-apex SANs and a*.<child-apex>listener per inheriting tenant (DNS-01). Four DNS providers supported (cloudflare, route53, digitalocean, rfc2136);Tenant.spec.hostviacozystack-api's admission chain; Layers 1, 2, 4, 5 are defense-in-depth via VAPs and the listener label selector (tenants do not holdgateway.networking.k8s.io/*RBAC by design);namespace.cozystack.io/gatewayon namespaces it annotates withcozystack.io/gateway-attached-by).Sections
allowedRoutesselector mechanics per listener role.attachedNamespaceslist, per-tenant Tenant example for owning and inheriting tenants, custom-apex case.v1alpha2, DNS-01 multi-apex DNS-provider requirement, supported ACME issuers, upstream application gaps.kubectlrecipes for the most likely stuck states (TenantGateway ReconcileError, Gateway not Programmed, Certificate not Ready, HostnameConflict, admission denials, inheriting child route not accepted,<pending>LB IP).Target branch
next/— the version-agnostic trunk. When the page ships with the upstream release that contains cozystack/cozystack#2470, it materialises automatically under the correspondingvX.Y/viamake release-next.Not included
The legacy
v1/networking/gateway-api.mdpage on the abandoneddocs/gateway-apibranch (from the Envoy Gateway proposal in cozystack/cozystack#2213) is unrelated to this PR. That proposal targeted a different architecture that has since been superseded.Release note