OCPCLOUD-2950: Adopt projected-service-account-tokens-for-credential-providers by theobarberbany · Pull Request #1987 · openshift/enhancements

theobarberbany · 2026-04-27T12:27:09Z

No description provided.

openshift-ci · 2026-04-27T12:27:14Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

openshift-ci · 2026-04-27T12:27:26Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign jhjaggars for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci-robot · 2026-04-27T12:46:32Z

@theobarberbany: This pull request references OCPCLOUD-2950 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the spike to target the "5.0.0" version, but no target version was set.

Details

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

damdo

Thanks @theobarberbany

Not an expert but on a first pass, the proposal looks sensible to me. Interested in hearing more from the other parties involved

openshift-ci · 2026-04-29T09:00:56Z

@theobarberbany: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/markdownlint	`63e3a61`	link	true	`/test markdownlint`

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

bennerv · 2026-05-04T20:32:40Z

+
+* As a cluster administrator, I want the credential provider configuration to ship with sensible defaults but be customizable, so that I can adapt it to my organization's cloud identity setup without maintaining my own MCO overrides from scratch.
+
+* As a security-conscious customer, I want the installer to not provision a default node identity for image pulls, so that I can reduce the blast radius of node compromise and use day-2 provisioned credentials instead of install-time baked-in identity.


This seems unnecessary especially since it's listed as a non-goal/out of scope below.

bennerv · 2026-05-04T20:34:12Z

+#### Out of scope but enabled by this work
+
+- **Removing install-time node identity from the installer.** KEP-4412 adoption is a prerequisite for this work. See [Removing node identity from install](#removing-node-identity-from-install) for the proposed approach.
+- **Removing the ROSA ECR token refresh shim.** ROSA currently works around the lack of credential provider support by injecting a Python script via MachineConfig as a systemd unit on a 4h timer to fetch ECR tokens. KEP-4412 replaces this with the kubelet's native credential provider framework. See [ARO-24037](https://redhat.atlassian.net/browse/ARO-24037).


Reference to ARO card but discusses ROSA. Assuming that's because the ARO card provides context is that RHCOS doesn't ship by default with the xcr-credentials-provider binaries and as a result ROSA is using this "hack" to do it.

JoelSpeed · 2026-05-05T11:34:09Z

+#### Out of scope but enabled by this work
+
+- **Removing install-time node identity from the installer.** KEP-4412 adoption is a prerequisite for this work. See [Removing node identity from install](#removing-node-identity-from-install) for the proposed approach.
+- **Removing the ROSA ECR token refresh shim.** ROSA currently works around the lack of credential provider support by injecting a Python script via MachineConfig as a systemd unit on a 4h timer to fetch ECR tokens. KEP-4412 replaces this with the kubelet's native credential provider framework. See [ARO-24037](https://redhat.atlassian.net/browse/ARO-24037).


How does this KEP support mirroring? I thought the issue was primarily the order in which these things were called? The mirroring config is after the credential provider today right?

JoelSpeed · 2026-05-05T11:44:25Z

+
+### API Extensions
+
+None. This enhancement does not introduce new Custom Resource Definitions or modify the OpenShift API surface. The `tokenAttributes` field is part of the upstream kubelet `CredentialProviderConfig` API.


Users opt-in to this feature using annotations on the service level? Any dynamic config we need to add to allow them to do this or does it basically just work today if we configure the MCO correctly (which we don't today right?)

is there any reason why a cluster admin might not want this ability/therefore would want a knob to control if it's on/off?

Based on the What KEP-4412 adoption looks like section, it seems that the "API surface" is "Admin wrote a SA with specific annotations regarding cloud provider"? The MCO enablement seems straightforward so the admin control would just be "I don't write a very specific annotation and the feature is off"

JoelSpeed · 2026-05-05T11:45:11Z

+
+#### OpenShift Kubernetes Engine
+
+TBD — needs input from OKE team.


I don't think there's any difference between OCP and OKE in regards to this area

JoelSpeed · 2026-05-05T11:45:55Z

+1. **ECR (cloud-provider-aws)**: Make the annotation key configurable via args or environment variable. Currently hardcoded to `eks.amazonaws.com/ecr-role-arn` — EKS-specific, not usable for OpenShift without a carry patch.
+2. **ACR (cloud-provider-azure)**: KEP-4412 support is already merged ([PR #9907](https://github.com/kubernetes-sigs/cloud-provider-azure/pull/9907)). Annotation keys (`kubernetes.azure.com/acr-client-id`, `kubernetes.azure.com/acr-tenant-id`) are hardcoded in `pkg/credentialprovider/consts.go` — need the same configurability fix as ECR.
+3. **GCR (cloud-provider-gcp)**: Implement KEP-4412 support from scratch. Opportunity to establish the configurable-annotation-key pattern from the start.


Why do we need something configurable vs documenting the upstream annotations for each cloud?

I was thinking to avoid mentioning cloud provider managed Kubernetes offerings in our docs, e.g EKS, thinking that could cause confusion? Or just be unexpected and therefore odd UX.

Does the option work beyond eks? I'd probably just look for a way to add a new version of that name upstream that isn't eks specific

JoelSpeed · 2026-05-05T11:47:47Z

+The upstream ask is making annotation keys configurable — not proposing specific keys upstream. Once configurability lands, OpenShift chooses its own defaults independently.
+
+TBD — need to decide on OpenShift annotation keys for each provider. This is user-facing UX and we want reviewer input. Options:
+- Per-provider annotations: `openshift.io/ecr-role-arn`, `openshift.io/acr-client-id`, `openshift.io/gcr-service-account`


What benefit would this provide over just using the upstream defaults?

JoelSpeed · 2026-05-05T11:48:39Z

+TBD — need to decide on OpenShift annotation keys for each provider. This is user-facing UX and we want reviewer input. Options:
+- Per-provider annotations: `openshift.io/ecr-role-arn`, `openshift.io/acr-client-id`, `openshift.io/gcr-service-account`
+- A single generic OpenShift annotation that each plugin interprets
+- Accept the upstream vendor-specific keys as-is (requires no carry, but ties OpenShift UX to EKS/AKS conventions)


The azure ones seem fine. The AWS one is less good as it calls out EKS in the name.

Aligning to upstream has benefits in terms of this clearly being an upstream feature and therefore upstream docs making sense

yuqi-zhang

Seems straightfoward enough from the MCO side, although I think I don't fully appreciate what Removing install-time node identity from the installer entails yet

yuqi-zhang · 2026-05-08T23:20:39Z

+4. Calls AWS Security Token Service `AssumeRoleWithWebIdentity` with the service account token as the web identity token
+5. Returns temporary IAM credentials used for the ECR `GetAuthorizationToken` call
+
+Kubelet config:


Just to double check, the existing CredentialProviderConfig template would look something like:

apiVersion: kubelet.config.k8s.io/v1 kind: CredentialProviderConfig providers: - name: ecr-credential-provider ... {{- if isWorkloadIdentity . }} tokenAttributes: serviceAccountTokenAudience: "sts.amazonaws.com" ... {{- end }}

?

How does the service account get managed? Is it purely user created?

yuqi-zhang · 2026-05-08T23:22:17Z

+The preferred direction is upstream work to allow credential provider plugins to accept a static credential (e.g. a GCP service account key, AWS access key pair) via config file or environment variable. This would allow:
+
+- CCO (mint mode) provisions a scoped credential for image pulls
+- MCO ships a CredentialsRequest declaring the credential requirement, following the same pattern used by other OpenShift operators (e.g. the image-registry operator, the ingress operator)


Is this something we plan to do relatively soon, or is it still up for discussion whether we will go forward with this approach?

yuqi-zhang · 2026-05-08T23:25:26Z

+
+### API Extensions
+
+None. This enhancement does not introduce new Custom Resource Definitions or modify the OpenShift API surface. The `tokenAttributes` field is part of the upstream kubelet `CredentialProviderConfig` API.


Based on the What KEP-4412 adoption looks like section, it seems that the "API surface" is "Admin wrote a SA with specific annotations regarding cloud provider"? The MCO enablement seems straightforward so the admin control would just be "I don't write a very specific annotation and the feature is off"

Adds projected-service-account-tokens-for-credential-providers

b09ecc9

openshift-ci Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Apr 27, 2026

theobarberbany changed the title ~~Adds projected-service-account-tokens-for-credential-providers~~ OCPCLOUD-2950: Adds projected-service-account-tokens-for-credential-providers Apr 27, 2026

openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Apr 27, 2026

theobarberbany force-pushed the tb/cred-provider-projected-service-accounts branch from ed89bf1 to 2262ea2 Compare April 27, 2026 13:23

damdo reviewed Apr 27, 2026

View reviewed changes

restructure

a31c15b

theobarberbany force-pushed the tb/cred-provider-projected-service-accounts branch from 2262ea2 to a31c15b Compare April 27, 2026 14:29

Adds clarification on tokens / service accounts and the auth flow

baf3f25

theobarberbany force-pushed the tb/cred-provider-projected-service-accounts branch from ca4bf4e to baf3f25 Compare April 28, 2026 10:09

Adds path 3: CCO mint mode + upstream work on credential providers

ee467f3

theobarberbany changed the title ~~OCPCLOUD-2950: Adds projected-service-account-tokens-for-credential-providers~~ OCPCLOUD-2950: Adopt projected-service-account-tokens-for-credential-providers Apr 28, 2026

theobarberbany marked this pull request as ready for review April 28, 2026 10:15

openshift-ci Bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Apr 28, 2026

openshift-ci Bot requested review from bear-redhat and lmzuccarelli April 28, 2026 10:16

theobarberbany force-pushed the tb/cred-provider-projected-service-accounts branch 3 times, most recently from a239a43 to 7f9b92f Compare April 28, 2026 18:44

restructure: remove OIDC node identity removal

63e3a61

theobarberbany force-pushed the tb/cred-provider-projected-service-accounts branch from 7f9b92f to 63e3a61 Compare April 29, 2026 08:46

twolff-gh mentioned this pull request May 1, 2026

WIP: ARO-24037 feat(azure): enable ACR image pulls via Projected SA openshift/hypershift#8397

Closed

5 tasks

bennerv reviewed May 4, 2026

View reviewed changes

JoelSpeed reviewed May 5, 2026

View reviewed changes

yuqi-zhang reviewed May 8, 2026

View reviewed changes


		* As a cluster administrator, I want the credential provider configuration to ship with sensible defaults but be customizable, so that I can adapt it to my organization's cloud identity setup without maintaining my own MCO overrides from scratch.

		* As a security-conscious customer, I want the installer to not provision a default node identity for image pulls, so that I can reduce the blast radius of node compromise and use day-2 provisioned credentials instead of install-time baked-in identity.


		### API Extensions

		None. This enhancement does not introduce new Custom Resource Definitions or modify the OpenShift API surface. The `tokenAttributes` field is part of the upstream kubelet `CredentialProviderConfig` API.


		#### OpenShift Kubernetes Engine

		TBD — needs input from OKE team.

Conversation

theobarberbany commented Apr 27, 2026

Uh oh!

openshift-ci Bot commented Apr 27, 2026

Uh oh!

openshift-ci Bot commented Apr 27, 2026

Uh oh!

openshift-ci-robot commented Apr 27, 2026 • edited by openshift-ci Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

damdo left a comment

Choose a reason for hiding this comment

Uh oh!

openshift-ci Bot commented Apr 29, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yuqi-zhang left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

openshift-ci-robot commented Apr 27, 2026 •

edited by openshift-ci Bot

Loading