Skip to content

Avoid RBAC errors when Operator can't list or watch Secrets#2793

Open
levan-m wants to merge 1 commit intomainfrom
levan-m/secret-watch-without-rbac
Open

Avoid RBAC errors when Operator can't list or watch Secrets#2793
levan-m wants to merge 1 commit intomainfrom
levan-m/secret-watch-without-rbac

Conversation

@levan-m
Copy link
Collaborator

@levan-m levan-m commented Mar 20, 2026

What does this PR do?

Fixes bug reported in #2791.

  • Don't start secrets watch in helm metadata forwarder if Operator doesn't have list/watch RBAC.
  • Update CredentialManager to avoid unnecessary secrets informer initialization.

Motivation

Additional Notes

Anything else we should know when reviewing?

Minimum Agent Versions

Are there minimum versions of the Datadog Agent and/or Cluster Agent required?

  • Agent: vX.Y.Z
  • Cluster Agent: vX.Y.Z

Describe your test plan

  1. install Operator
❯ helm install operator datadog/datadog-operator \
--set datadogMonitor.enabled=true \
--set datadogAgent.enabled=false \
--set datadogAgentInternal.enabled=false
  1. Update rbac to drop secret watch, list permissions, restart operator pod.
  kubectl get clusterrole operator-datadog-operator -o json \
    | jq '
        .rules |= map(
          if (.resources // [] | index("secrets")) != null
          then .resources |= map(select(. != "secrets"))
          else .
          end
        )
        | .rules += [{"apiGroups":[""],"resources":["secrets"],"verbs":["create","delete","get","patch","update"]}]
      ' \
    | kubectl apply -f -

kubectl rollout restart deployment/operator-datadog-operator
  1. Observer errors
    ..."logger":"klog","msg":"Failed to watch","logger":"UnhandledError","reflector":"pkg/mod/k8s.io/client-go@v0.33.3/tools/cache/reflector.go:285","type":"*v1.Secret","error":"failed to list *v1.Secret: secrets is forbidden: User \"system:serviceaccount:default:operator-datadog-operator\" cannot list resource \"secrets\" in API group \"\" in

  2. Update to fixed image; after restart Operator should log
    {"level":"INFO","ts":"2026-03-20T19:23:55.561Z","logger":"metadata.helm","msg":"No permission to list/watch Secrets, Helm metadata collection from Secrets will be disabled"}

Checklist

  • PR has at least one valid label: bug, enhancement, refactoring, documentation, tooling, and/or dependencies
  • PR has a milestone or the qa/skip-qa label
  • All commits are signed (see: signing commits)

@levan-m levan-m added this to the v1.25.0 milestone Mar 20, 2026
@levan-m levan-m requested a review from a team March 20, 2026 19:31
@levan-m levan-m added bug Something isn't working team/container-platform labels Mar 20, 2026
Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 084333630d

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +168 to +171
ResourceAttributes: &authorizationv1.ResourceAttributes{
Verb: verb,
Resource: "secrets",
},

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Scope Secret RBAC check to watched namespaces

canListWatchSecrets builds a SelfSubjectAccessReview without resourceAttributes.namespace, which Kubernetes interprets as an all-namespaces check for namespaced resources. In namespace-scoped deployments (for example when WATCH_NAMESPACE/DD_AGENT_WATCH_NAMESPACE limits cache scope and RBAC is granted per namespace), this returns denied even though list/watch is allowed in the watched namespace(s), so secretAccessEnabled is set to false and Secret-backed Helm releases are never processed. This is a functional regression for namespace-scoped RBAC setups.

Useful? React with 👍 / 👎.

@codecov-commenter
Copy link

Codecov Report

❌ Patch coverage is 2.12766% with 46 lines in your changes missing coverage. Please review.
✅ Project coverage is 38.75%. Comparing base (d134930) to head (0843336).
⚠️ Report is 5 commits behind head on main.

Files with missing lines Patch % Lines
pkg/controller/utils/metadata/helm_metadata.go 0.00% 44 Missing ⚠️
cmd/main.go 0.00% 1 Missing ⚠️
pkg/config/creds.go 50.00% 1 Missing ⚠️

❌ Your patch status has failed because the patch coverage (2.12%) is below the target coverage (80.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #2793      +/-   ##
==========================================
- Coverage   38.78%   38.75%   -0.03%     
==========================================
  Files         309      309              
  Lines       26852    26871      +19     
==========================================
  Hits        10414    10414              
- Misses      15658    15677      +19     
  Partials      780      780              
Flag Coverage Δ
unittests 38.75% <2.12%> (-0.03%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
cmd/main.go 6.64% <0.00%> (ø)
pkg/config/creds.go 62.06% <50.00%> (ø)
pkg/controller/utils/metadata/helm_metadata.go 26.68% <0.00%> (-1.66%) ⬇️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d134930...0843336. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working team/container-platform

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants