Trim metadata logging and add logging fields

[khewonc]

What does this PR do?

Use the same logging fields as DDAI controller (see #2519). Also increase the logging verbosity of some logs and remove extraneous logs as it's a lot of logging at the debug level.

Motivation

https://datadoghq.atlassian.net/browse/CONTP-1334

Additional Notes

There is still excessive logging for DatadogMetricForwarders. That needs to be changed in a future PR

Minimum Agent Versions

Are there minimum versions of the Datadog Agent and/or Cluster Agent required?

• Agent: vX.Y.Z
• Cluster Agent: vX.Y.Z

Describe your test plan

Check the operator logs for New CRD detected. Previously, you'd see:

{"level":"DEBUG","ts":"2026-03-09T14:06:26.946Z","logger":"metadata.crd","msg":"New CRD detected","key":"DatadogAgent/default/datadog-agent"}

Now you should see:

{"level":"DEBUG","ts":"2026-03-09T20:29:26.401Z","logger":"metadata.crd","msg":"New CRD detected","kind":"DatadogAgent","namespace":"default","name":"datadog-agent"}

Overall, there should also be fewer logs from the metadata.* logger compared to previous releases.

Checklist

• PR has at least one valid label: bug, enhancement, refactoring, documentation, tooling, and/or dependencies
• PR has a milestone or the qa/skip-qa label
• All commits are signed (see: signing commits)

[codecov-commenter]

Codecov Report

❌ Patch coverage is 36.00000% with 16 lines in your changes missing coverage. Please review.
✅ Project coverage is 38.82%. Comparing base (a2fb04f) to head (b21a725).

Files with missing lines	Patch %	Lines
pkg/controller/utils/metadata/operator_metadata.go	22.22%	7 Missing ⚠️
pkg/controller/utils/metadata/helm_metadata.go	0.00%	5 Missing ⚠️
pkg/controller/utils/metadata/crd_metadata.go	70.00%	3 Missing ⚠️
pkg/controller/utils/datadog/metrics_forwarder.go	0.00%	1 Missing ⚠️

❌ Your patch status has failed because the patch coverage (36.00%) is below the target coverage (80.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #2725   +/-   ##
=======================================
  Coverage   38.81%   38.82%           
=======================================
  Files         308      308           
  Lines       26705    26700    -5     
=======================================
  Hits        10365    10365           
+ Misses      15561    15556    -5     
  Partials      779      779           

Flag	Coverage Δ
unittests	38.82% <36.00%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
pkg/controller/utils/metadata/shared_metadata.go	85.71% <ø> (-0.23%)	⬇️
pkg/controller/utils/datadog/metrics_forwarder.go	32.40% <0.00%> (ø)
pkg/controller/utils/metadata/crd_metadata.go	42.52% <70.00%> (+1.04%)	⬆️
pkg/controller/utils/metadata/helm_metadata.go	28.66% <0.00%> (+0.09%)	⬆️
pkg/controller/utils/metadata/operator_metadata.go	33.65% <22.22%> (ø)

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a2fb04f...b21a725. Read the comment docs.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.