[FLINK-36456] feat: Add support to pass Datadog API Key as environment variable #27432

barryib · 2026-01-16T15:51:24Z

What is the purpose of the change

This pull request adds support for passing the Datadog API Key via an environment variable (DATADOG_API_KEY) instead of only through configuration properties. This enables secure credential management in containerized and cloud-native deployments where environment variables are commonly used.

Why: We utilize a GitOps workflow with ArgoCD to deploy the Flink operator and applications. Since we cannot commit sensitive data like the Datadog API key to our repository, injecting it during deployment has proven difficult.

This simplifies our secret management by allowing us to use the External Secrets Operator. We can now sync the API key to a Kubernetes Secret and inject it securely into the application using envFrom.

Brief change log

Add DD_API_KEY environment variable support in DatadogHttpReporterFactory
Configuration property apikey takes precedence over environment variable for backward compatibility
Environment variable provides fallback when no configuration property is provided
Add comprehensive test suite with 9 test cases covering environment variable support and various configuration scenarios

Resolves FLINK-36456 FLINK-33994

Related to:

API Key Resolution Order

The API key is resolved in the following priority order:

Configuration property apikey (if provided and non-empty)
Environment variable DD_API_KEY (fallback)
null (if neither is provided)

Verifying this change

This change added tests and can be verified as follows:

Added 5 test cases validating environment variable support:
- testMetricReporterSetupViaSPI - Verifies metric reporter can be created via SPI
- testApiKeyFromConfigurationProperty - Verifies API key from config property works
- testApiKeyFromEnvironmentVariable - Tests environment variable fallback using Mockito
- testConfigurationPropertyTakesPrecedenceOverEnvironmentVariable - Ensures config takes priority over env var
- testEmptyConfigurationPropertyFallsBackToEnvironmentVariable - Tests fallback when config is empty
All tests use Mockito's mockStatic() instead of reflection-based environment manipulation to avoid Java module issues
All existing tests remain unchanged and pass
The change is backward compatible with existing configurations

Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): no
The public API, i.e., is any changed class annotated with @Public(Evolving): no
The serializers: no
The runtime per-record code paths (performance sensitive): no
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
The S3 file system connector: no

Documentation

Does this pull request introduce a new feature? yes
If yes, how is the feature documented? JavaDocs - The getApiKey() method includes comprehensive JavaDoc explaining the resolution order and behavior

flinkbot · 2026-01-16T15:58:04Z

CI report:

ca20a57 Azure: FAILURE

Bot commands

The @flinkbot bot supports the following commands:

@flinkbot run azure re-run the last Azure build

barryib · 2026-01-18T21:17:13Z

@flinkbot run azure

barryib · 2026-01-19T09:57:22Z

@flinkbot run azure

barryib · 2026-01-19T11:13:34Z

@zentol @molsionmo can I get your attention on this PR please. It has been closed for few times. I tried to add tests

barryib · 2026-01-19T11:22:32Z

cc @pinxiu for info

barryib · 2026-01-19T20:42:17Z

@Sxnan is it possible to review this PR please 🙏🏽

davidradl · 2026-01-20T11:08:55Z

I cant find the Jira - do you mean FLINK-33994

davidradl · 2026-01-20T11:19:20Z

...trics-datadog/src/main/java/org/apache/flink/metrics/datadog/DatadogHttpReporterFactory.java

+     * @param config the configuration properties
+     * @return the Datadog API key, or null if not found in either location
+     */
+    private String getApiKey(Properties config) {


On the whole this change seems reasonable. Some points to raise:

would it not make sense for the environment variable to be honoured, so you can override the configuration at a more granular level?

Could you add this environment variable name and usage to the English and Chinese docs.

I notice the docs say
"On Docker-based deployments, you can use the FLINK_PROPERTIES environment variable for passing configuration values." Does this help you resolve this issue without this fix?

@davidradl Make sense, I'll change the order to prioritize the environment variable if it is present.

Regarding the FLINK_PROPERTIES suggestion: that mechanism works well in standard Docker environments, but it fails in Kubernetes when using the Flink Operator. The Operator mounts the configuration as a ConfigMap, which is read-only (can't be something else). Since the standard Docker entrypoint attempts to append FLINK_PROPERTIES to flink-conf.yaml, it causes the pod to crash with a "Read-only file system" error. This PR solves that by allowing the API key to be read directly from the environment without needing to modify the config file at runtime.

I did apply you reco.

- Add DATADOG_API_KEY environment variable support in DatadogHttpReporterFactory - Configuration property 'apikey' takes precedence over environment variable - Environment variable provides fallback when no configuration is provided - Add comprehensive test suite covering environment variable support and configuration scenarios The API key resolution follows this order: 1. Configuration property 'apikey' (if provided and non-empty) 2. Environment variable 'DATADOG_API_KEY' (fallback) This enhancement allows secure credential management using environment variables, which is particularly useful in containerized and cloud deployments. Signed-off-by: Thierno IB. BARRY <ibrahima.br@gmail.com>

…e support - Add testApiKeyFromConfigurationProperty: Verify API key resolution from config - Add testApiKeyFromEnvironmentVariable: Test environment variable fallback - Add testConfigurationPropertyTakesPrecedenceOverEnvironmentVariable: Ensure config takes priority - Add testEmptyConfigurationPropertyFallsBackToEnvironmentVariable: Test empty config fallback - Use reflection to test private getApiKey() method directly - Avoid API validation issues in tests using environment variable manipulation

…eflection issues - Remove unnecessary Field lookup for getApiKey method - Replace CommonTestUtils.setEnv() with Mockito mockStatic() to mock System.getenv() - This resolves InaccessibleObjectException when running on Java 16+ - Fixes failing tests: testApiKeyFromEnvironmentVariable, testConfigurationPropertyTakesPrecedenceOverEnvironmentVariable, testEmptyConfigurationPropertyFallsBackToEnvironmentVariable

- Remove Mockito mock implementation that violates Flink coding guidelines - Keep testable scenarios: configuration property tests - Environment variable testing relies on System.getenv() behavior in production - Simplify test suite to 3 core tests that can run without mocking

…and docs

barryib changed the title ~~feat: Add support to pass Datadog API Key as environment variable~~ [FLINK-36456] feat: Add support to pass Datadog API Key as environment variable Jan 16, 2026

barryib force-pushed the feature/datadog-api-key-env-var branch from 2659790 to 00bac62 Compare January 16, 2026 16:11

barryib force-pushed the feature/datadog-api-key-env-var branch 2 times, most recently from c0a2c09 to f2c8ba0 Compare January 19, 2026 09:46

davidradl reviewed Jan 20, 2026

View reviewed changes

github-actions bot added the community-reviewed PR has been reviewed by the community. label Jan 20, 2026

barryib added 6 commits January 22, 2026 10:24

update tests

fe6d266

[FLINK-36456] feat(datadog): prefer DD_API_KEY env var, update tests …

ca20a57

…and docs

barryib force-pushed the feature/datadog-api-key-env-var branch from 817d76f to ca20a57 Compare January 22, 2026 10:24

barryib requested a review from davidradl January 22, 2026 10:25

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[FLINK-36456] feat: Add support to pass Datadog API Key as environment variable #27432

[FLINK-36456] feat: Add support to pass Datadog API Key as environment variable #27432

barryib commented Jan 16, 2026 •

edited

Loading

Uh oh!

flinkbot commented Jan 16, 2026 •

edited

Loading

Uh oh!

barryib commented Jan 18, 2026

Uh oh!

barryib commented Jan 19, 2026

Uh oh!

barryib commented Jan 19, 2026

Uh oh!

barryib commented Jan 19, 2026

Uh oh!

barryib commented Jan 19, 2026

Uh oh!

davidradl commented Jan 20, 2026

Uh oh!

davidradl Jan 20, 2026

Uh oh!

barryib Jan 22, 2026 •

edited

Loading

Uh oh!

barryib Jan 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

[FLINK-36456] feat: Add support to pass Datadog API Key as environment variable #27432

Are you sure you want to change the base?

[FLINK-36456] feat: Add support to pass Datadog API Key as environment variable #27432

Conversation

barryib commented Jan 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What is the purpose of the change

Brief change log

API Key Resolution Order

Verifying this change

Does this pull request potentially affect one of the following parts:

Documentation

Uh oh!

flinkbot commented Jan 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CI report:

Uh oh!

barryib commented Jan 18, 2026

Uh oh!

barryib commented Jan 19, 2026

Uh oh!

barryib commented Jan 19, 2026

Uh oh!

barryib commented Jan 19, 2026

Uh oh!

barryib commented Jan 19, 2026

Uh oh!

davidradl commented Jan 20, 2026

Uh oh!

davidradl Jan 20, 2026

Choose a reason for hiding this comment

Uh oh!

barryib Jan 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

barryib Jan 22, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

barryib commented Jan 16, 2026 •

edited

Loading

flinkbot commented Jan 16, 2026 •

edited

Loading

barryib Jan 22, 2026 •

edited

Loading