Skip to content

CNTRLPLANE-244: fix(ci): configure Codecov PR comments and add unit tests#8085

Merged
openshift-merge-bot[bot] merged 1 commit intoopenshift:mainfrom
bryan-cox:test-codecov-integration
Mar 30, 2026
Merged

CNTRLPLANE-244: fix(ci): configure Codecov PR comments and add unit tests#8085
openshift-merge-bot[bot] merged 1 commit intoopenshift:mainfrom
bryan-cox:test-codecov-integration

Conversation

@bryan-cox
Copy link
Copy Markdown
Member

@bryan-cox bryan-cox commented Mar 26, 2026

What this PR does / why we need it:

Fixes Codecov PR comments not appearing on PRs that have pending Prow e2e commit statuses.

Codecov's wait_for_ci setting defaults to true. From the Codecov YAML Reference, under codecov.notify:

"Should Codecov wait for all CI statuses to complete before sending ours. Note: Codecov considers all non-codecov statuses to be CI statuses"

Default: true

Prow creates pending commit statuses for optional e2e tests (e.g. ci/prow/e2e-aws, ci/prow/e2e-aks) that say "Waiting for pipeline condition to trigger this job". These statuses never resolve unless explicitly triggered via /test, so Codecov waits indefinitely and never posts PR comments or status checks.

Evidence:

PR Pending CI statuses Codecov comment posted?
#8095 (LiangquanLi930) None — e2e tests resolved as failure Yes
#8096 (dgoodwin) Only tide — no e2e statuses created (config-only PR) Yes
#8085 (bryan-cox) 5 pending Prow e2e statuses + tide No
#8088 (bryan-cox) 7 pending Prow e2e statuses + tide No

Changes:

  • wait_for_ci: false — post PR comments immediately after coverage upload
  • files added to comment layout — show per-file coverage diffs
  • Unit tests added for String(), GetRevision(), and GetKubeVersionForSupportedVersion()

Which issue(s) this PR fixes:

Fixes Codecov PR comment integration

Special notes for your reviewer:

The wait_for_ci fix won't take effect on this PR since Codecov reads codecov.yml from the default branch (main). After merge, push to any open PR to verify comments appear.

Checklist:

  • Subject and description added to both, commit and PR.
  • Relevant issues have been referenced.
  • This change includes docs.
  • This change includes unit tests.

@openshift-ci-robot
Copy link
Copy Markdown

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: LGTM mode

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Mar 26, 2026

Important

Review skipped

Auto reviews are limited based on label configuration.

🚫 Review skipped — only excluded labels are configured. (1)
  • do-not-merge/work-in-progress

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository YAML (base), Organization UI (inherited)

Review profile: CHILL

Plan: Pro

Run ID: 5d5a126f-7115-44e5-984b-74c3b21cca16

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Added two unit tests for supported version functions (TestString, TestGetRevision). Updated ExternalDNS deployment generation for the Azure provider to append environment variable AZURE_SDK_MAX_RETRIES=5 and CLI argument --azure-zones-cache-duration=1h. Extended TestExternalDNSDeployment_Build to assert the new Azure env var and CLI argument. Set ExternalDNSInterval to "3m" in e2e install options. Added codecov.notify.wait_for_ci: false to codecov.yml. No exported/public API signatures were changed.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented Mar 26, 2026

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bryan-cox

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 26, 2026
@openshift-ci openshift-ci Bot requested review from jparrill and sjenning March 26, 2026 18:51
@openshift-ci openshift-ci Bot added area/control-plane-operator Indicates the PR includes changes for the control plane operator - in an OCP release area/hypershift-operator Indicates the PR includes changes for the hypershift operator and API - outside an OCP release and removed do-not-merge/needs-area labels Mar 26, 2026
@bryan-cox bryan-cox force-pushed the test-codecov-integration branch from 599899f to bd4c7cc Compare March 27, 2026 16:41
@openshift-ci openshift-ci Bot added area/cli Indicates the PR includes changes for CLI area/testing Indicates the PR includes changes for e2e testing labels Mar 27, 2026
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@test/e2e/util/install.go`:
- Line 57: The global assignment installOpts.ExternalDNSInterval = "3m" slows
reconciliation for all providers; make this Azure-specific by applying the 3m
interval only when the test target is Azure (e.g., guard the assignment with a
provider/platform check such as provider == "azure" or platform.IsAzure());
leave installOpts.ExternalDNSInterval unset (or set to the default) for
non-Azure providers so only Azure e2e suites use the throttling mitigation.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Organization UI (inherited)

Review profile: CHILL

Plan: Pro

Run ID: ae285ba7-34a6-43c1-83f5-5ed437c86487

📥 Commits

Reviewing files that changed from the base of the PR and between bd4c7cc and 92d20c9.

📒 Files selected for processing (3)
  • cmd/install/assets/hypershift_operator.go
  • cmd/install/assets/hypershift_operator_test.go
  • test/e2e/util/install.go

Comment thread test/e2e/util/install.go Outdated
installOpts.ExternalDNSCredentials = opts.ExternalDNSCredentials
installOpts.ExternalDNSDomainFilter = opts.ExternalDNSDomainFilter
installOpts.ExternalDNSProvider = opts.ExternalDNSProvider
installOpts.ExternalDNSInterval = "3m"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Scope the 3m external-dns interval to Azure instead of all providers.

Line 57 globally slows reconciliation (--interval=3m) for all e2e providers, but the throttling mitigation is Azure-specific. This can add avoidable runtime to non-Azure suites.

Suggested fix
-	installOpts.ExternalDNSInterval = "3m"
+	if opts.ExternalDNSProvider == "azure" {
+		installOpts.ExternalDNSInterval = "3m"
+	}

As per coding guidelines, "Focus on major issues impacting performance, readability, maintainability and security. Avoid nitpicks and avoid verbosity."

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
installOpts.ExternalDNSInterval = "3m"
if opts.ExternalDNSProvider == "azure" {
installOpts.ExternalDNSInterval = "3m"
}
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@test/e2e/util/install.go` at line 57, The global assignment
installOpts.ExternalDNSInterval = "3m" slows reconciliation for all providers;
make this Azure-specific by applying the 3m interval only when the test target
is Azure (e.g., guard the assignment with a provider/platform check such as
provider == "azure" or platform.IsAzure()); leave
installOpts.ExternalDNSInterval unset (or set to the default) for non-Azure
providers so only Azure e2e suites use the throttling mitigation.

@bryan-cox bryan-cox force-pushed the test-codecov-integration branch 2 times, most recently from 3f9e5da to 1fe813d Compare March 27, 2026 19:33
@bryan-cox bryan-cox changed the title test(supportedversion): add tests for String and GetRevision fix(ci): disable Codecov wait_for_ci to unblock PR comments Mar 27, 2026
@bryan-cox bryan-cox force-pushed the test-codecov-integration branch from 53bbac0 to df0b7bb Compare March 27, 2026 19:43
@bryan-cox
Copy link
Copy Markdown
Member Author

/retest

@bryan-cox bryan-cox force-pushed the test-codecov-integration branch from df0b7bb to 62d34a5 Compare March 30, 2026 09:50
@bryan-cox bryan-cox changed the title fix(ci): disable Codecov wait_for_ci to unblock PR comments CNTRLPLANE-244: fix(ci): disable Codecov wait_for_ci to unblock PR comments Mar 30, 2026
@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Mar 30, 2026
@openshift-ci-robot
Copy link
Copy Markdown

openshift-ci-robot commented Mar 30, 2026

@bryan-cox: This pull request references CNTRLPLANE-244 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "4.22.0" version, but no target version was set.

Details

In response to this:

What this PR does / why we need it:

Fixes Codecov PR comments not appearing on PRs that have pending Prow e2e commit statuses.

Codecov's wait_for_ci setting defaults to true. From the Codecov YAML Reference, under codecov.notify:

"Should Codecov wait for all CI statuses to complete before sending ours. Note: Codecov considers all non-codecov statuses to be CI statuses"

Default: true

Prow creates pending commit statuses for optional e2e tests (e.g. ci/prow/e2e-aws, ci/prow/e2e-aks) that say "Waiting for pipeline condition to trigger this job". These statuses never resolve unless explicitly triggered via /test, so Codecov waits indefinitely and never posts PR comments or status checks.

Evidence:

PR Pending CI statuses Codecov comment posted?
#8095 (LiangquanLi930) None — e2e tests resolved as failure Yes
#8096 (dgoodwin) Only tide — no e2e statuses created (config-only PR) Yes
#8085 (bryan-cox) 5 pending Prow e2e statuses + tide No
#8088 (bryan-cox) 7 pending Prow e2e statuses + tide No

Changes:

  • wait_for_ci: false — post PR comments immediately after coverage upload
  • behavior: new — post a fresh comment on each push instead of updating in-place
  • files added to comment layout — show per-file coverage diffs
  • Unit tests added for String(), GetRevision(), and GetKubeVersionForSupportedVersion()

Which issue(s) this PR fixes:

Fixes Codecov PR comment integration

Special notes for your reviewer:

The wait_for_ci fix won't take effect on this PR since Codecov reads codecov.yml from the default branch (main). After merge, push to any open PR to verify comments appear.

Checklist:

  • Subject and description added to both, commit and PR.
  • Relevant issues have been referenced.
  • This change includes docs.
  • This change includes unit tests.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@bryan-cox bryan-cox force-pushed the test-codecov-integration branch from c7ec702 to 41e2a7a Compare March 30, 2026 10:11
@codecov
Copy link
Copy Markdown

codecov Bot commented Mar 30, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 26.68%. Comparing base (c25481f) to head (f1b582e).
⚠️ Report is 15 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8085      +/-   ##
==========================================
+ Coverage   26.56%   26.68%   +0.12%     
==========================================
  Files        1087     1090       +3     
  Lines      105042   105242     +200     
==========================================
+ Hits        27902    28087     +185     
+ Misses      74731    74730       -1     
- Partials     2409     2425      +16     

see 6 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@bryan-cox bryan-cox force-pushed the test-codecov-integration branch 2 times, most recently from 472ee9d to d775623 Compare March 30, 2026 10:27
- Set wait_for_ci: false so Codecov posts PR comments immediately
  instead of waiting for pending Prow e2e statuses that never resolve
- Add per-file coverage diffs to PR comment layout
- Add unit tests for String, GetRevision, and
  GetKubeVersionForSupportedVersion

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@bryan-cox bryan-cox changed the title CNTRLPLANE-244: fix(ci): disable Codecov wait_for_ci to unblock PR comments fix(ci): configure Codecov PR comments and add unit tests Mar 30, 2026
@openshift-ci-robot openshift-ci-robot removed the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Mar 30, 2026
@openshift-ci-robot
Copy link
Copy Markdown

@bryan-cox: No Jira issue is referenced in the title of this pull request.
To reference a jira issue, add 'XYZ-NNN:' to the title of this pull request and request another refresh with /jira refresh.

Details

In response to this:

What this PR does / why we need it:

Fixes Codecov PR comments not appearing on PRs that have pending Prow e2e commit statuses.

Codecov's wait_for_ci setting defaults to true. From the Codecov YAML Reference, under codecov.notify:

"Should Codecov wait for all CI statuses to complete before sending ours. Note: Codecov considers all non-codecov statuses to be CI statuses"

Default: true

Prow creates pending commit statuses for optional e2e tests (e.g. ci/prow/e2e-aws, ci/prow/e2e-aks) that say "Waiting for pipeline condition to trigger this job". These statuses never resolve unless explicitly triggered via /test, so Codecov waits indefinitely and never posts PR comments or status checks.

Evidence:

PR Pending CI statuses Codecov comment posted?
#8095 (LiangquanLi930) None — e2e tests resolved as failure Yes
#8096 (dgoodwin) Only tide — no e2e statuses created (config-only PR) Yes
#8085 (bryan-cox) 5 pending Prow e2e statuses + tide No
#8088 (bryan-cox) 7 pending Prow e2e statuses + tide No

Changes:

  • wait_for_ci: false — post PR comments immediately after coverage upload
  • files added to comment layout — show per-file coverage diffs
  • Unit tests added for String(), GetRevision(), and GetKubeVersionForSupportedVersion()

Which issue(s) this PR fixes:

Fixes Codecov PR comment integration

Special notes for your reviewer:

The wait_for_ci fix won't take effect on this PR since Codecov reads codecov.yml from the default branch (main). After merge, push to any open PR to verify comments appear.

Checklist:

  • Subject and description added to both, commit and PR.
  • Relevant issues have been referenced.
  • This change includes docs.
  • This change includes unit tests.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@bryan-cox bryan-cox force-pushed the test-codecov-integration branch from d775623 to f1b582e Compare March 30, 2026 10:29
@bryan-cox bryan-cox changed the title fix(ci): configure Codecov PR comments and add unit tests CNTRLPLANE-244: fix(ci): configure Codecov PR comments and add unit tests Mar 30, 2026
@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Mar 30, 2026
@openshift-ci-robot
Copy link
Copy Markdown

openshift-ci-robot commented Mar 30, 2026

@bryan-cox: This pull request references CNTRLPLANE-244 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "4.22.0" version, but no target version was set.

Details

In response to this:

What this PR does / why we need it:

Fixes Codecov PR comments not appearing on PRs that have pending Prow e2e commit statuses.

Codecov's wait_for_ci setting defaults to true. From the Codecov YAML Reference, under codecov.notify:

"Should Codecov wait for all CI statuses to complete before sending ours. Note: Codecov considers all non-codecov statuses to be CI statuses"

Default: true

Prow creates pending commit statuses for optional e2e tests (e.g. ci/prow/e2e-aws, ci/prow/e2e-aks) that say "Waiting for pipeline condition to trigger this job". These statuses never resolve unless explicitly triggered via /test, so Codecov waits indefinitely and never posts PR comments or status checks.

Evidence:

PR Pending CI statuses Codecov comment posted?
#8095 (LiangquanLi930) None — e2e tests resolved as failure Yes
#8096 (dgoodwin) Only tide — no e2e statuses created (config-only PR) Yes
#8085 (bryan-cox) 5 pending Prow e2e statuses + tide No
#8088 (bryan-cox) 7 pending Prow e2e statuses + tide No

Changes:

  • wait_for_ci: false — post PR comments immediately after coverage upload
  • files added to comment layout — show per-file coverage diffs
  • Unit tests added for String(), GetRevision(), and GetKubeVersionForSupportedVersion()

Which issue(s) this PR fixes:

Fixes Codecov PR comment integration

Special notes for your reviewer:

The wait_for_ci fix won't take effect on this PR since Codecov reads codecov.yml from the default branch (main). After merge, push to any open PR to verify comments appear.

Checklist:

  • Subject and description added to both, commit and PR.
  • Relevant issues have been referenced.
  • This change includes docs.
  • This change includes unit tests.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@bryan-cox
Copy link
Copy Markdown
Member Author

/retest

@enxebre
Copy link
Copy Markdown
Member

enxebre commented Mar 30, 2026

/lgtm

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Mar 30, 2026
@openshift-ci-robot
Copy link
Copy Markdown

Tests from second stage were triggered manually. Pipeline can be controlled only manually, until HEAD changes. Use command to trigger second stage.

@bryan-cox
Copy link
Copy Markdown
Member Author

/verified by @bryan-cox

@openshift-ci-robot openshift-ci-robot added the verified Signifies that the PR passed pre-merge verification criteria label Mar 30, 2026
@openshift-ci-robot
Copy link
Copy Markdown

@bryan-cox: This PR has been marked as verified by @bryan-cox.

Details

In response to this:

/verified by @bryan-cox

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@celebdor
Copy link
Copy Markdown
Collaborator

/lgtm
/override "ci/prow/e2e-aks"
/override "ci/prow/e2e-aws"
/override "ci/prow/e2e-aws-upgrade-hypershift-operator"
/override "ci/prow/e2e-kubevirt-aws-ovn-reduced"
/override "ci/prow/e2e-v2-aws"

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented Mar 30, 2026

@celebdor: Overrode contexts on behalf of celebdor: ci/prow/e2e-aks, ci/prow/e2e-aws, ci/prow/e2e-aws-upgrade-hypershift-operator, ci/prow/e2e-kubevirt-aws-ovn-reduced, ci/prow/e2e-v2-aws

Details

In response to this:

/lgtm
/override "ci/prow/e2e-aks"
/override "ci/prow/e2e-aws"
/override "ci/prow/e2e-aws-upgrade-hypershift-operator"
/override "ci/prow/e2e-kubevirt-aws-ovn-reduced"
/override "ci/prow/e2e-v2-aws"

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented Mar 30, 2026

@bryan-cox: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@openshift-merge-bot openshift-merge-bot Bot merged commit d2e7543 into openshift:main Mar 30, 2026
29 checks passed
@bryan-cox bryan-cox deleted the test-codecov-integration branch March 30, 2026 14:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. area/cli Indicates the PR includes changes for CLI area/control-plane-operator Indicates the PR includes changes for the control plane operator - in an OCP release area/hypershift-operator Indicates the PR includes changes for the hypershift operator and API - outside an OCP release area/testing Indicates the PR includes changes for e2e testing jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. verified Signifies that the PR passed pre-merge verification criteria

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants