Skip to content

NO-JIRA: Fix races in machineset sync units#551

Open
mdbooth wants to merge 3 commits intoopenshift:mainfrom
openshift-cloud-team:machinesetsync-races
Open

NO-JIRA: Fix races in machineset sync units#551
mdbooth wants to merge 3 commits intoopenshift:mainfrom
openshift-cloud-team:machinesetsync-races

Conversation

@mdbooth
Copy link
Copy Markdown
Contributor

@mdbooth mdbooth commented May 7, 2026

  • Fix test race deleting CAPA machine template
  • Fix incorrect uses of k8sClient.Create in Eventually
  • Fix machineset sync test races between MAPI and CAPI object creation

Summary by CodeRabbit

  • Tests
    • Improved synchronization and reliability of MachineSet reconciliation tests for ClusterAPI scenarios: added post-status-update hooks, explicit waits for informer cache/observed-generation propagation, and more robust deletion/finalizer flows.
    • Centralized and enhanced test resource creation to support option-aware creates, consistent dry-run handling, and uniform retry/error patterns across test suites.

@openshift-merge-bot
Copy link
Copy Markdown
Contributor

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: LGTM mode

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label May 7, 2026
@openshift-ci-robot
Copy link
Copy Markdown

@mdbooth: This pull request explicitly references no jira issue.

Details

In response to this:

  • Fix test race deleting CAPA machine template
  • Fix incorrect uses of k8sClient.Create in Eventually
  • Fix machineset sync test races between MAPI and CAPI object creation

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 7, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 6d151287-d3d9-4c9a-88e9-76beab9a518e

📥 Commits

Reviewing files that changed from the base of the PR and between ee8c828 and eb108fa.

📒 Files selected for processing (1)
  • pkg/controllers/machinesetsync/machineset_sync_controller_test.go
🚧 Files skipped from review as they are similar to previous changes (1)
  • pkg/controllers/machinesetsync/machineset_sync_controller_test.go

Walkthrough

The PR updates MachineSet-sync tests to add informer-cache synchronization (waits for Status.ObservedGeneration and template observation), introduces an afterMAPIUpdate hook to defer scenario setup until after MAPI status updates, and changes the kCreate test helper to accept variadic client.CreateOption for option-aware creates.

Changes

MachineSet sync tests

Layer / File(s) Summary
Test helper signature
pkg/controllers/machinesetsync/machineset_sync_controller_test.go
kCreate(ctx context.Context, obj client.Object)kCreate(ctx context.Context, obj client.Object, opts ...client.CreateOption); forwards opts to k8sClient.Create.
Data / Hook
pkg/controllers/machinesetsync/machineset_sync_controller_test.go
Adds afterMAPIUpdate callback and runs it from JustBeforeEach after updating MAPI AuthoritativeAPI and Status.ObservedGeneration.
Informer-cache wait helper
pkg/controllers/machinesetsync/machineset_sync_controller_test.go
Adds eventuallyManagerInformerCache helper that polls mgr.GetClient() until objects reach expected state in the manager’s informer cache.
Cache waits (CAPI)
pkg/controllers/machinesetsync/machineset_sync_controller_test.go
Adds Eventually polling against the manager client to wait for CAPI MachineSet Status.ObservedGeneration to match Generation in multiple scenarios.
Cache waits (CAPA template)
pkg/controllers/machinesetsync/machineset_sync_controller_test.go
Waits for CAPA MachineTemplate objects to be observed by the manager’s informer cache before proceeding.
Deletion / Finalizer flow
pkg/controllers/machinesetsync/machineset_sync_controller_test.go
Refactors CAPI deletion scenarios to set up finalizer preconditions via afterMAPIUpdate and simulates finalizer removal using slices.DeleteFunc during cleanup assertions.
Test wiring / helper usage
pkg/controllers/machinesetsync/machineset_vap_test.go
Replaces direct k8sClient.Create() and inline retry logic with Eventually(kCreate(...)) across namespace, AWSCluster/Cluster, templates, MachineSets, and sentinel object creation; uses client.DryRunAll where relevant.
Imports / minor cleanup
pkg/controllers/machinesetsync/machineset_sync_controller_test.go, pkg/controllers/machinesetsync/machineset_vap_test.go
Adds slices import and removes inline api-errors retry code in VAP tests due to centralized create helper usage.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

🚥 Pre-merge checks | ✅ 12
✅ Passed checks (12 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and specifically describes the main change: fixing race conditions in machineset sync unit tests, which aligns directly with the PR objectives and file modifications.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed All 123 Ginkgo test declarations in modified files use stable, deterministic names. No dynamic content (fmt.Sprintf, concatenation, UUIDs, timestamps, generated identifiers) found in test titles.
Test Structure And Quality ✅ Passed Tests have proper setup/cleanup, JustBeforeEach for race prevention, assertion messages, timeout config (2s), and consistent komega helper usage with appropriate timeouts.
Microshift Test Compatibility ✅ Passed Custom check is for new Ginkgo e2e tests. This PR only modifies unit tests in pkg/controllers/, not e2e tests. Check not applicable to unit test changes.
Single Node Openshift (Sno) Test Compatibility ✅ Passed This PR modifies unit/controller tests in envtest, not e2e tests. The check applies to new e2e tests only. No new tests are added, only existing tests are refactored to fix races. Not applicable.
Topology-Aware Scheduling Compatibility ✅ Passed PR modifies only test files. No production code, manifests, or operators changed. No scheduling constraints, affinity rules, or topology selectors introduced. Check not applicable.
Ote Binary Stdout Contract ✅ Passed No process-level stdout writes found. Changes limited to test blocks and helper functions. kCreate modified only for flexibility. Suite logging properly configured.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed These are unit/integration tests using local envtest with no IPv4 assumptions or external connectivity requirements. Tests use local Kubernetes object builders with no public internet access.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Tip

💬 Introducing Slack Agent: The best way for teams to turn conversations into code.

Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.

  • Generate code and open pull requests
  • Plan features and break down work
  • Investigate incidents and troubleshoot customer tickets together
  • Automate recurring tasks and respond to alerts with triggers
  • Summarize progress and report instantly

Built for teams:

  • Shared memory across your entire org—no repeating context
  • Per-thread sandboxes to safely plan and execute work
  • Governance built-in—scoped access, auditability, and budget controls

One agent for your entire SDLC. Right inside Slack.

👉 Get started


Comment @coderabbitai help to get the list of available commands and usage tips.

@openshift-ci openshift-ci Bot requested review from damdo and racheljpg May 7, 2026 16:50
@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented May 7, 2026

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign damdo for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@mdbooth mdbooth force-pushed the machinesetsync-races branch from 126b371 to ee8c828 Compare May 8, 2026 09:36
@mdbooth
Copy link
Copy Markdown
Contributor Author

mdbooth commented May 8, 2026

/test e2e-aws-capi-techpreview

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented May 8, 2026

@mdbooth: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

jira/valid-reference Indicates that this PR references a valid Jira ticket of any type.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants