Idempotent CREATE POLICY (CAS retry on 409 version conflict)#119
Merged
Idempotent CREATE POLICY (CAS retry on 409 version conflict)#119
Conversation
ISM policies are versioned in OpenSearch — a plain PUT to an existing
policy fails with HTTP 409 version_conflict_engine_exception. Authors
adopting the [Migration(N, journal: false)] reconciliation pattern need
CREATE POLICY to be idempotent so the policy body can be upserted from
source-of-truth on every startup; the previous behavior surfaced 409
as Failed and forced authors to back out the pattern.
DispatchCreatePolicyAsync now retries on 409:
1. PUT _plugins/_ism/policies/<id>
2. On 409: GET the existing policy, extract _seq_no and _primary_term
from the response root (ISM surfaces these at the doc root, not
under _source — unusual but documented OpenSearch behavior).
3. PUT again with ?if_seq_no=N&if_primary_term=M (CAS query params).
4. On second 409: hard fail (concurrent writer between GET and retry).
No behavior change when the policy doesn't exist (first PUT succeeds,
no GET, no retry). Mirrors LockHandle.RenewLockAsync's optimistic-
concurrency pattern.
Integration test extends OpenSearchTemplatePolicyIntegrationTests with
a create-twice-with-mutated-body scenario that exercises the CAS retry
path and verifies the post-condition body via GET. Gated by the
INTEGRATIONS define and runs in the nightly Testcontainers workflow.
Provider README ISM section grows a paragraph documenting the
idempotency contract.
Note: dispatch-level unit tests (mocking IOpenSearchLowLevelClient) are
out of scope — the repo has no existing dispatcher unit-test
infrastructure; all dispatch coverage lives in the integration suite.
Adding mock plumbing for one method is net-new work for marginal
incremental value over the integration test.
Match the provider README paragraph added with the dispatcher change. Site doc kept ASCII-only per site-build constraint.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Make
CREATE POLICYidempotent so authors can use it inside[Migration(N, journal: false)]reconciliation migrations without hitting HTTP 409 every restart after the first.ISM policies are versioned in OpenSearch — a plain
PUTto an existing policy fails with409 version_conflict_engine_exception. Previously the dispatcher surfaced that asFailed. Now it transparently:PUT /_plugins/_ism/policies/<id>(or the legacy prefix whenIsmEndpointDetectStepresolved that path).GETthe existing policy, read_seq_noand_primary_termfrom the response root (ISM surfaces these at the document root, not under_source— unusual but documented).PUTwith?if_seq_no=N&if_primary_term=Mquery parameters.No behavior change when the policy doesn't exist — the first
PUTsucceeds, no GET, no retry. MirrorsLockHandle.RenewLockAsync's optimistic-concurrency pattern.Why
PR #118 (currently in review) elevates the
[Migration(N, journal: false)]+APPLY POLICYreconciliation pattern as one of three first-class temporal scopes for ISM attachment (sample 9001). The natural extension a team takes after copying that sample is to putCREATE POLICYin the samejournal: falsemigration so the policy body itself stays in sync with source-of-truth on every deploy. Without idempotentCREATE POLICY, that pattern fails on the second startup with a confusing 409. This PR closes the gap so the reconciliation pattern is symmetric across attachment AND definition.Tests
OpenSearchTemplatePolicyIntegrationTestswithCreatePolicy_TwiceWithMutatedBody_SecondCallSucceedsViaCasRetry— creates a policy, mutates its description, creates again, asserts the second call succeeds and the post-conditionGETshows the mutated body. Gated by theINTEGRATIONSdefine and runs in the nightly Testcontainers workflow.StatementDispatchermock infrastructure (all dispatch coverage lives in the integration suite); standing up mock plumbing for one method would be net-new work with marginal incremental value over the integration test.dotnet format --verify-no-changesclean.Out of scope
if_seq_no/if_primary_termas a generally-available primitive for other verbs. The doc author flagged this as an optional follow-on; not needed for any current verb. Defer until a second consumer materializes.Test plan
CreatePolicy_TwiceWithMutatedBody_*test.Sequencing
This branches off
main. Lands cleanly after PR #118 merges (no file conflicts — different files). The PR description references samples and README sections from #118; if this lands first, those references will be slightly forward-looking until #118 follows.