fix: properly handle temperature deprecation for models that don't support it#76
Open
agent932 wants to merge 1 commit into
Open
fix: properly handle temperature deprecation for models that don't support it#76agent932 wants to merge 1 commit into
agent932 wants to merge 1 commit into
Conversation
…pport it Three related fixes in providers.ts: 1. Add claude-opus-4-7 and claude-opus-4-8 to FIXED_TEMPERATURE_MODELS so temperature is clamped before the request is sent. 2. Broaden isTempUnsupported catch condition to also match Anthropic's error format (status=400, invalid_request_error) which does not set err.code='unsupported_value' — causing the retry to never fire. 3. Fix buildCompletionConfig to accept temperature as number | undefined and omit the parameter entirely when undefined, rather than retrying with temperature=1 which is also rejected by these models. Update tryCompletion and runStream signatures accordingly. Fixes inference failures when using claude-opus-4-7 or claude-opus-4-8.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When claude-opus-4-7 or claude-opus-4-8 is selected, inference fails with:
The existing retry logic checks for
err.code === 'unsupported_value'but Anthropic returnsinvalid_request_errorwith nocodefield, so the retry never fires. Even if it did, retrying withtemperature=1is also rejected by these models.Fix
claude-opus-4-7andclaude-opus-4-8toFIXED_TEMPERATURE_MODELSas a first-pass guard.isTempUnsupportedin both catch blocks to match Anthropic's error format.buildCompletionConfigto accepttemperature | undefinedand omit the field entirely when undefined. Retry now passesundefinedinstead of1.Tested
Verified on Linux Docker deployment with claude-opus-4-7.