Skip to content

feat: add thinking mode to claude4#243

Merged
Lavriz merged 3 commits into
mainfrom
add-thinking
May 23, 2025
Merged

feat: add thinking mode to claude4#243
Lavriz merged 3 commits into
mainfrom
add-thinking

Conversation

@Lavriz
Copy link
Copy Markdown
Contributor

@Lavriz Lavriz commented May 23, 2025

@1712n/dni-nlp-backend do you want to have separate claude4 models with and without thinking? what i added now always adds this thinking mode to the models

@Lavriz Lavriz self-assigned this May 23, 2025
@Lavriz
Copy link
Copy Markdown
Contributor Author

Lavriz commented May 23, 2025

@claude review this PR

@claude
Copy link
Copy Markdown
Contributor

claude Bot commented May 23, 2025

Claude finished @Lavriz's task —— View job


Code Review Analysis

  • Read and analyze the changed file
  • Review code quality and implementation
  • Provide feedback on architectural decision
  • Post comprehensive review

Review Findings

Code Quality & Implementation

The implementation looks clean and follows good practices:

  • Clear conditional logic: The thinking mode is properly scoped to only Claude 4 models (src/providers/anthropic.ts:55-62)
  • Appropriate configuration: The thinking setup includes reasonable defaults:
    • type: 'enabled'
    • budget_tokens: 32_000 (appropriate for the increased max_tokens: 128_000)
    • temperature: 1 (required constraint when thinking is enabled)

🤔 Architectural Decision: Separate Models vs Always-On

Regarding your question about having separate claude4 models with/without thinking:

Current approach (always-on) is recommended for these reasons:

  1. User Experience: Users get the benefit of thinking mode automatically without needing to choose between model variants
  2. Simplicity: Avoids model proliferation and reduces configuration complexity
  3. Performance: Thinking mode generally improves response quality with minimal downsides
  4. Consistency: Matches how other providers handle similar features (e.g., Gemini's thinking models)

Alternative approach (separate models) would only be beneficial if:

  • There are significant latency concerns (thinking adds processing time)
  • Token budget is very limited (thinking consumes budget tokens)
  • You want to offer a "faster" option for simple queries

📝 Minor Suggestions

  1. Documentation: Consider adding a comment explaining why temperature is forced to 1:

    query.temperature = 1; // Required by Anthropic API when thinking is enabled
  2. Future consideration: If you later want both options, you could add model variants like:

    • claude-sonnet-4-20250514 (with thinking, current)
    • claude-sonnet-4-20250514-fast (without thinking)

Verdict

The current implementation is solid. Recommend merging - the always-on thinking approach provides better user experience without significant drawbacks.

@evgenydmitriev
Copy link
Copy Markdown
Contributor

@claude enable thinking mode for Claude_4_Opus model by exposing another model called Claude_4_Opus_thinking to the user. Make sure the regular Claude_4_Opus model is still available. Update the documentation.

@claude
Copy link
Copy Markdown
Contributor

claude Bot commented May 23, 2025

Claude finished @evgenydmitriev's task —— View job


Task: Enable thinking mode for Claude_4_Opus with separate model variants

  • Read current implementation in src/providers/anthropic.ts
  • Check existing documentation and model listings
  • Modify provider to expose both Claude_4_Opus and Claude_4_Opus_thinking
  • Update documentation to reflect new model
  • Test implementation and commit changes

Implementation Complete

Successfully implemented separate Claude 4 Opus model variants as requested:

Changes Made

  1. Added new model: claude-opus-4-20250514-thinking to ModelName enum
  2. Updated provider mapping: Added to Anthropic models list in providers/index.ts
  3. Modified request logic in anthropic.ts:
    • claude-opus-4-20250514-thinking → maps to base model with thinking enabled (128k tokens, temperature=1)
    • claude-opus-4-20250514 → base model without thinking (128k tokens, user temperature)
    • claude-sonnet-4-20250514 → unchanged (still has thinking enabled)
  4. Updated README.md: Added new model to available models list with description
  5. Added test coverage: Test ensures thinking variant properly maps to base model with thinking config

Result

Users can now choose between:

  • claude-opus-4-20250514 - Regular Claude 4 Opus without thinking
  • claude-opus-4-20250514-thinking - Claude 4 Opus with thinking mode enabled

Commit: b5c5e6c


- Add separate Claude_4_Opus_thinking model that enables thinking mode
- Keep regular Claude_4_Opus model without thinking mode  
- Update model mappings and provider configuration
- Add test coverage for thinking model variant
- Update documentation with new model

Co-authored-by: evgenydmitriev <evgenydmitriev@users.noreply.github.com>
@evgenydmitriev
Copy link
Copy Markdown
Contributor

@claude make similar modifications for Claude_4_Sonnet and Claude_4_Sonnet_thinking models

@claude
Copy link
Copy Markdown
Contributor

claude Bot commented May 23, 2025

Claude finished @evgenydmitriev's task —— View job


✅ Task Complete: Claude_4_Sonnet and Claude_4_Sonnet_thinking model variants

Successfully implemented separate Claude 4 Sonnet model variants following the same pattern as Claude_4_Opus models.

Todo List:

  • Analyze current implementation pattern
  • Read existing code to understand the structure
  • Create Claude_4_Sonnet_thinking model enum entry
  • Modify anthropic.ts provider logic for both sonnet variants
  • Update providers/index.ts with new model
  • Update README.md documentation
  • Add test coverage for new model variants
  • Commit all changes

Implementation Complete:

Changes Made:

  1. Added new model: claude-sonnet-4-20250514-thinking to ModelName enum
  2. Updated provider mapping: Added to Anthropic models list in providers/index.ts
  3. Modified request logic in anthropic.ts:
    • claude-sonnet-4-20250514-thinking → maps to base model with thinking enabled (128k tokens, temperature=1)
    • claude-sonnet-4-20250514 → base model without thinking (128k tokens, user temperature)
  4. Updated README.md: Added new model to available models list with description
  5. Added test coverage: Tests ensure both sonnet variants work correctly

Result:
Users can now choose between:

  • claude-sonnet-4-20250514 - Claude 4 Sonnet without thinking
  • claude-sonnet-4-20250514-thinking - Claude 4 Sonnet with thinking mode enabled

Commit: ddba634


- Add claude-sonnet-4-20250514-thinking to ModelName enum
- Update Anthropic provider to support both sonnet variants
- Claude_4_Sonnet (without thinking) preserves user temperature
- Claude_4_Sonnet_thinking enables thinking mode with temperature=1
- Update README.md with new model documentation
- Add comprehensive test coverage for both sonnet variants

Co-authored-by: evgenydmitriev <evgenydmitriev@users.noreply.github.com>
@Lavriz Lavriz merged commit c97eec1 into main May 23, 2025
3 checks passed
@Lavriz Lavriz deleted the add-thinking branch May 23, 2025 07:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants