Skip to content

add max_tokens arg to server method#449

Open
grossjct wants to merge 1 commit intoallenai:mainfrom
grossjct:fix/issue-446
Open

add max_tokens arg to server method#449
grossjct wants to merge 1 commit intoallenai:mainfrom
grossjct:fix/issue-446

Conversation

@grossjct
Copy link
Copy Markdown

@grossjct grossjct commented Apr 1, 2026

Closes #446

Changes proposed in this pull request:

  • add a max_tokens argument to the server method (run_server.py) instead of using a fixed value

Before submitting

  • I've read and followed all steps in the Making a pull request
    section of the CONTRIBUTING docs.
  • I've updated or added any relevant docstrings following the syntax described in the
    Writing docstrings section of the CONTRIBUTING docs.
  • If this PR fixes a bug, I've added a test that will fail without my fix.
  • If this PR adds a new feature, I've added tests that sufficiently cover my new functionality.

Copilot AI review requested due to automatic review settings April 1, 2026 15:27
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses #446 by making the OpenAI-compatible server runner configurable for output length, replacing a hard-coded max_tokens value with a function argument.

Changes:

  • Added a max_tokens parameter to run_server(...) with a default of 8000.
  • Wired the request payload to use the provided max_tokens value instead of a fixed constant.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@grossjct
Copy link
Copy Markdown
Author

grossjct commented Apr 8, 2026

@jakep-allenai any comment on this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

max_tokens argument for server method instead of fixed value

2 participants