feat(hub): PLA-100: add model selection for all agents

[KevinWu06]

Summary

This change is to use models that the user passes in for the OpenAI Realtime, LLM Agent, and the NLU Agent.

https://linear.app/arklex/issue/PLA-100/hub-add-model-selection-in-flow-page

Description

Before, we hardcoded the models that we used. This change allows for the user to optionally pass in the model that they want to use. If no model is passed in, then we default to a pre-set model config (gpt-5.1 for rule-based agent and LLM Agent, gpt-realtime-1.5 for OpenAIRealtime Agent). Also fixes an issue where the max response length was being ignored for the rule-based agent.

Tests

• Pre-commit code format check: Run pip install pre-commit and pre-commit install before committing changes
• Attach one of run-coverage-tests label, run-diff-coverage-tests label, or run-integration-tests label (to skip test coverage) and ensure the check passes.
• Test case 1: Tested OpenAI realtime, updated model. Retrieved the OpenAI API request id for the interaction to verify the model being used. Verified that the model was what I had set it to.
• Test case 2: Tested LLM Agent, updated model. Retrieved the OpenAI API request id for the interaction to verify the model being used. Verified that the model was what I had set it to.
• Test case 3: Tested NLU agent, updated model. Retrieved the OpenAI API request id for the interaction to verify the model being used. Verified that the model was what I had set it to.

Reviewers

@arklexai/agent-leads

[Copilot]

Pull request overview

Adds end-user model override support across Arklex agent types (rule-based NLU, OpenAI Agent SDK LLM agents, and OpenAI Realtime voice agents), replacing previously hardcoded model choices and ensuring the rule-based agent’s response-length instruction is actually applied to the system prompt.

Changes:

• Add optional model fields to agent node data models and propagate per-agent model selection in AgentGraph.
• Add NLU graph support for swapping LLMConfig at runtime when an NLU agent specifies a model override.
• Allow OpenAI Realtime voice agent sessions to use a configurable model name instead of a hardcoded one.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
arklex/resources/agents/rule_based_agent/nlu_agent.py	Adds optional model override and applies format_system_prompt() as sys_instruct.
arklex/resources/agents/realtime_voice_agent/openai_realtime_agent.py	Adds configurable realtime session model name and threads it into session settings.
arklex/resources/agents/llm_based_agent/openai_agent.py	Extends agent config schema with optional per-agent model override.
arklex/orchestrator/task_graph/nlu_graph.py	Adds update_llm_config() to rebuild the intent detector with a new LLMConfig.
arklex/orchestrator/task_graph/agent_graph.py	Uses per-agent model overrides for OpenAI Agent SDK agents and passes model into the realtime start agent.

Comments suppressed due to low confidence (2)

arklex/resources/agents/realtime_voice_agent/openai_realtime_agent.py:205

• connect() forwards self.model into RealtimeSessionModelSettings.model_name. If self.model is None or an empty string (possible if passed through from config), session setup will likely fail. Add a defensive fallback/validation before constructing RealtimeSessionModelSettings.

        runner = RealtimeRunner(
            starting_agent=self.realtime_agent,
            config=RealtimeRunConfig(
                model_settings=RealtimeSessionModelSettings(
                    model_name=self.model,
                    input_audio_format=self.input_audio_format,
                    output_audio_format=self.output_audio_format,
                    input_audio_transcription=RealtimeInputAudioTranscriptionConfig(
                        model="gpt-4o-transcribe",
                        language=self.transcription_language,

arklex/resources/agents/realtime_voice_agent/openai_realtime_agent.py:123

• model was added as a constructor argument, but it isn't documented in the Args: section of the docstring. Please update the docstring so callers know how to override the default realtime model.

        """
        Initialize the OpenAI Realtime Agent.

        Args:
            realtime_agent: The RealtimeAgent instance to use for the session
            telephony_mode: Whether the agent is in telephone mode (uses g711_ulaw audio format)
            voice: The voice for the agent (default: "alloy")
            transcription_language: The language for the transcription (optional)
            speed: The speech speed for the agent (default: 1.0)
            turn_detection: The turn detection configuration for the agent
            voicemail_tool: Optional tool to execute when a call goes to voicemail. This is triggered by
            twilio AMD.
        """

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[itsarbit]

The per-agent model selection approach is right. Two things:

1. There's a hardcoded "gpt-realtime-1.5" in agent_graph.py line 381. The default should live in OpenAIRealtimeAgentData (which already has model: str | None = None), not in the orchestrator. The graph should just pass start_agent_data.model and let the agent constructor handle the default.

2. update_llm_config is added to NLUGraph but only called once at construction in NLUAgent.init. A model_copy in init before constructing NLUGraph would eliminate the setter entirely with no behavior change. Simpler is better here.

Everything else is clean. The model field additions are consistent across all agent data models and the sys_instruct fix for the rule-based agent is a good catch.

[KevinWu06]

The per-agent model selection approach is right. Two things:

1. There's a hardcoded "gpt-realtime-1.5" in agent_graph.py line 381. The default should live in OpenAIRealtimeAgentData (which already has model: str | None = None), not in the orchestrator. The graph should just pass start_agent_data.model and let the agent constructor handle the default.
2. update_llm_config is added to NLUGraph but only called once at construction in NLUAgent.init. A model_copy in init before constructing NLUGraph would eliminate the setter entirely with no behavior change. Simpler is better here.

Everything else is clean. The model field additions are consistent across all agent data models and the sys_instruct fix for the rule-based agent is a good catch.

I updated the PR with the requested changes