Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for o3-mini reasoning models #75

Closed
reinoldus opened this issue Feb 2, 2025 · 0 comments · Fixed by #86
Closed

Support for o3-mini reasoning models #75

reinoldus opened this issue Feb 2, 2025 · 0 comments · Fixed by #86

Comments

@reinoldus
Copy link
Contributor

reinoldus commented Feb 2, 2025

Hi all,

I wanted to try out o3-mini as a model, but the parameters have changed it a bit on openai's side, instructor supports them, but currently the library doesn't give much flexibility in terms of what parameters we would like to send to the model.

Here is the problematic code, max_tokens and temperature are not supported by o3-mini, but there are other parameters instead "max_completion_tokens" and "reasoning_effort="high""

        response = self.client.chat.completions.create(
            messages=messages,
            model=self.model,
            response_model=response_model,
            temperature=self.temperature,
            max_tokens=self.max_tokens,
        )

We could simple add these to the "BaseAgentConfig", but that would require a code change every time some parameters change on the api.

A better solution might be to add an dict to the BaseAgentConfig "model_api_parameters" or so and just **kwargs them into instructor. This would be a breaking change though unless we leave temperature and max_tokens untouched.

class BaseAgentConfig(BaseModel):
    client: instructor.client.Instructor = Field(..., description="Client for interacting with the language model.")
    model: str = Field("gpt-4o-mini", description="The model to use for generating responses.")
    memory: Optional[AgentMemory] = Field(None, description="Memory component for storing chat history.")
    system_prompt_generator: Optional[SystemPromptGenerator] = Field(
        None, description="Component for generating system prompts."
    )
    input_schema: Optional[Type[BaseModel]] = Field(None, description="The schema for the input data.")
    output_schema: Optional[Type[BaseModel]] = Field(None, description="The schema for the output data.")
    model_config = {"arbitrary_types_allowed": True}
    temperature: Optional[float] = Field(
        0,
        description="Temperature for response generation, typically ranging from 0 to 1.",
    )
    max_tokens: Optional[int] = Field(
        None,
        description="Maximum number of token allowed in the response generation.",
    )
    model_api_parameters: Optional[dict] = Field(None, description="Pass arbitrary parameters to your api provider")

mbcrawfo pushed a commit to mbcrawfo/atomic-agents that referenced this issue Feb 10, 2025
mbcrawfo pushed a commit to mbcrawfo/atomic-agents that referenced this issue Feb 10, 2025
mbcrawfo pushed a commit to mbcrawfo/atomic-agents that referenced this issue Feb 10, 2025
mbcrawfo pushed a commit to mbcrawfo/atomic-agents that referenced this issue Feb 10, 2025
mbcrawfo added a commit to mbcrawfo/atomic-agents that referenced this issue Feb 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant