Skip to content

FIX: Validate presence_penalty and frequency_penalty in OpenAI and LiteLLM prompt targets - #3530

Open
Nithin (nithin42) wants to merge 1 commit into
microsoft:mainfrom
nithin42:fix/validate-penalty-parameters
Open

Nithin (nithin42) wants to merge 1 commit into
microsoft:mainfrom
nithin42:fix/validate-penalty-parameters

Conversation

@nithin42

Copy link
Copy Markdown

Description

Fixes #3498

OpenAIChatTarget, OpenAICompletionTarget, and LiteLLMChatTarget document presence_penalty and frequency_penalty as taking values between -2.0 and 2.0, but previously neither class validated these parameters in init. Out-of-range values (< -2.0 or > 2.0) were stored in target identifiers and would fail downstream during API requests.

This change:

  1. Adds validate_presence_penalty and validate_frequency_penalty helpers in pyrit.prompt_target.common.utils (raising PyritException if the value is not None and outside [-2.0, 2.0]).
  2. Calls both validation helpers during init in OpenAIChatTarget, OpenAICompletionTarget, and LiteLLMChatTarget.

Tests and Documentation

  • Added unit tests for validate_presence_penalty and validate_frequency_penalty in tests/unit/prompt_target/test_target_utils.py covering valid bounds (-2.0, 0.0, 2.0, None) and out-of-bounds inputs.
  • Added parameter rejection unit tests in:
    • tests/unit/prompt_target/target/test_openai_chat_target.py
    • tests/unit/prompt_target/target/test_azure_openai_completion_target.py
    • tests/unit/prompt_target/target/test_litellm_chat_target.py
  • Ran targeted pytest suite (258 passed).
  • Verified code formatting with ruff check and ruff format --check.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BUG: OpenAIChatTarget, OpenAICompletionTarget, and LiteLLMChatTarget do not validate presence_penalty or frequency_penalty

1 participant