Skip to main content

​
Scorers

​
get_scorer_version

def get_scorer_version(self,
                       scorer_id: UUID,
                       version: int) -> Union[Unset, BaseScorerVersionResponse]
Arguments
  • name (str): Name of the scorer
  • version (int): Version of the scorer.
Returns
  • Scorer response if found, otherwise None.:

​
list

def list(self,
         name: Optional[str]=None,
         types: Optional[list[ScorerTypes]]=None) -> list[ScorerResponse]
Lists scorers, optionally filtering by name and/or type. The filters are combined with an AND condition, while the values within the types list are combined with an OR condition. For example, calling list(name="my_scorer", types=[ScorerTypes.llm, ScorerTypes.code]) will return scorers where the name is “my_scorer” AND the type is either “llm” OR “code”. Arguments
  • name: Name of the scorer to filter by.
  • types: List of scorer types to filter by. Defaults to all scorers.
Returns
  • A list of scorers that match the filter criteria.:

​
ScorerSettings

​
create

def create(self,
           project_id: str,
           run_id: str,
           scorers: list[ScorerConfig]) -> Optional[Union[HTTPValidationError, RunScorerSettingsResponse]]
Arguments
  • project_id: ID of the project
  • run_id: ID of the run
  • scorers: List of scorer configurations
Returns
  • Upserted scorer settings.:
⌘I