Function: createScorer()
Parameters
name
string
The name of the scorer.
scorerType
ScorerTypes
The type of the scorer.
description?
string
(Optional) A description for the scorer.
tags?
string[]
(Optional) Tags to associate with the scorer.
defaults?
(Optional) Default settings for the scorer. Required for LLM scorers.cot_enabled?
null | boolean
Cot Enabled
Description
Whether to enable chain of thought for this scorer. Defaults to False for llm scorers.
filters?
|null
| (
| {
case_sensitive?: boolean;
filter_type?: "string";
name: "node_name";
operator: "eq" | "ne" | "contains";
value: string;
}
| {
filter_type?: "map";
key: string;
name: "metadata";
operator: "eq" | "ne" | "not_in" | "one_of";
value: string | string[];
})[]
Filters
Description
List of filters to apply to the scorer.
input_type?
|null
| "basic"
| "llm_spans"
| "retriever_spans"
| "sessions_normalized"
| "sessions_trace_io_only"
| "tool_spans"
| "trace_input_only"
| "trace_io_only"
| "trace_normalized"
| "trace_output_only"
| "agent_spans"
| "workflow_spans"
Description
What type of input to use for model-based scorers (sessions_normalized, trace_io_only, etc..).
model_name?
null | string
Model Name
num_judges?
null | number
Num Judges
output_type?
|null
| "boolean"
| "categorical"
| "count"
| "discrete"
| "freeform"
| "percentage"
| "multilabel"
Description
What type of output to use for model-based scorers (boolean, categorical, etc.).
scoreable_node_types?
null | string[]
Scoreable Node Types
Description
List of node types that can be scored by this scorer. Defaults to llm/chat.
modelType?
(Optional) The model type for the scorer."llm" | "code" | "slm"
defaultVersionId?
string
(Optional) The default version ID for the scorer.
scoreableNodeTypes?
("agent" | "llm" | "retriever" | "tool" | "workflow" | "trace" | "session")[]
(Optional) The node types that can be scored.
outputType?
OutputType
(Optional) The output type for the scorer.
inputType?
InputType
(Optional) The input type for the scorer.
Returns
Promise<ScorerResponse>
A promise that resolves to the created scorer.