POST
/
v2
/
scorers
/
{scorer_id}
/
version
/
preset
Create Preset Scorer Version
curl --request POST \
  --url https://api.galileo.ai/v2/scorers/{scorer_id}/version/preset \
  --header 'Content-Type: application/json' \
  --header 'Galileo-API-Key: <api-key>' \
  --data '{
  "model_name": "<string>",
  "num_judges": 123,
  "scoreable_node_types": [
    "<string>"
  ],
  "cot_enabled": true,
  "output_type": "boolean",
  "input_type": "basic"
}'
{
  "id": "<string>",
  "version": 123,
  "generated_scorer": {
    "id": "<string>",
    "name": "<string>",
    "instructions": "<string>",
    "chain_poll_template": {
      "metric_system_prompt": "<string>",
      "metric_description": "<string>",
      "value_field_name": "<string>",
      "explanation_field_name": "<string>",
      "template": "<string>",
      "metric_few_shot_examples": [
        {
          "generation_prompt_and_response": "<string>",
          "evaluating_response": "<string>"
        }
      ]
    },
    "user_prompt": "<string>",
    "created_by": "<string>",
    "created_at": "2023-11-07T05:31:56Z",
    "updated_at": "2023-11-07T05:31:56Z",
    "scoreable_node_types": [
      "chain"
    ],
    "scorer_configuration": {
      "model_alias": "gpt-4.1-mini",
      "num_judges": 3,
      "output_type": "boolean",
      "scoreable_node_types": [
        "<string>"
      ],
      "cot_enabled": false,
      "ground_truth": false
    }
  },
  "registered_scorer": {
    "id": "<string>",
    "name": "<string>",
    "created_at": "2023-11-07T05:31:56Z",
    "updated_at": "2023-11-07T05:31:56Z",
    "created_by": "<string>",
    "score_type": "<string>",
    "data_type": "unknown",
    "scoreable_node_types": [
      "<string>"
    ]
  },
  "finetuned_scorer": {
    "id": "<string>",
    "name": "<string>",
    "lora_task_id": 123,
    "prompt": "<string>",
    "executor": "action_completion_luna",
    "created_at": "2023-11-07T05:31:56Z",
    "updated_at": "2023-11-07T05:31:56Z",
    "created_by": "<string>"
  },
  "model_name": "<string>",
  "num_judges": 123,
  "scoreable_node_types": [
    "<string>"
  ],
  "cot_enabled": true,
  "output_type": "boolean",
  "input_type": "basic",
  "created_at": "2023-11-07T05:31:56Z",
  "updated_at": "2023-11-07T05:31:56Z",
  "chain_poll_template": {
    "metric_system_prompt": "<string>",
    "metric_description": "<string>",
    "value_field_name": "<string>",
    "explanation_field_name": "<string>",
    "template": "<string>",
    "metric_few_shot_examples": [
      {
        "generation_prompt_and_response": "<string>",
        "evaluating_response": "<string>"
      }
    ]
  }
}

Authorizations

​
Galileo-API-Key
string
header
required

Path Parameters

​
scorer_id
string<uuid4>
required

Body

application/json
​
model_name
string | null
​
num_judges
integer | null
​
scoreable_node_types
string[] | null
​
cot_enabled
boolean | null
​
output_type
enum<string> | null

Enumeration of output types.

Available options:
boolean,
categorical,
count,
discrete,
freeform,
percentage,
multilabel
​
input_type
enum<string> | null

Enumeration of input types.

Available options:
basic,
llm_spans,
retriever_spans,
sessions_normalized,
sessions_trace_io_only,
tool_spans,
trace_input_only,
trace_io_only,
trace_normalized,
trace_output_only

Response

Successful Response

​
id
string<uuid4>
required
​
version
integer
required
​
created_at
string<date-time>
required
​
updated_at
string<date-time>
required
​
generated_scorer
object | null
​
registered_scorer
object | null
​
finetuned_scorer
object | null
​
model_name
string | null
​
num_judges
integer | null
​
scoreable_node_types
string[] | null
​
cot_enabled
boolean | null
​
output_type
enum<string> | null

Enumeration of output types.

Available options:
boolean,
categorical,
count,
discrete,
freeform,
percentage,
multilabel
​
input_type
enum<string> | null

What type of input to use for model-based scorers (sessions_normalized, trace_io_only, etc.). Enumeration of input types.

Available options:
basic,
llm_spans,
retriever_spans,
sessions_normalized,
sessions_trace_io_only,
tool_spans,
trace_input_only,
trace_io_only,
trace_normalized,
trace_output_only
​
chain_poll_template
object | null

Template for a chainpoll metric prompt, containing all the info necessary to send a chainpoll prompt.