ℹ️ These docs are for the v2.0 version of Galileo. Documentation for v1.0 version can be found here.
Python SDK Reference
TypeScript SDK Reference
API Reference
- POSTLog Traces
- GETGet Trace
- PATCHUpdate Trace
- GETGet Span
- PATCHUpdate Span
- POSTTraces Available Columns
- POSTMetrics Testing Available Columns
- POSTSpans Available Columns
- POSTQuery Traces
- POSTCount Traces
- POSTLog Spans
- POSTQuery Spans
- POSTCount Spans
- POSTQuery Metrics
- POSTCreate Session
- POSTQuery Sessions
- POSTCount Sessions
- GETGet Session
- POSTSessions Available Columns
- POSTGet Aggregated Trace View
- POSTExport Records
- POSTRecompute Metrics
- POSTDelete Traces
- POSTDelete Spans
- POSTDelete Sessions
- POST
curl --request POST \
--url https://api.galileo.ai/v2/projects/{project_id}/sessions/available_columns \
--header 'Content-Type: application/json' \
--header 'Galileo-API-Key: <api-key>' \
--data '
{
"log_stream_id": "<string>",
"experiment_id": "<string>",
"metrics_testing_id": "<string>",
"start_time": "2023-11-07T05:31:56Z",
"end_time": "2023-11-07T05:31:56Z"
}
'{
"columns": [
{
"id": "<string>",
"category": "standard",
"scorer_config": {
"id": "<string>",
"scorer_type": "llm",
"model_name": "<string>",
"num_judges": 123,
"filters": [
{
"value": "<string>",
"operator": "eq",
"name": "node_name",
"filter_type": "string",
"case_sensitive": true
}
],
"scoreable_node_types": [
"<string>"
],
"cot_enabled": true,
"output_type": "boolean",
"input_type": "basic",
"name": "<string>",
"model_type": "slm",
"scorer_version": {
"id": "<string>",
"version": 123,
"scorer_id": "<string>",
"generated_scorer": {
"id": "<string>",
"name": "<string>",
"chain_poll_template": {
"template": "<string>",
"metric_system_prompt": "<string>",
"metric_description": "<string>",
"value_field_name": "rating",
"explanation_field_name": "explanation",
"metric_few_shot_examples": [
{
"generation_prompt_and_response": "<string>",
"evaluating_response": "<string>"
}
],
"response_schema": {}
},
"instructions": "<string>",
"user_prompt": "<string>"
},
"registered_scorer": {
"id": "<string>",
"name": "<string>",
"score_type": "<string>"
},
"finetuned_scorer": {
"id": "<string>",
"name": "<string>",
"lora_task_id": 123,
"prompt": "<string>",
"luna_input_type": "span",
"luna_output_type": "float",
"class_name_to_vocab_ix": {},
"executor": "action_completion_luna"
},
"model_name": "<string>",
"num_judges": 123,
"scoreable_node_types": [
"<string>"
],
"cot_enabled": true,
"output_type": "boolean",
"input_type": "basic"
}
},
"scorer_id": "<string>",
"label": "<string>",
"description": "<string>",
"group_label": "<string>",
"insight_type": "vertical_bar",
"data_type": "uuid",
"data_unit": "percentage",
"multi_valued": false,
"allowed_values": [
"<unknown>"
],
"threshold": {
"inverted": false,
"buckets": [
123
],
"display_value_levels": [
"<string>"
]
},
"sortable": true,
"filterable": true,
"is_empty": false,
"applicable_types": [
"llm"
],
"complex": false,
"is_optional": false
}
]
}Sessions Available Columns
curl --request POST \
--url https://api.galileo.ai/v2/projects/{project_id}/sessions/available_columns \
--header 'Content-Type: application/json' \
--header 'Galileo-API-Key: <api-key>' \
--data '
{
"log_stream_id": "<string>",
"experiment_id": "<string>",
"metrics_testing_id": "<string>",
"start_time": "2023-11-07T05:31:56Z",
"end_time": "2023-11-07T05:31:56Z"
}
'{
"columns": [
{
"id": "<string>",
"category": "standard",
"scorer_config": {
"id": "<string>",
"scorer_type": "llm",
"model_name": "<string>",
"num_judges": 123,
"filters": [
{
"value": "<string>",
"operator": "eq",
"name": "node_name",
"filter_type": "string",
"case_sensitive": true
}
],
"scoreable_node_types": [
"<string>"
],
"cot_enabled": true,
"output_type": "boolean",
"input_type": "basic",
"name": "<string>",
"model_type": "slm",
"scorer_version": {
"id": "<string>",
"version": 123,
"scorer_id": "<string>",
"generated_scorer": {
"id": "<string>",
"name": "<string>",
"chain_poll_template": {
"template": "<string>",
"metric_system_prompt": "<string>",
"metric_description": "<string>",
"value_field_name": "rating",
"explanation_field_name": "explanation",
"metric_few_shot_examples": [
{
"generation_prompt_and_response": "<string>",
"evaluating_response": "<string>"
}
],
"response_schema": {}
},
"instructions": "<string>",
"user_prompt": "<string>"
},
"registered_scorer": {
"id": "<string>",
"name": "<string>",
"score_type": "<string>"
},
"finetuned_scorer": {
"id": "<string>",
"name": "<string>",
"lora_task_id": 123,
"prompt": "<string>",
"luna_input_type": "span",
"luna_output_type": "float",
"class_name_to_vocab_ix": {},
"executor": "action_completion_luna"
},
"model_name": "<string>",
"num_judges": 123,
"scoreable_node_types": [
"<string>"
],
"cot_enabled": true,
"output_type": "boolean",
"input_type": "basic"
}
},
"scorer_id": "<string>",
"label": "<string>",
"description": "<string>",
"group_label": "<string>",
"insight_type": "vertical_bar",
"data_type": "uuid",
"data_unit": "percentage",
"multi_valued": false,
"allowed_values": [
"<unknown>"
],
"threshold": {
"inverted": false,
"buckets": [
123
],
"display_value_levels": [
"<string>"
]
},
"sortable": true,
"filterable": true,
"is_empty": false,
"applicable_types": [
"llm"
],
"complex": false,
"is_optional": false
}
]
}Authorizations
Path Parameters
Body
Log stream id associated with the traces.
Experiment id associated with the traces.
Metrics testing id associated with the traces.
Response
Successful Response
Show child attributes
Show child attributes
Column id. Must be universally unique.
Category of the column.
standard, metric, user_metadata, dataset_metadata, dataset, feedback, tags For metric columns only: Scorer config that produced the metric.
Show child attributes
Show child attributes
llm, code, luna, preset List of filters to apply to the scorer.
Filters on node names in scorer jobs.
- NodeNameFilter
- MetadataFilter
Show child attributes
Show child attributes
eq, ne, contains "node_name""string"List of node types that can be scored by this scorer. Defaults to llm/chat.
Whether to enable chain of thought for this scorer. Defaults to False for llm scorers.
What type of output to use for model-based scorers (boolean, categorical, etc.).
boolean, categorical, count, discrete, freeform, percentage, multilabel What type of input to use for model-based scorers (sessions_normalized, trace_io_only, etc..).
basic, llm_spans, retriever_spans, sessions_normalized, sessions_trace_io_only, tool_spans, trace_input_only, trace_io_only, trace_normalized, trace_output_only, agent_spans, workflow_spans Type of model to use for this scorer. slm maps to luna, and llm maps to plus
slm, llm, code ScorerVersion to use for this scorer. If not provided, the latest version will be used.
Show child attributes
Show child attributes
Show child attributes
Show child attributes
Template for a chainpoll metric prompt, containing all the info necessary to send a chainpoll prompt.
Show child attributes
Show child attributes
Chainpoll prompt template.
System prompt for the metric.
Description of what the metric should do.
Field name to look for in the chainpoll response, for the rating.
Field name to look for in the chainpoll response, for the explanation.
Few-shot examples for the metric.
Response schema for the output
Show child attributes
Show child attributes
span, trace_object, trace_input_output_only float, string, string_list Executor pipeline. Defaults to finetuned scorer pipeline but can run custom galileo score pipelines.
action_completion_luna, action_advancement_luna, agentic_session_success, agentic_session_success, agentic_workflow_success, agentic_workflow_success, agent_efficiency, agent_flow, bleu, chunk_attribution_utilization_luna, chunk_attribution_utilization, completeness_luna, completeness, context_adherence, context_adherence_luna, context_relevance, context_relevance_luna, conversation_quality, correctness, ground_truth_adherence, input_pii, input_pii_gpt, input_sexist, input_sexist, input_sexist_luna, input_sexist_luna, input_tone, input_tone_gpt, input_toxicity, input_toxicity_luna, instruction_adherence, output_pii, output_pii_gpt, output_sexist, output_sexist, output_sexist_luna, output_sexist_luna, output_tone, output_tone_gpt, output_toxicity, output_toxicity_luna, prompt_injection, prompt_injection_luna, prompt_perplexity, rouge, tool_error_rate, tool_error_rate_luna, tool_selection_quality, tool_selection_quality_luna, uncertainty, user_intent_change List of node types that can be scored by this scorer. Defaults to llm/chat.
Whether to enable chain of thought for this scorer. Defaults to False for llm scorers.
What type of output to use for model-based scorers (sessions_normalized, trace_io_only, etc.).
boolean, categorical, count, discrete, freeform, percentage, multilabel What type of input to use for model-based scorers (sessions_normalized, trace_io_only, etc.).
basic, llm_spans, retriever_spans, sessions_normalized, sessions_trace_io_only, tool_spans, trace_input_only, trace_io_only, trace_normalized, trace_output_only, agent_spans, workflow_spans For metric columns only: Scorer id that produced the metric. This is deprecated and will be removed in future versions.
Display label of the column in the UI.
Description of the column.
Display label of the column group.
Insight type.
vertical_bar, horizontal_bar Data type of the column. This is used to determine how to format the data on the UI.
uuid, text, integer, floating_point, boolean, timestamp, string_list, tag, dataset, prompt, playground, rank Data unit of the column (optional).
percentage, nano_seconds, milli_seconds, dollars, count_and_total Whether the column is multi-valued.
Allowed values for this column.
Thresholds for the column, if this is a metrics column.
Show child attributes
Show child attributes
Whether the column should be inverted for thresholds, i.e. if True, lower is better.
Threshold buckets for the column. If the column is a metric, these are the thresholds for the column.
Ordered list of strings that raw values get transformed to for displaying.
Whether the column is sortable.
Whether the column is filterable.
Indicates whether the column is empty and should be hidden.
List of types applicable for this column.
llm, retriever, tool, workflow, agent, trace, session Whether the column requires special handling in the UI. Setting this to True will hide the column in the UI until the UI adds support for it.
Whether the column is optional.
Was this page helpful?