​

Function: createCustomLlmMetric()

function createCustomLlmMetric(
  name: string,
  instructions: string,
  chainPollTemplate: object,
  modelName: string,
  numJudges: number,
  description: string,
  tags: string[],
): Promise<void>;

Defined in: src/utils/metrics.ts:21

Creates a custom LLM metric.

​

Parameters

​

name

string

The name of the custom metric.

​

instructions

string

Instructions for the LLM scorer version.

​

chainPollTemplate

The chain poll template for the scorer version.

​

explanation_field_name?

string

Explanation Field Name

Description

Field name to look for in the chainpoll response, for the explanation.

Default

explanation;

​

metric_description?

null | string

Metric Description

Description

Description of what the metric should do.

​

metric_few_shot_examples?

object[]

Metric Few Shot Examples

Description

Few-shot examples for the metric.

​

metric_system_prompt?

null | string

Metric System Prompt

Description

System prompt for the metric.

​

template

string

Template

Description

Chainpoll prompt template.

​

value_field_name?

string

Value Field Name

Description

Field name to look for in the chainpoll response, for the rating.

Default

rating;

​

modelName

string = 'GPT-4o'

(Optional) The model name to use. Defaults to ‘GPT-4o’.

​

numJudges

number = 3

(Optional) The number of judges to use. Defaults to 3.

​

description

string = ''

(Optional) A description for the metric.

​

tags

string[] = []

(Optional) Tags to associate with the metric.

​

Returns

Promise<void>

A promise that resolves when the metric is created.