function runExperiment<T>( params: RunExperimentParams<T>,): Promise<RunExperimentOutput>;
Defined in: src/utils/experiments.ts:268Runs an experiment by processing each row of a dataset through a specified function.
If metrics are provided, they will be used to evaluate the experiment.Usage:
Copy
Ask AI
// Run an experiment with a runner functionconst results = await runExperiment({ name: 'my-experiment', dataset: [{ country: 'France'}], function: async (input) => { const response = await openai.chat.completions.create({ model: 'gpt-4o-mini', messages: [ { role: 'user', content: `What is the capital of ${input['country']}?` } ] }); return response.choices[0].message.content; }, metrics: ['accuracy'], projectName: 'my-project'});// Run an experiment with a prompt templateconst promptTemplate = await createPromptTemplate({ template: [{ role: 'user', content: 'What is the capital of {{ country }}?' }], name: 'my-prompt-template', projectName: 'my-project'});const results = await runExperiment({ name: 'my-experiment', dataset: [{ country: 'France' }], promptTemplate metrics: ['accuracy'], projectName: 'my-project'});