Overview
Getting Started
SDK/API
- Python SDK Reference
- TypeScript SDK Reference
- Overview
- Wrappers
- Core Logging
- Datasets
- Experiments
- Prompts
- Typescript SDK Reference
How-to Guides
- Overview
- Conversational AI
- Retrieval-Augmented Generation
- Agentic AI
Cookbooks
- Use Cases
- Features
Integrations
Concepts
- Metrics
- Projects
- Logging
- Datasets
- Annotations
- Experiments
- Playground
API Reference
- health
- auth
- trace
- log_stream
- experiment
- feedback
- annotation
References
TypeScript SDK Reference
Datasets
Galileo Datasets allow you to create, manage, and version collections of data for testing and evaluating your LLM applications.
Creating and Using Datasets
You can create and use datasets for experimentation:
import { createDataset, getDataset } from "galileo";
// Create a dataset from a file path
const datasetFromFile = await createDataset("/path/to/dataset.csv", "my-dataset-name");
// Create a dataset from an array of objects
const datasetFromArray = await createDataset(
[
{ input: "Spain", output: "Europe" },
{ input: "Japan", output: "Asia" },
],
"countries",
);
// Create a dataset from a dictionary of arrays
const datasetFromDict = await createDataset(
{
input: ["Spain", "Japan"],
output: ["Europe", "Asia"],
},
"countries-from-dict",
);
// Get a dataset by name
const dataset = await getDataset({ name: "countries" });
Using Datasets in Experiments
Datasets can be used in experiments to evaluate your LLM applications:
import { runExperiment } from "galileo";
import { OpenAI } from "openai";
async function runDatasetExperiment() {
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
const runner = async (input: any) => {
const result = await openai.chat.completions.create({
model: "gpt-4o",
messages: [{ content: `Which continent does the following country belong to ${input["input"]}!`, role: "user" }],
});
return result;
};
await runExperiment({
name: `Test Experiment`,
datasetName: "countries",
function: runner,
metrics: ["correctness"],
projectName: "my-project",
});
}
// Run the experiment
runDatasetExperiment();
Using Custom Datasets
You can also use locally generated datasets for experiments:
import { runExperiment } from "galileo";
import { OpenAI } from "openai";
async function runCustomDatasetExperiment() {
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
const dataset = [
{
input: "Spain",
output: "Europe",
},
{
input: "Japan",
output: "Asia",
},
];
const runner = async (input: any) => {
const result = await openai.chat.completions.create({
model: "gpt-4o",
messages: [{ content: `Which continent does the following country belong to ${input["input"]}!`, role: "user" }],
});
return result;
};
await runExperiment({
name: `Test Experiment`,
dataset: dataset,
function: runner,
metrics: ["tox"],
projectName: "my-project",
});
}
// Run the experiment
runCustomDatasetExperiment();
Assistant
Responses are generated using AI and may contain mistakes.