Datasets allow you to store and reuse well-defined data for use in experiments. Datasets can be stored and versioned in Galileo, and available for experiments running both in the console as well as in code.

​
Dataset fields

Each record in a Galileo dataset can have three top-level fields:
  1. input - Input variables that can be passed to your application to recreate a test case.
  2. output - Reference outputs to evaluate your application. These can be the ground truth for BLEU, ROUGE, and Ground Truth Adherence metrics, or reference outputs for manual reference.
  3. metadata - Additional data you can use to filter or group your dataset.

​
Create datasets

When you create a dataset, it is uploaded to Galileo and available to future experiments. Datasets need to have unique names, and are available to all projects across your organization. 
from galileo.datasets import create_dataset

# Create a dataset with test data
test_data = [
    {
        "input": "Which continent is Spain in?",
        "output": "Europe",
    },
    {
        "input": "Which continent is Japan in?",
        "output": "Asia",
    },
]

dataset = create_dataset(
    name="countries",
    content=test_data
)
See the create_dataset Python SDK docs or createDataset TypeScript SDK docs for more details.

​
Get existing datasets

Once a dataset has been created in Galileo, you can retrieve it to use in your experiments by name or ID. 
from galileo.datasets import get_dataset

# Get a dataset by name
dataset = get_dataset(
    name="countries"
)

# Get a dataset by ID
dataset = get_dataset(
    id="dataset-id"
)

# Get its content
dataset.get_content()
See the get_dataset Python SDK docs or getDataset TypeScript SDK docs for more details.

​
Add rows to existing datasets

Once a dataset has been created, you can manually add rows to it. 
from galileo.datasets import get_dataset

# Get an existing dataset
dataset = get_dataset(
    name="countries"
)

# Add new rows to the dataset
dataset.add_rows([
    {
        "input": "Which continent is Morocco in?",
        "output": "Africa",
    },
    {
        "input": "Which continent is Australia in?",
        "output": "Oceania",
    },
])
See the add_rows Python SDK docs or addRowsToDataset TypeScript SDK docs for more details.

​
Generate synthetic data to extend a dataset

Galileo can use an LLM integration to generate rows of synthetic data that you can then add to a dataset. This synthetic data is generated using a mixture of prompts, instructions, few-shot examples, and data types. Once these rows have been generated, they can be added to a new or existing dataset. 
from galileo.datasets import extend_dataset

# Generate synthetic data
dataset = extend_dataset(
    prompt_settings={'model_alias': 'GPT-4o'},
    prompt="Nutrition and health chatbot",
    instructions="Questions that an average-health person would be interested in",
    examples=[
        "Is cereal for breakfast healthy?",
        "How many cups of coffee is unhealthy?",
    ],
    data_types=['General Query'],
    count=10,
)

# Print the generated dataset contents
for row in dataset:  
    print(row.values_dict.additional_properties["input"])
See the extend_dataset Python SDK docs or extendDataset TypeScript SDK docs for more details.

​
List datasets

You can retrieve all the datasets for a project. 
from galileo.datasets import list_datasets

# List all datasets in a project
datasets = list_datasets()

# List datasets with a custom limit
datasets = list_datasets(
    limit=50,
)
See the list_datasets Python SDK docs or getDatasets TypeScript SDK docs for more details.

​
Delete datasets

If a dataset is no longer needed, you can delete it by name or ID. 
from galileo.datasets import delete_dataset

# Delete a dataset by name
delete_dataset(name="countries")

# Delete a dataset by ID
delete_dataset(id="dataset-id")
See the delete_dataset Python SDK docs or deleteDataset TypeScript SDK docs for more details.

​
Work with dataset versions

Galileo automatically creates new versions of datasets when they are modified. You can access different versions by getting the dataset history. 
from galileo.datasets import get_dataset_version_history

# Get the version history
datasets = get_dataset_version_history(
    dataset_name="countries"
)

# List out the rows added with each version
for dataset in datasets.versions:
    print(f"""
    Version index: {dataset.version_index},
    rows added: {dataset.rows_added}
    """)
See the get_dataset_version_history Python SDK docs for more details.

​
Use datasets in experiments

Datasets are primarily used for running experiments to evaluate the performance of your LLM applications: 
from galileo.datasets import get_dataset
from galileo.experiments import run_experiment
from galileo.prompts import get_prompt
from galileo.schema.metrics import GalileoScorers

# Get an existing dataset
dataset = get_dataset(
    name="countries"
)

# Get an existing prompt
prompt = get_prompt(
    name="geography-prompt"
)

# Run an experiment with the dataset and prompt
results = run_experiment(
    "geography-experiment",
    dataset=dataset,
    prompt_template=prompt,
    metrics=[GalileoScorers.completeness],
    project="my-project",
)

Datasets

Learn about more datasets, the data driving your experiments.

Experiments

Learn how to use datasets and experiments to improve your application.

Prompt Templates

Learn how to create and use prompt templates in experiments