​
Introduction

This document provides a comprehensive reference for the Galileo Python SDK, covering all the key classes, methods, and functions available for logging and analyzing your LLM applications.

​
Installation

pip install galileo

​
Initialization/Authentication

You can configure Galileo using environment variables:

# Scoped to an Organization
GALILEO_API_KEY=...

# Optional, set a default Project
GALILEO_PROJECT=...

# Optional, set a default Log Stream
GALILEO_LOG_STREAM=...

​
Core Components

​
OpenAI Wrapper

The simplest way to get started is to use our OpenAI client wrapper:

import os
from galileo.openai import openai

# Initialize the Galileo wrapped OpenAI client
client = openai.OpenAI(api_key=os.environ.get("OPENAI_API_KEY"))

def call_openai():
    chat_completion = client.chat.completions.create(
        messages=[{"role": "user", "content": "Say this is a test"}],
        model="gpt-4o"
    )
    return chat_completion.choices[0].message.content

# This will create a single span trace with the OpenAI call
response = call_openai()
print(response)

​
@log Decorator

The @log decorator allows you to capture function inputs and outputs as spans:

from galileo import log

@log
def my_function(input_text):
    # Function implementation
    return result

​
galileo_context

The galileo_context context manager provides control over trace context:

import os
from galileo import galileo_context
from galileo.openai import openai

# Initialize the Galileo wrapped OpenAI client
client = openai.OpenAI(api_key=os.environ.get("OPENAI_API_KEY"))

# This will log to the project and log stream specified in the context manager
with galileo_context(project="gen-ai-project", log_stream="test2"):
    chat_completion = client.chat.completions.create(
        messages=[{"role": "user", "content": "Say this is a test"}],
        model="gpt-4o"
    )
    print(chat_completion.choices[0].message.content)

​
GalileoLogger

For more manual control over logging:

from galileo import GalileoLogger

# This will log to the project and log stream specified in the logger constructor
logger = GalileoLogger(project="gen-ai-project", log_stream="test3")
trace = logger.start_trace("Say this is a test")

logger.add_llm_span(
    input="Say this is a test",
    output="Hello, this is a test",
    model="gpt-4o",
    num_input_tokens=10,
    num_output_tokens=3,
    total_tokens=13,
    duration_ns=1000,
)

logger.conclude(output="Hello, this is a test", duration_ns=1000)
logger.flush() # This will upload the trace to Galileo

​
Datasets

For creating and managing datasets:

from galileo.datasets import create_dataset, get_dataset, list_datasets, delete_dataset

# Create a dataset
dataset = create_dataset(
    name="my-dataset",
    content=[{"input": "example", "expected": "result"}]
)

# Get a dataset
dataset = get_dataset(name="my-dataset")

# List datasets
datasets = list_datasets()

# Delete a dataset
delete_dataset(name="my-dataset")

# Add rows to a dataset
dataset.add_rows([{"input": "new example", "expected": "new result"}])

​
Prompts

For creating and using prompt templates:

from galileo.prompts import create_prompt_template, get_prompt_template
from galileo_core.schemas.logging.llm import Message, MessageRole

# Create a prompt template
prompt = create_prompt_template(
    name="my-prompt",
    project="my-project",
    messages=[
        Message(role=MessageRole.SYSTEM, content="You are a helpful assistant."),
        Message(role=MessageRole.USER, content="Answer this question: {{question}}")
    ]
)

# Get a prompt template
prompt = get_prompt_template(
    project="my-project",
    name="my-prompt"
)

​
Experiments

For running experiments:

from galileo.experiments import run_experiment, create_experiment, get_experiment, get_experiments

# Run an experiment
results = run_experiment(
    "my-experiment",
    dataset=get_dataset(name="my-dataset"),
    prompt_template=get_prompt_template(project="my-project", name="my-prompt"),
    metrics=["correctness"],
    project="my-project"
)

# Create an experiment
experiment = create_experiment(
    project_id="my-project-id",
    experiment_name="my-experiment"
)

# Get an experiment
experiment = get_experiment(
    project_id="my-project-id",
    experiment_name="my-experiment"
)

# List experiments
experiments = get_experiments(project_id="my-project-id")

​
LangChain Integration

For integrating with LangChain:

from galileo.handlers.langchain import GalileoCallback
from langchain_openai import ChatOpenAI
from langchain.schema import HumanMessage

# Create a callback handler
callback = GalileoCallback()

# Initialize the LLM with the callback
llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0.7, callbacks=[callback])

# Create a message with the user's query
messages = [HumanMessage(content="What is LangChain and how is it used with OpenAI?")]

# Make the API call
response = llm.invoke(messages)

print(response.content)

​
API Reference

​
galileo.openai

  • openai.OpenAI: A wrapped version of the OpenAI client that automatically logs all API calls.

​
galileo.log

  • @log: A decorator for capturing function inputs and outputs as spans.
    • Parameters:
      • span_type: The type of span to create (“workflow”, “llm”, “retriever”, “tool”).
      • name: A custom name for the span.
      • project: The project to log to.
      • log_stream: The log stream to log to.
      • params: Additional parameters to include in the span.

​
galileo.galileo_context

  • galileo_context: A context manager for controlling logging behavior.
    • Parameters:
      • project: The project to log to.
      • log_stream: The log stream to log to.
    • Methods:
      • init(project, log_stream): Initialize the context.
      • flush(): Flush all traces to Galileo.

​
galileo.GalileoLogger

  • GalileoLogger: A class for manual control over logging.
    • Parameters:
      • project: The project to log to.
      • log_stream: The log stream to log to.
    • Methods:
      • start_trace(input, name, tags, metadata, duration_ns, created_at): Start a new trace.
      • add_llm_span(input, output, model, ...): Add an LLM span to the trace.
      • add_retriever_span(input, output, name, ...): Add a retriever span to the trace.
      • add_tool_span(input, output, name, ...): Add a tool span to the trace.
      • add_workflow_span(input, output, name, ...): Add a workflow span to the trace.
      • conclude(output, duration_ns, status_code, conclude_all): Conclude the trace.
      • flush(): Flush the trace to Galileo.

​
galileo.datasets

  • create_dataset(name, content): Create a new dataset.
  • get_dataset(name, id, with_content): Get an existing dataset.
  • list_datasets(limit): List all available datasets.
  • delete_dataset(name, id): Delete a dataset.

​
galileo.prompts

  • create_prompt_template(name, project, messages): Create a new prompt template.
  • get_prompt_template(project, name): Get an existing prompt template.

​
galileo.experiments

  • run_experiment(name, dataset, prompt, metrics, project): Run an experiment.
  • create_experiment(project_id, experiment_name): Create a new experiment.
  • get_experiment(project_id, experiment_name): Get an existing experiment.
  • get_experiments(project_id): List all experiments in a project.

​
galileo.handlers.langchain

  • GalileoCallback: A callback handler for LangChain.
    • Parameters:
      • galileo_logger: A custom logger instance.
      • start_new_trace: Whether to start a new trace for each chain.
      • flush_on_chain_end: Whether to flush traces when chains end.