Introduction
This document provides a comprehensive reference for the Galileo Python SDK (latest release), covering all the key classes, methods, and functions available for logging and analyzing your LLM applications.
Installation
Initialization/Authentication
You can configure Galileo using environment variables:
# Scoped to an Organization
GALILEO_API_KEY=...
# Optional, set a default Project
GALILEO_PROJECT=...
# Optional, set a default Log Stream
GALILEO_LOG_STREAM=...
Core Components
OpenAI Wrapper
The simplest way to get started is to use our OpenAI client wrapper:
import os
from galileo.openai import openai
# Initialize the Galileo wrapped OpenAI client
client = openai.OpenAI(api_key=os.environ.get("OPENAI_API_KEY"))
def call_openai():
chat_completion = client.chat.completions.create(
messages=[{"role": "user", "content": "Say this is a test"}],
model="gpt-4o"
)
return chat_completion.choices[0].message.content
# This will create a single span trace with the OpenAI call
response = call_openai()
print(response)
@log Decorator
The @log decorator allows you to capture function inputs and outputs as spans:
from galileo import log
@log
def my_function(input_text):
# Function implementation
return result
galileo_context
The galileo_context context manager provides control over trace context:
import os
from galileo import galileo_context
from galileo.openai import openai
# Initialize the Galileo wrapped OpenAI client
client = openai.OpenAI(api_key=os.environ.get("OPENAI_API_KEY"))
# This will log to the project and log stream specified in the context manager
with galileo_context(project="gen-ai-project", log_stream="test2"):
chat_completion = client.chat.completions.create(
messages=[{"role": "user", "content": "Say this is a test"}],
model="gpt-4o"
)
print(chat_completion.choices[0].message.content)
GalileoLogger
For more manual control over logging:
from galileo import GalileoLogger
# This will log to the project and log stream specified in the logger constructor
logger = GalileoLogger(project="gen-ai-project", log_stream="test3")
trace = logger.start_trace("Say this is a test")
logger.add_llm_span(
input="Say this is a test",
output="Hello, this is a test",
model="gpt-4o",
num_input_tokens=10,
num_output_tokens=3,
total_tokens=13,
duration_ns=1000,
)
logger.conclude(output="Hello, this is a test", duration_ns=1000)
logger.flush() # This will upload the trace to Galileo
Datasets
For creating and managing datasets:
from galileo.datasets import create_dataset, get_dataset, list_datasets, delete_dataset
# Create a dataset
dataset = create_dataset(
name="my-dataset",
content=[{"input": "example", "output": "result"}]
)
# Get a dataset
dataset = get_dataset(name="my-dataset")
# List datasets
datasets = list_datasets()
# Delete a dataset
delete_dataset(name="my-dataset")
# Add rows to a dataset
dataset.add_rows([{"input": "new example", "output": "new result"}])
Prompts
For creating and using prompt templates:
from galileo import Message, MessageRole
from galileo.prompts import create_prompt_template, get_prompt_template
# Create a prompt template
prompt = create_prompt_template(
name="my-prompt",
project="my-project",
messages=[
Message(role=MessageRole.system, content="You are a helpful assistant."),
Message(role=MessageRole.user, content="Answer this question: {{question}}")
]
)
# Get a prompt template
prompt = get_prompt_template(
project="my-project",
name="my-prompt"
)
Experiments
For running experiments:
from galileo.experiments import run_experiment, create_experiment, get_experiment, get_experiments
# Run an experiment
results = run_experiment(
"my-experiment",
dataset=get_dataset(name="my-dataset"),
prompt_template=get_prompt_template(project="my-project", name="my-prompt"),
metrics=["correctness"],
project="my-project"
)
# Create an experiment
experiment = create_experiment(
project_id="my-project-id",
experiment_name="my-experiment"
)
# Get an experiment
experiment = get_experiment(
project_id="my-project-id",
experiment_name="my-experiment"
)
# List experiments
experiments = get_experiments(project_id="my-project-id")
LangChain Integration
For integrating with LangChain:
from galileo.handlers.langchain import GalileoCallback
from langchain_openai import ChatOpenAI
from langchain.schema import HumanMessage
# Create a callback handler
callback = GalileoCallback()
# Initialize the LLM with the callback
llm = ChatOpenAI(model="gpt-4o", temperature=0.7, callbacks=[callback])
# Create a message with the user's query
messages = [HumanMessage(content="What is LangChain and how is it used with OpenAI?")]
# Make the API call
response = llm.invoke(messages)
print(response.content)
API Reference
galileo.openai
openai.OpenAI: A wrapped version of the OpenAI client that automatically logs all API calls.
galileo.log
@log: A decorator for capturing function inputs and outputs as spans.
- Parameters:
span_type: The type of span to create (“workflow”, “llm”, “retriever”, “tool”).name: A custom name for the span.project: The project to log to.log_stream: The log stream to log to.params: Additional parameters to include in the span.
galileo.galileo_context
galileo_context: A context manager for controlling logging behavior.
- Parameters:
project: The project to log to.log_stream: The log stream to log to.
- Methods:
init(project, log_stream): Initialize the context.flush(): Flush all traces to Galileo.
galileo.GalileoLogger
GalileoLogger: A class for manual control over logging.
- Parameters:
project: The project to log to.log_stream: The log stream to log to.
- Methods:
start_trace(input, name, tags, metadata, duration_ns, created_at): Start a new trace.add_llm_span(input, output, model, ...): Add an LLM span to the trace.add_retriever_span(input, output, name, ...): Add a retriever span to the trace.add_tool_span(input, output, name, ...): Add a tool span to the trace.add_workflow_span(input, output, name, ...): Add a workflow span to the trace.conclude(output, duration_ns, status_code, conclude_all): Conclude the trace.flush(): Flush the trace to Galileo.
galileo.datasets
create_dataset(name, content): Create a new dataset.get_dataset(name, id): Get an existing dataset.list_datasets(limit): List all available datasets.delete_dataset(name, id): Delete a dataset.
galileo.prompts
create_prompt_template(name, project, messages): Create a new prompt template.get_prompt_template(project, name): Get an existing prompt template.
galileo.experiments
run_experiment(name, dataset, prompt, metrics, project): Run an experiment.create_experiment(project_id, experiment_name): Create a new experiment.get_experiment(project_id, experiment_name): Get an existing experiment.get_experiments(project_id): List all experiments in a project.
galileo.handlers.langchain
GalileoCallback: A callback handler for LangChain.
- Parameters:
galileo_logger: A custom logger instance.start_new_trace: Whether to start a new trace for each chain.flush_on_chain_end: Whether to flush traces when chains end.
