The Python SDK allows you to log all prompts, responses, and statistics around your LLM usage. There are three main ways to log your application:

  1. Using a wrapper (Recommended) - Instead of importing common LLMs like openai, use Galileo’s wrapper which automatically logs everything, no other code changes required!
  2. Using a decorator - By decorating a function that calls an LLM with the @log decorator, the Galileo SDK logs all AI prompts within.
  3. Directly using the GalileoLogger class (Manual) - As a last resort, you can directly use the base class, but this requires calling multiple methods per LLM call.

Regardless of how you go about logging your AI application, you will still need to initialize your API keys and install the Galileo SDK by following the steps below.

​
Key Concepts

Throughout this reference guide there are several ideas which will be used extensively.

  1. Project - All logs are stored within a project in Galileo. You can create and manage your projects using the Galileo UI.
  2. Log Streams - Log streams are a way to organize logs in Galileo. You can create and manage your log streams using the Galileo UI.
  3. Traces - These track a collection of Logs which represent a “single response”. For multi-step LLM calls, this helps debug how the response was built, and where issues may have occurred.
  4. Spans - Spans are a single step in a trace. They can be a workflow if they contain multiple sub-spans, llm for a step invoking an LLM call, retriever for when you retrieve data, or tool for agentic tool calls.

As your application runs, it will stream logs back to Galileo in a series of traces that then get analyzed using Metrics you set up. Traces that seem problematic can then be reviewed step by step to determine what part of the pipeline needs changing, or if the Metrics need tweaking.

​
Installation

Install Galileo’s Python SDK to your project by running:

pip install galileo

​
Initialization/Authentication

Create or update a .env file with the following values:

# Scoped to an Organization
GALILEO_API_KEY=...

# Optional, set a default Project
GALILEO_PROJECT=...
# Optional, set a default Log Stream
GALILEO_LOG_STREAM=...

​
Logging

​
Using LLM Wrappers

The simplest way to get started is to use our OpenAI client wrapper. This example will automatically produce a single-span trace in the Logstream UI:

import os
from galileo.openai import openai

# Initialize the Galileo wrapped OpenAI client
client = openai.OpenAI(api_key=os.environ.get("OPENAI_API_KEY"))

def call_openai():
    chat_completion = client.chat.completions.create(
        messages=[{"role": "user", "content": "Say this is a test"}],
        model="gpt-4o"
    )
    return chat_completion.choices[0].message.content

# This will create a single span trace with the OpenAI call
response = call_openai()
print(response)

​
Using the @log Decorator

The @log decorator is used to capture the inputs and outputs of a function as a span. By default, a workflow span is created when span_type isn’t specified. Here are the different span types:

  • Workflow: A span that can have child spans, useful for nesting several child spans to denote a thread within a trace. If you add the @log decorator to a parent method, calls that are made within that scope are automatically logged in the same trace.
  • Llm: Captures the input, output, and settings of an LLM call. This span gets automatically created when our OpenAI client library wrapper is used. Cannot have nested children.
  • Retriever: Contains the output documents of a retrieval operation.
  • Tool: Captures the input and output of a tool call. Used to decorate functions that are invoked as tools.

This example will create a trace with a workflow span and two nested llm spans:

from galileo import log
from galileo.openai import openai

client = openai.OpenAI(api_key=os.environ.get("OPENAI_API_KEY"))

def call_openai():
    chat_completion = client.chat.completions.create(
        messages=[{"role": "user", "content": "Say this is a test"}], 
        model="gpt-4o"
    )
    return chat_completion.choices[0].message.content

@log
def make_nested_call():
    call_openai()
    call_openai()

# This will create a trace with a workflow span and two nested LLM spans
response = make_nested_call()
print(response)

Here’s how to create a retriever span using the decorator:

from galileo import log

@log(span_type="retriever")
def retrieve_documents(query: str):
    return ["doc1", "doc2"]

# This will create a trace with a retriever span containing the documents in the output
retrieve_documents(query="history")

Here’s how to create a tool span using the decorator:

from galileo import log

@log(span_type="tool")
def tool_call(input: str = "tool call input"):
    return "tool call output"

# This will create a trace with a tool span containing the tool call output
tool_call(input="question")

For more detailed information and examples, see the @log Decorator documentation.

​
Pure invocation using the GalileoLogger

This is the most verbose way to log your application. It requires manually calling the GalileoLogger class and adding spans to the trace. We recommend using the other two methods whenever possible.

from galileo import GalileoLogger

# This will log to the project and log stream specified in the logger constructor
logger = GalileoLogger(project="gen-ai-project", log_stream="test3")
trace = logger.start_trace("Say this is a test")

logger.add_llm_span(
    input="Say this is a test",
    output="Hello, this is a test",
    model="gpt-4o",
    num_input_tokens=10,
    num_output_tokens=3,
    total_tokens=13,
    duration_ns=1000,
)

logger.conclude(output="Hello, this is a test", duration_ns=1000)
logger.flush() # This will upload the trace to Galileo

​
Grouping and Uploading Logs Faster: galileo_context()

Regardless of the method you use to add logs, the Galileo context manager can be useful for a few things:

  • Automatically starting a trace and ensuring anything that happens in its scope is logged as a span within the trace.
  • For long running app runtimes like Streamlit, the request never terminates. You can use the context manager to start a trace and ensure that traces are flushed when the manager exits.
  • You might want to route a part of your app to a different Project or Log Stream. You can use the context manager to set the trace scope.

Using the context manager to create a trace with a nested LLM span (which is automatically flushed when the manager exits):

import os
from galileo import galileo_context
from galileo.openai import openai

# Initialize the Galileo wrapped OpenAI client
client = openai.OpenAI(api_key=os.environ.get("OPENAI_API_KEY"))

# This will log to the project and log stream specified in the context manager
with galileo_context(project="gen-ai-project", log_stream="test2"):
    chat_completion = client.chat.completions.create(
        messages=[{"role": "user", "content": "Say this is a test"}],
        model="gpt-4o"
    )
    print(chat_completion.choices[0].message.content)

In some cases (like long-running processes), it may be necessary to explicitly flush the trace to upload it to Galileo:

import os
from galileo import galileo_context
from galileo.openai import openai

galileo_context.init(project="your-project-name", log_stream="your-log-stream-name")

# Initialize the Galileo wrapped OpenAI client
client = openai.OpenAI(api_key=os.environ.get("OPENAI_API_KEY"))

def call_openai():
    chat_completion = client.chat.completions.create(
        messages=[{"role": "user", "content": "Say this is a test"}],
        model="gpt-4o"
    )
    return chat_completion.choices[0].message.content

# This will create a single span trace with the OpenAI call
call_openai()

# This will upload the trace to Galileo
galileo_context.flush()

​
Additional Documentation

For more detailed information on specific topics, please refer to the following pages: