Spans

​

The role of spans in representing units of work

1. Granular Representation: Spans break down complex workflows into discrete, manageable units, making it easier to understand and analyze each step.
2. Contextual Information: Each span contains rich contextual information about the operation it represents, including inputs, outputs, and configuration details.
3. Hierarchical Organization: Spans can be nested within other spans, creating a hierarchical structure that reflects the organization of your application’s workflows.
4. Performance Tracking: Spans capture timing information, allowing you to identify bottlenecks and optimize performance.
5. Error Isolation: When issues occur, spans help isolate the specific operation where the error originated.

​

How spans contribute to system evaluation and debugging

1. Pinpoint Precision: Spans allow you to pinpoint exactly where in a workflow issues are occurring, rather than just knowing that something went wrong.
2. Input-Output Analysis: By examining the inputs and outputs of each span, you can identify where unexpected results are being generated.
3. Performance Profiling: Span durations help you identify which operations are taking the most time, guiding optimization efforts.
4. Error Context: When errors occur, spans provide the context needed to understand what led to the error and how to fix it.
5. Quality Assessment: Spans enable evaluation of individual components of your system, helping you identify which parts are performing well and which need improvement.

​

Key components of a span

​

Essential components

1. Input: The data or query provided to the operation
2. Output: The response or result generated by the operation
3. Span Type: The type of operation (e.g., llm, workflow, retriever, tool)
4. Duration: The time taken to complete the operation
5. Timestamp: When the operation occurred
6. Parent Span: The span that contains this span (if applicable)
7. Child Spans: Spans contained within this span (if applicable)

​

Span types and their behavior

1. LLM Spans: Capture interactions with language models, including:
   • Prompts sent to the model
   • Responses received from the model
   • Model configuration (e.g., model name, temperature)
   • Token usage and other performance metrics
2. Workflow Spans: Represent logical groupings of operations that can contain child spans, useful for:
   • Organizing complex processes
   • Creating hierarchical structures
   • Grouping related operations
3. Retriever Spans: Record document retrieval operations, including:
   • Search queries
   • Retrieved documents
   • Relevance scores
   • Retrieval configuration
4. Tool Spans: Document tool calls in agent-based systems, including:
   • Tool inputs
   • Tool outputs
   • Error information (if applicable)
   • Tool configuration

​

The role of metadata in spans

1. Contextual Information: Metadata provides additional context about the operation, such as user IDs, session information, or application-specific data.
2. Filtering and Searching: Metadata can be used to filter and search for specific spans, making it easier to find relevant information.
3. Correlation: Metadata helps correlate spans across different traces or systems, enabling end-to-end visibility.
4. Custom Attributes: You can add custom attributes to metadata to capture information specific to your application or use case.
5. Debugging Context: In debugging scenarios, metadata can provide crucial context about the state of your application when the operation occurred.

​

Creating and managing spans

​

Using the Galileo SDK

from galileo import GalileoLogger

logger = GalileoLogger(project="my-project", log_stream="my-log-stream")

# Start a trace
trace = logger.start_trace(
    input="Tell me about Mars",
    output=None,  # Will be set later
    name="Mars Information",
    metadata={"source": "example"},
    tags=["space", "planets"]
)

# Add an LLM span
logger.add_llm_span(
    input="Tell me about Mars",
    output="Mars is the fourth planet from the Sun and the second-smallest planet in the Solar System...",
    config={"model": "gpt-4o", "temperature": 0.7},
    name="Mars Information",
    duration_ns=1000000000,  # 1 second
    metadata={"request_id": "req-123"},
    tags=["llm", "space"]
)

# Conclude the trace
logger.conclude(output="Mars is the fourth planet from the Sun...")

# Flush the trace to Galileo
logger.flush()

​

Using integration wrappers

import os
from galileo.openai import openai

# This will automatically create a trace with an LLM span for the OpenAI call
client = openai.OpenAI(api_key=os.environ["OPENAI_API_KEY"])
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "What is the capital of France?"}]
)
print(response.choices[0].message.content)

​

Required inputs and configurations

• Input: The data being processed by the operation
• Output (for concluded spans): The result of the operation
• Span Type: The type of operation (llm, workflow, retriever, or tool)
• Name (for workflow and tool spans): A descriptive name for the operation

• Metadata: Additional contextual information
• Tags: Labels for categorization
• Parent Span: The span that contains this span (if applicable)
• Model Configuration (for llm spans): Details about the model used

​

Customization options

• Custom Metadata: Add application-specific metadata to provide additional context
• Custom Tags: Add tags to categorize spans and make them easier to filter
• Nested Spans: Create hierarchical structures by nesting spans within other spans
• Error Handling: Capture and log errors for better debugging
• Custom Span Types: Define custom span types for application-specific operations

​

Using spans in practice

​

Common use cases

1. LLM Interaction Tracking: Capture details of interactions with language models, including prompts, responses, and model configurations.
2. RAG Pipeline Analysis: Track each step in a retrieval-augmented generation pipeline, from document retrieval to LLM generation.
3. Agent Workflow Monitoring: Monitor the actions of AI agents, including tool calls and decision-making processes.
4. Performance Profiling: Identify bottlenecks and optimization opportunities by analyzing span durations.
5. Error Diagnosis: When issues occur, examine spans to understand what went wrong and why.
6. Quality Assessment: Evaluate the quality of individual components in your AI system.

​

Representing different parts of a workflow

• LLM Spans: Represent interactions with language models
• Retriever Spans: Represent document retrieval operations
• Tool Spans: Represent tool calls or external API interactions
• Workflow Spans: Represent logical groupings of operations or multi-step processes

​

Best practices

1. Meaningful Names: Use descriptive names for spans to make them easier to understand and analyze.
2. Appropriate Granularity: Create spans at an appropriate level of granularity - too fine-grained and you’ll have too many spans to manage, too coarse-grained and you’ll miss important details.
3. Include Relevant Metadata: Add metadata that will be useful for filtering, analysis, and debugging.
4. Proper Hierarchy: Organize spans in a logical hierarchy that reflects the structure of your application’s workflows.
5. Error Handling: Log errors in spans to help with debugging and analysis.
6. Complete Spans: Always conclude spans to ensure they have both input and output information.
7. Consistent Naming: Use consistent naming conventions for similar spans across your application.

​

Integration with the Galileo ecosystem

​

Integration with other components

1. Logs: Spans contain log data, providing structure and context for individual log entries.
2. Traces: Spans are organized into traces, which represent complete interactions or workflows.
3. Metrics: Span data feeds into metrics, allowing you to measure and monitor key aspects of your AI application.
4. Experiments: Spans can be used in experiments to compare different models, prompts, or configurations.
5. Datasets: Spans can be converted into datasets for fine-tuning or evaluation.
6. Alerts: Patterns in spans can trigger alerts to notify you of issues or anomalies.

​

Data flow

1. Your application creates spans using the Galileo SDK
2. Spans are organized into traces
3. Traces are flushed to Galileo (automatically or manually)
4. Galileo processes and stores the spans
5. Spans are available for viewing, analysis, and use in other Galileo features
6. Metrics and evaluations are computed based on span data
7. Insights derived from spans inform improvements to your AI application

​

Dependencies and connections

• Traces: All spans belong to a trace
• Projects: All spans belong to a project (via their trace)
• Log Streams: Spans are organized into Log streams (via their trace)
• API Keys: Authentication is required to send spans to Galileo
• SDK: The Galileo SDK is used to create and send spans

​

Examples and practical applications

​

Basic span creation

from galileo import GalileoLogger

logger = GalileoLogger(project="my-project", log_stream="my-log-stream")

# Start a trace
trace = logger.start_trace(
    input="Tell me about Mars",
    output=None,  # Will be set later
    name="Mars Information",
    metadata={"source": "example"},
    tags=["space", "planets"]
)

# Add an LLM span
logger.add_llm_span(
    input="Tell me about Mars",
    output="Mars is the fourth planet from the Sun and the second-smallest planet in the Solar System...",
    config={"model": "gpt-4o", "temperature": 0.7},
    name="Mars Information",
    duration_ns=1000000000,  # 1 second
    metadata={"request_id": "req-123"},
    tags=["llm", "space"]
)

# Conclude the trace
logger.conclude(output="Mars is the fourth planet from the Sun...")

# Flush the trace to Galileo
logger.flush()

​

Spans with detailed metadata

from galileo import GalileoLogger

logger = GalileoLogger(project="my-project", log_stream="my-log-stream")

# Start a trace with metadata
trace = logger.start_trace(
    input="What's the weather in New York?",
    output=None,  # Will be set later
    name="Weather Query",
    metadata={
        "user_id": "user-123",
        "session_id": "session-456",
        "client_info": {"device": "mobile", "os": "iOS"}
    },
    tags=["weather", "query"]
)

# Add an LLM span with detailed metadata
logger.add_llm_span(
    input="What's the weather in New York?",
    output="The weather in New York is currently cloudy with a temperature of 65°F.",
    config={"model": "gpt-4o", "temperature": 0.7},
    name="Weather Information",
    duration_ns=1000000000,  # 1 second
    metadata={
        "request_id": "req-789",
        "latency_ms": 250,
        "token_count": {"input": 10, "output": 15}
    },
    tags=["llm", "weather"]
)

# Conclude the trace
logger.conclude(output="The weather in New York is currently cloudy with a temperature of 65°F.")

# Flush the trace to Galileo
logger.flush()

​

Multi-Span RAG example

from galileo import GalileoLogger

logger = GalileoLogger(project="my-project", log_stream="my-log-stream")

# Start a trace for a RAG application
trace = logger.start_trace(
    input="What are the health benefits of meditation?",
    output=None,  # Will be set later
    name="Meditation Benefits Query",
    metadata={"source": "example"},
    tags=["health", "meditation", "rag"]
)

# Start a workflow span for the retrieval process
retrieval_workflow = logger.start_workflow_span(
    name="Document Retrieval",
    input="What are the health benefits of meditation?",
    metadata={"workflow_type": "retrieval"},
    tags=["retrieval"]
)

# Add a retriever span for the query
logger.add_retriever_span(
    input="health benefits meditation",
    documents=[
        {"content": "Meditation has been shown to reduce stress and anxiety...", "metadata": {"source": "health-journal.pdf"}},
        {"content": "Regular meditation practice can lower blood pressure...", "metadata": {"source": "medical-research.txt"}}
    ],
    name="Document Search",
    duration_ns=1000000000,  # 1 second
    metadata={"retriever_type": "vector_search"},
    tags=["retrieval", "vector"]
)

# End the retrieval workflow
logger.end_workflow_span(retrieval_workflow, output="Retrieved 2 documents")

# Start a workflow span for the generation process
generation_workflow = logger.start_workflow_span(
    name="Response Generation",
    input="Based on these documents, what are the health benefits of meditation?",
    metadata={"workflow_type": "generation"},
    tags=["generation"]
)

# Add an LLM span for generating the response
logger.add_llm_span(
    input="Based on these documents, what are the health benefits of meditation? Documents: [...]",
    output="Based on the research, meditation offers several health benefits including reduced stress and anxiety, lower blood pressure...",
    config={"model": "gpt-4o", "temperature": 0.3},
    name="Answer Generation",
    duration_ns=2000000000,  # 2 seconds
    metadata={"request_id": "req-789"},
    tags=["llm", "health"]
)

# End the generation workflow
logger.end_workflow_span(generation_workflow, output="Generated response successfully")

# Conclude the trace
logger.conclude(output="Based on the research, meditation offers several health benefits...")

# Flush the trace to Galileo
logger.flush()

​

Next steps

​

Advanced span analysis

• Span Visualization: Use the Galileo UI to visualize spans and understand the flow of data through your application.
• Performance Analysis: Analyze span timing information to identify bottlenecks and opportunities for improvement.
• Error Analysis: Use spans to understand the root causes of errors and exceptions.

​

Related Galileo workflows

• Trace Analysis: Examine how spans are organized into traces to understand complete workflows.
• Metrics: Set up metrics based on span data to monitor key aspects of your AI application.
• Experiments: Use spans to compare different models, prompts, or configurations.
• Datasets: Convert spans into datasets for fine-tuning or evaluation.
• Alerts: Configure alerts based on patterns in your spans.

​

Further resources

• Traces - Understand how spans are organized into traces
• Log Streams - Explore how to organize spans and traces into streams
• Projects - Learn about the top-level organization of Galileo resources
• Metrics - Discover how to measure key aspects of your AI application
• Experiments - Learn how to compare different approaches