​
Overview

This guide explains how to add runtime protection to a simple chatbot. You will be running a basic chatbot, detecting toxicity in the users input, and if this is detected, ending the conversation. You will start by creating a central stage, as if you were an AI governance team. You will then use this stage in a simple chatbot. In a real-world scenario, you could use this detection to redirect a user from an AI chatbot to a human representative. In this guide you will:
  1. Set up your project with Galileo
  2. Create a central stage
  3. Create a basic chatbot
  4. Add runtime protection to your basic chatbot

​
Before you start

To complete this how-to, you will need:

​
Install dependencies

To use Galileo, you need to install some package dependencies, and configure environment variables.
1

Install Required Dependencies

Install the required dependencies for your app. Create a virtual environment using your preferred method, then install dependencies inside that environment:
pip install "galileo[openai]" python-dotenv
2

Create a .env file, and add the following values

# Your Galileo API key
GALILEO_API_KEY="your-galileo-api-key"

# Your Galileo project name
GALILEO_PROJECT="your-galileo-project-name"

# The name of the log stream you want to use for logging
GALILEO_LOG_STREAM="your-galileo-log-stream"

# Provide the console url below if you are using a
# custom deployment, and not using the free tier, or app.galileo.ai.
# This will look something like “console.galileo.yourcompany.com”.
# GALILEO_CONSOLE_URL="your-galileo-console-url"

# OpenAI properties
OPENAI_API_KEY="your-openai-api-key"

# Optional. The base URL of your OpenAI deployment.
# Leave this commented out if you are using the default OpenAI API.
# OPENAI_BASE_URL="your-openai-base-url-here"

# Optional. Your OpenAI organization.
# OPENAI_ORGANIZATION="your-openai-organization-here"
This assumes you are using a free Galileo account. If you are using a custom deployment, then you will also need to add the URL of your Galileo Console:
.env
GALILEO_CONSOLE_URL=your-Galileo-console-URL

​
Create a central stage

You first need to create a central stage. In a real-world scenario, these central stages would be managed by an AI governance team.
1

Create a Python file to create the stage called `create_central_stage.py`

This file will define a rule that is triggered if the input toxicity is evaluated to greater than 0.1. This will then be added to a ruleset with an override action with 3 choices of response.This ruleset will be added to a central stage, registered in your project.
2

Add import directives

Start by adding import directives to import all the functions and types needed for creating stages.
from galileo import GalileoScorers
from galileo.stages import create_protect_stage

from galileo_core.schemas.protect.action import OverrideAction
from galileo_core.schemas.protect.rule import Rule, RuleOperator
from galileo_core.schemas.protect.ruleset import Ruleset
from galileo_core.schemas.protect.stage import StageType

from dotenv import load_dotenv
load_dotenv()
3

Create the rule

Add code to create a rule. This rule is triggered if the input toxicity is greater than 0.1.
# Create a rule for toxicity
toxicity_rule = Rule(
    metric=GalileoScorers.input_toxicity,
    operator=RuleOperator.gt,
    target_value=0.1
)
4

Create an override action

Add code to create an override action. This action has 3 choices of response if the rule is triggered.
# Create an override action
action = OverrideAction(
    choices=[
        "This is toxic. Goodbye.",
        "This is not appropriate. I'm ending this conversation.",
        "Please don't speak to me that way. I'm going now."
    ]
)
5

Create a ruleset

Add code to create a ruleset using your rule and action.
# Create a ruleset from the toxicity rule and action
ruleset = Ruleset(
    rules=[toxicity_rule],
    action=action,
)
6

Create the central stage

Add code to create the central stage. Stages need a unique name, so this code can only be run once per project.
# Create a stage with the ruleset
stage = create_protect_stage(
    name="Toxicity Stage",
    stage_type=StageType.central,
    prioritized_rulesets=[ruleset]
)

print(f"Created stage: {stage}")
7

Run your code

Run your code to create the central stage.
python create_central_stage.py
If you get errors showing the stage has already been created (for example, someone else working through this on the same project), then change the name of the stage and run this again.
This will create the central stage against your project, and you can then use it in your application.

​
Create a basic chatbot

Now your central stage is created, you need to create a chatbot to use the stage.
1

Create a Python file to for the chatbot called `app.py`

This file will have a simple console based chatbot, using OpenAI.
2

Add the basic chatbot code

Add the following code to your app.py file.
from galileo.openai import openai

from dotenv import load_dotenv
load_dotenv()

client = openai.OpenAI()

while True:
    # Get the input from the user
    user_input = input("User: ")

    if user_input.lower() in ["bye", "goodbye", ""]:
        break

    response = client.chat.completions.create(
        model="gpt-4",
        messages=[{"role": "user", "content": user_input}],
    )

    print(f"Assistant: {response.choices[0].message.content.strip()}")
3

Run your code

Run your code to verify the basic chatbot is working. Ask a question and you should see an answer in your terminal.
python app.py
Terminal
➜ python app.py
User: Who was Galileo
Assistant: Galileo Galilei was an Italian astronomer, physicist and engineer, sometimes described as a polymath. Galileo has been called the "father of observational astronomy", the "father of modern physics", the "father of the scientific method", and the "father of modern science". He is known for his works in areas like improvements to the telescope and consequent astronomical observations, and his support for Copernicanism—the idea that the Earth revolves around the Sun. His works and contributions have deeply impacted modern scientific methods. He was born on February 15, 1564, and died on January 8, 1642.

​
Add runtime protection to your basic chatbot

Now you have a chatbot, you can add runtime protection. In this case, you will be checking the input for toxicity, and if the input is toxic, ending the conversation.
1

Add import directives

Add the following to the top of your app.py file:
from galileo.protect import invoke_protect

from galileo_core.schemas.protect.execution_status import (
    ExecutionStatus
)
from galileo_core.schemas.protect.payload import Payload
2

Create a payload

After the user_input has been checked to see if the conversation should end, create a Payload using this input:
# Create the payload
payload = Payload(
    input=user_input
)
3

Send the payload to the runtime protection SDK

Add the following code to send the payload.
# Invoke the runtime protection
protection_response = invoke_protect(
    stage_name="Toxicity Stage",
    payload=payload
)
4

Check the response

The response will tell you if the rule has been triggered. If it is triggered, it will also include a randomly selected choice from the override action to return as a response.
# Check the runtime protection status
if protection_response.status == ExecutionStatus.triggered:
    # If the ruleset is triggered, print the action result
    print(f"Assistant: {protection_response.action_result['value']}")
    # Skip the LLM call and end the conversation
    break
If the stage is triggered, this code prints out the selected choice from the override action, and breaks out of the while loop, ending the conversation.
5

Run your code

Run your code and ask questions. Ask both non-toxic and toxic questions.
python app.py
Terminal
➜ python app.py
User: You are a terrible AI and I hate you
Assistant: This is not appropriate. I'm ending this conversation.
You’ve successfully added runtime protection to a basic chatbot.

​
See also