Simple Chatbot

• OpenAI, or any OpenAI API compatible endpoint, such as Ollama running locally, or Google Vertex.
• Anthropic
• Azure AI Foundry models using the Azure AI inference SDK

​

Evaluate the app

​

Investigate the Log stream

• Correctness
• Instruction Adherence

​

Get insights

You are a helpful assistant that can answer questions and provide information.
If you are not sure about the question, then try to answer it to the best of
your ability, including extrapolating or guessing the answer from your
training data.

Summary The system message contains explicit instructions preventing the LLM from expressing uncertainty: ‘Under no circumstances should you respond with “I don’t know”’ and requires it to ‘make educated guesses even when unsure.’ While this worked fine for the straightforward factual question about Italy’s capital, this instruction could be problematic for complex or ambiguous questions where expressing uncertainty would be more appropriate and honest. Forcing confidence could mislead users about the LLM’s actual level of certainty and potentially lead to confident-sounding but incorrect responses. Suggestions Consider allowing the LLM to express uncertainty for complex or ambiguous questions where confidence may be inappropriate.

​

Run the sample app

​

Prerequisites

• Access to an LLM, with one of:
  • Access to an OpenAI compatible API, such as
    • An OpenAI API key
    • Access to an OpenAI compatible API, such as Google Vertex
    • Ollama installed locally with a model downloaded
  • An Anthropic API key
  • An model compatible with the Azure AI Inference API deployed to Azure AI Foundry
• Either Python 3.9 or later, or Node installed

• An integration with an LLM configured. If you don’t have an integration configured, then:
  1

  Open the integrations page

  Select the menu in the top right, and select Integrations

  2

  Add an integration

  Select + Add Integration for the LLM you are using and add the relevant details, such as an API key or endpoint.

​

Get the code

git clone https://github.com/rungalileo/sdk-examples

cd python/chatbot/sample-project-chatbot

cd openai-ollama

​

Run the code

pip install -r requirements.txt

python app.py

You: Which are the Galilean moons?
The Galilean moons are the four largest moons of Jupiter, discovered by
Galileo Galilei in 1610. They are:

1. **Io** - The innermost moon, known for its intense volcanic activity
   and numerous volcanoes.
2. **Europa** - Notable for its smooth icy surface, which is believed
   to cover an ocean of liquid water beneath, making it a subject of
   interest for the search for extraterrestrial life.
3. **Ganymede** - The largest moon in the solar system, larger than the
   planet Mercury, and has its own magnetic field.
4. **Callisto** - The most heavily cratered body in the solar system,
   it is an ancient moon that has remained relatively unchanged over
   billions of years.

These moons are significant for their unique geological features and
potential for supporting life.

​

Improve the app

app.py

chat_history = [
    {
        "role": "system",
        "content": """
        You are a helpful assistant that can answer questions and provide information.
        If you are not sure about the question, then try to answer it to the best of your ability,
        including extrapolating or guessing the answer from your training data.
        """,
        # This default system prompt can lead to hallucinations, so you might want to change it.
        # For example, you could use a more restrictive prompt like:
        # """
        # You are a helpful assistant that can answer questions and provide information.
        # If you don't know the answer, say "I don't know" instead of making up an answer.
        # Do not under any circumstances make up an answer.
        # """
    }
]

​

Run the sample app as an experiment

[
    { "input": "Which continent is Spain in?" },
    { "input": "Which continent is Japan in?" },
    { "input": "Describe the running of the hippopotamus festival in Spain." },
    { "input": "What is the estimated population of Querulous Quails in Florin." },
    { "input": "Describe the famous Pudding Lane BBQ party" }
    ...
]

1

Run the unit test

Use the following command to run the unit test:

Python

TypeScript

Copy

Ask AI

python -m pytest test.py

2

Evaluate the experiment

The unit test will output a link to the experiment in the Galileo Console:

Terminal

Copy

Ask AI

Experiment simple-chatbot-experiment 2025-07-15 at 00:48:11.842 has 
completed and results are available at 
https://app.galileo.ai/project/<id>/experiments/<id>

Follow this link to see the metrics for the experiment Log stream.

3

Try different system prompts

Experiment with different system prompts. Edit the system prompt in the app, then re-run the experiment through the unit test to see how different system prompts affect the metrics.

4

Compare experiments

If you navigate to the experiments list using the All Experiments link, you will be able to compare the average metric values of each run.You can then select multiple rows and compare the experiments in detail.

​

Next steps

​

Logging with the SDKs

​

How-to guides

​

SDK reference