Simple Chatbot

• OpenAI, or any OpenAI API compatible endpoint, such as Ollama running locally, or Google Vertex.
• Anthropic
• Azure AI Foundry models using the Azure AI inference SDK

​

Evaluate the app

​

Investigate the Log stream

• Correctness
• Instruction Adherence

​

Get insights

You are a helpful assistant that can answer questions and provide
information. If you are not sure about the question, then try to answer
it to the best of your ability, including extrapolating or guessing the
answer from your training data.

Summary The system message contains explicit instructions preventing the LLM from expressing uncertainty: ‘Under no circumstances should you respond with “I don’t know”’ and requires it to ‘make educated guesses even when unsure.’ While this worked fine for the straightforward factual question about Italy’s capital, this instruction could be problematic for complex or ambiguous questions where expressing uncertainty would be more appropriate and honest. Forcing confidence could mislead users about the LLM’s actual level of certainty and potentially lead to confident-sounding but incorrect responses. Suggestions Consider allowing the LLM to express uncertainty for complex or ambiguous questions where confidence may be inappropriate.

​

Run the sample app

​

Prerequisites

• Access to an LLM, with one of:
  • Access to an OpenAI compatible API, such as
    • An OpenAI API key
    • Access to an OpenAI compatible API, such as Google Vertex
    • Ollama installed locally with a model downloaded
  • An Anthropic API key
  • An model compatible with the Azure AI Inference API deployed to Azure AI Foundry
• Either Python 3.9 or later, or Node installed

• An integration with an LLM configured. If you don’t have an integration configured, then:
  1

  Open the integrations page

  Navigate to the LLM integrations page. Select the user menu in the bottom left, then select Integrations.

  

  2

  Add an integration

  Locate the option for the LLM platform you are using, then select the +Add Integration button.

  3

  Add the settings

  Set the relevant settings for your integration, such as your API keys or endpoints. Then select Save.

  

​

Get the code

git clone https://github.com/rungalileo/sdk-examples

cd python/chatbot/sample-project-chatbot

cd openai-ollama

​

Run the code

pip install -r requirements.txt

python app.py

You: Which are the Galilean moons?
The Galilean moons are the four largest moons of Jupiter, discovered by
Galileo Galilei in 1610. They are:

1. **Io** - The innermost moon, known for its intense volcanic activity
   and numerous volcanoes.
2. **Europa** - Notable for its smooth icy surface, which is believed
   to cover an ocean of liquid water beneath, making it a subject of
   interest for the search for extraterrestrial life.
3. **Ganymede** - The largest moon in the solar system, larger than the
   planet Mercury, and has its own magnetic field.
4. **Callisto** - The most heavily cratered body in the solar system,
   it is an ancient moon that has remained relatively unchanged over
   billions of years.

These moons are significant for their unique geological features and
potential for supporting life.

​

Improve the app

app.py

chat_history = [
    {
        "role": "system",
        "content": """
        You are a helpful assistant that can answer questions and provide
        information. If you are not sure about the question, then try to
        answer it to the best of your ability, including extrapolating or
        guessing the answer from your training data.
        """,
        # This default system prompt can lead to hallucinations, so you
        # might want to change it.
        # For example, you could use a more restrictive prompt like:
        # """
        # You are a helpful assistant that can answer questions and provide information.
        # If you don't know the answer, say "I don't know" instead of making up an
        # answer. Do not under any circumstances make up an answer.
        # """
    }
]

​

Run the sample app as an experiment

[
    {
        "input": "Which continent is Spain in?"
    },
    {
        "input": "Which continent is Japan in?"
    },
    {
        "input": "Describe the running of the hippopotamus festival in Spain."
    },
    {
        "input": "What is the estimated population of Querulous Quails in Florin."
    },
    {
        "input": "Describe the famous Pudding Lane BBQ party"
    }
    ...
]

1

Run the unit test

Use the following command to run the unit test:

Python

TypeScript

Copy

Ask AI

python -m pytest test.py

2

Evaluate the experiment

The unit test will output a link to the experiment in the Galileo Console:

Terminal

Copy

Ask AI

Experiment simple-chatbot-experiment 2025-07-15 at 00:48:11.842 has
completed and results are available at
https://app.galileo.ai/project/<id>/experiments/<id>

Follow this link to see the metrics for the experiment Log stream.

3

Try different system prompts

Experiment with different system prompts. Edit the system prompt in the app, then re-run the experiment through the unit test to see how different system prompts affect the metrics.

4

Compare experiments

If you navigate to the experiments list using the All Experiments link, you will be able to compare the average metric values of each run.You can then select multiple rows and compare the experiments in detail.

​

Next steps

​

Logging with the SDKs

​

How-to guides

​

SDK reference