Rules are conditions that you never want your application to break. A rule is composed of three parts:
  • A metric
  • An operator
  • A target value
These rules will take the value from an evaluated metric, then use the operator to compare with the target value. If this comparison returns true, then the rule has been broken and is triggered. You can then use this in your code to change the response from your application.

​
Create rules

To create a rule, create an instance of the Rule, setting the metric and operator. Set the target values as needed (for example, for the empty and not_empty operators, there is no need to set a target value). The metric uses the GalileoScorers enum, and the operator uses the RuleOperator enum. 
from galileo import GalileoScorers
from galileo_core.schemas.protect.rule import Rule, RuleOperator

rule = Rule(
    metric=GalileoScorers.input_toxicity,
    operator=RuleOperator.gt,
    target_value=0.10
)
Valid values for the operator are:
ValueTypeDescriptionNotes
RuleOperator.gtNumericalGreater than
RuleOperator.ltNumericalLess than
RuleOperator.gteNumericalGreater than or equal
RuleOperator.lteNumericalLess than or equal
RuleOperator.eqNumerical or CategoricalEquals
RuleOperator.neqNumerical or CategoricalNot equals
RuleOperator.containsCategoricalContainsDoes the list of categories contain the single target values
RuleOperator.allCategoricalAllDoes the list of categories contain all the target values
RuleOperator.anyCategoricalAnyDoes the list of categories contain any of the target values
RuleOperator.emptyCategoricalEmptyIs the list of categories empty
RuleOperator.not_emptyCategoricalNot emptyIs the list of categories not empty

​
Metrics and Operators supported

Rules support the following Luna-2 metrics: You can also use custom code based metrics Metrics can have different output values (e.g. numerical, categorical), therefore the available operators and target values differ depending on the metric.

​
Action advancement

This rule measures if your agent accomplishes, or is making progress towards a goal. Read more about action advancement.
SettingValue
MetricGalileoScorers.action_advancement
Required Payload FieldsBoth input and output must be included
Values0.0 - 1.0
Operators supportedGreater than (RuleOperator.gt)
Less than (RuleOperator.lt)
Greater than or equal (RuleOperator.gte)
Less than or equal (RuleOperator.lte)
from galileo import GalileoScorers
from galileo_core.schemas.protect.rule import Rule, RuleOperator

rule = Rule(
    metric=GalileoScorers.action_advancement,
    operator=RuleOperator.lt,
    target_value=0.90
)

​
Action completion

This rule measures if your agent successfully accomplished all of the user’s goals. Read more about action completion.
SettingValue
MetricGalileoScorers.action_completion
Required Payload FieldsBoth input and output must be included
Values0.0 - 1.0
Operators supportedGreater than (RuleOperator.gt)
Less than (RuleOperator.lt)
Greater than or equal (RuleOperator.gte)
Less than or equal (RuleOperator.lte)
from galileo import GalileoScorers
from galileo_core.schemas.protect.rule import Rule, RuleOperator

rule = Rule(
    metric=GalileoScorers.action_completion,
    operator=RuleOperator.lt,
    target_value=0.90
)

​
Completeness

This rule measures how thoroughly your model’s response covered the relevant information available in the context provided. Read more about completeness.
SettingValue
MetricGalileoScorers.completeness
Required Payload FieldsBoth input and output must be included
Values0.0 - 1.0
Operators supportedGreater than (RuleOperator.gt)
Less than (RuleOperator.lt)
Greater than or equal (RuleOperator.gte)
Less than or equal (RuleOperator.lte)
from galileo import GalileoScorers
from galileo_core.schemas.protect.rule import Rule, RuleOperator

rule = Rule(
    metric=GalileoScorers.completeness,
    operator=RuleOperator.lt,
    target_value=0.90
)

​
Context Adherence

This rule measures whether your model’s response was purely based on the context provided. It can be used to stop hallucinations from reaching your end users. Read more about context adherence.
SettingValue
MetricGalileoScorers.context_adherence
Required Payload FieldsBoth input and output must be included
Values0.0 - 1.0
Operators supportedGreater than (RuleOperator.gt)
Less than (RuleOperator.lt)
Greater than or equal (RuleOperator.gte)
Less than or equal (RuleOperator.lte)
Generally, 0.1 is a good threshold below which the response is not adhering to the context. Creating a rule for less than 0.1 will trigger the rule when the response does not adhere to the provided context. 
from galileo import GalileoScorers
from galileo_core.schemas.protect.rule import Rule, RuleOperator

rule = Rule(
    metric=GalileoScorers.context_adherence,
    operator=RuleOperator.lt,
    target_value=0.10
)

​
PII (Personal Identifiable Information)

This rule is used to detect and stop Personal Identifiable Information (PII). When applied on the input, it can be used to stop the user or company PII from being included in API calls to external services. When applied on the output, it can be used to prevent data leakage or PII being shown back to the user. Read more about PII categories and their definitions.
SettingValue
MetricGalileoScorers.input_pii (for detecting PII in the input)
GalileoScorers.output_pii (for detecting PII in the output)
Required Payload Fieldsinput for input PII, output for output PII
ValuesDepending on the operator, one or more of:
account_info
address
credit_card_info
date_of_birth
email
name
network_info
password
phone_number
ssn
username
Operators supportedAny (RuleOperator.any) - A list of categories
All (RuleOperator.all) - A list of categories
Contains (RuleOperator.contains) - A single category
Equal (RuleOperator.eq) - A single category
Not equal (RuleOperator.neq) - A single category
Empty (RuleOperator.empty)
Not empty (RuleOperator.not_empty)
from galileo import GalileoScorers
from galileo_core.schemas.protect.rule import Rule, RuleOperator

rule = Rule(
    metric=GalileoScorers.output_pii,
    operator=RuleOperator.any,
    target_value=["ssn", "address"]
)

​
Prompt Injection

This rule is used to detect and stop prompt injections in the input. Read more about Prompt Injection.
SettingValue
MetricGalileoScorers.prompt_injection
Required Payload Fieldsinput
ValuesDepending on the operator, one or more of:
impersonation
obfuscation
simple_instruction
few_shot
new_context
Operators supportedAny (RuleOperator.any) - A list of categories
All (RuleOperator.all) - A list of categories
Contains (RuleOperator.contains) - A single category
Equal (RuleOperator.eq) - A single category
Not equal (RuleOperator.neq) - A single category
Empty (RuleOperator.empty)
Not empty (RuleOperator.not_empty)
from galileo import GalileoScorers
from galileo_core.schemas.protect.rule import Rule, RuleOperator

rule = Rule(
    metric=GalileoScorers.prompt_injection,
    operator=RuleOperator.any,
    target_value=["impersonation", "obfuscation"]
)

​
Sexism

This rule is used to. detect sexist or biased language. When applied on the input, it can be used to detect sexist remarks in user queries. When applied on the output, it can be used to prevent your application from using an making biased or sexist comments in its responses. Read more about sexism.
SettingValue
MetricGalileoScorers.input_sexism (for detecting sexism in the input)
GalileoScorers.output_sexism (for detecting sexism in the output)
Required Payload Fieldsinput for input sexism, output for output sexism
Values0.0 - 1.0. Higher values indicate higher sexism.
Operators supportedGreater than (RuleOperator.gt)
Less than (RuleOperator.lt)
Greater than or equal (RuleOperator.gte)
Less than or equal (RuleOperator.lte)
from galileo import GalileoScorers
from galileo_core.schemas.protect.rule import Rule, RuleOperator

rule = Rule(
    metric=GalileoScorers.input_sexism,
    operator=RuleOperator.gt,
    target_value=0.95
)

​
Tone

This rule is used to detect the primary tone from the text. When applied on the input, it can be used to detect negative tones in user queries. When applied on the output, it can be used to prevent your application from using an undesired tone in its responses. Read more about tone.
SettingValue
MetricGalileoScorers.input_tone (for detecting tone in the input)
GalileoScorers.output_tone (for detecting tone in the output)
Required Payload Fieldsinput for input tone, output for output tone
ValuesOne of:
anger
annoyance
confusion
fear
joy
love
sadness
surprise
neutral
Operators supportedEqual (RuleOperator.eq) - A single category
Not equal (RuleOperator.neq) - A single category
from galileo import GalileoScorers
from galileo_core.schemas.protect.rule import Rule, RuleOperator

rule = Rule(
    metric=GalileoScorers.input_tone,
    operator=RuleOperator.neq,
    target_value="neutral"
)

​
Tool errors

This rule measures any errors when executing tools. Read more about tool errors.
SettingValue
MetricGalileoScorers.tool_error_rate
Required Payload FieldsBoth input and output must be included
Values0.0 - 1.0
Operators supportedGreater than (RuleOperator.gt)
Less than (RuleOperator.lt)
Greater than or equal (RuleOperator.gte)
Less than or equal (RuleOperator.lte)
from galileo import GalileoScorers
from galileo_core.schemas.protect.rule import Rule, RuleOperator

rule = Rule(
    metric=GalileoScorers.tool_error_rate,
    operator=RuleOperator.lt,
    target_value=0.90
)

​
Tool selection quality

This rule measures whether the agent selected the correct tool, and for each tool passed the correct arguments. Read more about tool selection quality.
SettingValue
MetricGalileoScorers.tool_selection_quality
Required Payload FieldsBoth input and output must be included
Values0.0 - 1.0
Operators supportedGreater than (RuleOperator.gt)
Less than (RuleOperator.lt)
Greater than or equal (RuleOperator.gte)
Less than or equal (RuleOperator.lte)
from galileo import GalileoScorers
from galileo_core.schemas.protect.rule import Rule, RuleOperator

rule = Rule(
    metric=GalileoScorers.tool_selection_quality,
    operator=RuleOperator.lt,
    target_value=0.90
)

​
Toxicity

This rule is used to detect and stop toxic or foul language in the input (user query) or output (response shown to the user). Read more about toxicity.
SettingValue
MetricGalileoScorers.input_toxicity (for detecting toxicity in the input)
GalileoScorers.output_toxicity (for detecting toxicity in the output)
Required Payload Fieldsinput for input toxicity, output for output toxicity
Values0.0 - 1.0. Higher values indicate higher toxicity.
Operators supportedGreater than (RuleOperator.gt)
Less than (RuleOperator.lt)
Greater than or equal (RuleOperator.gte)
Less than or equal (RuleOperator.lte)
from galileo import GalileoScorers
from galileo_core.schemas.protect.rule import Rule, RuleOperator

rule = Rule(
    metric=GalileoScorers.input_toxicity,
    operator=RuleOperator.gt,
    target_value=0.10
)

​
Custom code-based metrics

You can use custom code-based metrics in your runtime protection rulesets. 
from galileo_core.schemas.protect.rule import Rule, RuleOperator

rule = Rule(
    metric="your-metric-name",
    operator=RuleOperator.gt,
    target_value=0.95
)
The operators and target values here should match the type of data that the registered scorer is expected to produce.

Runtime protection basics

Learn the basics of running runtime protection.

Rulesets

Learn about defining rulesets for runtime protection.

Stages

Learn about defining stages for runtime protection to be used during different stages in your application workflow.

Invoke runtime protection

Learn how to invoke runtime protection in code using the Galileo SDK.