Skip to main content

​
create_experiment

def create_experiment(project_id: Optional[str]=None,
                      experiment_name: Optional[str]=None,
                      project_name: Optional[str]=None) -> ExperimentResponse
Create an experiment with the specified parameters. The project can be specified by providing exactly one of the project name (via the ‘project’ parameter or the GALILEO_PROJECT environment variable) or the project ID (via the ‘project_id’ parameter or the GALILEO_PROJECT_ID environment variable). Arguments
  • project_id: Optional project Id. Takes preference over the GALILEO_PROJECT_ID environment variable. Leave empty if using project
  • experiment_name: Name of the experiment. Required.
  • project: Optional project name. Takes preference over the GALILEO_PROJECT environment variable. Leave empty if using project_id
Raises
  • ValueError: If experiment_name is not provided, or if the project cannot be resolved from project_id or project.
  • HTTPValidationError: If there’s a validation error in returning an ExperimentResponse.
Returns
  • ExperimentResponse: The created experiment response.

​
get_experiment

def get_experiment(project_id: Optional[str]=None,
                   experiment_name: Optional[str]=None,
                   project_name: Optional[str]=None) -> Optional[ExperimentResponse]
Get an experiment with the specified parameters. The project can be specified by providing exactly one of the project name (via the ‘project’ parameter or the GALILEO_PROJECT environment variable) or the project ID (via the ‘project_id’ parameter or the GALILEO_PROJECT_ID environment variable). Arguments
  • project_id: Optional project Id. Takes preference over the GALILEO_PROJECT_ID environment variable. Leave empty if using project
  • experiment_name: Name of the experiment. Required.
  • project_name: Optional project name. Takes preference over the GALILEO_PROJECT environment variable. Leave empty if using project_id
Raises
  • ValueError: If experiment_name is not provided, or if the project cannot be resolved from project_id or project.
  • HTTPValidationError: If there’s a validation error in returning an ExperimentResponse.
Returns
  • ExperimentResponse results or ``None`` if not found.:

​
get_experiments

def get_experiments(project_id: Optional[str]=None,
                    project_name: Optional[str]=None) -> Optional[Union[HTTPValidationError, list[ExperimentResponse]]]
Get experiments from the specified Project. Arguments
  • project_id: Optional project Id. Takes preference over the GALILEO_PROJECT_ID environment variable. Leave empty if using project
  • project_name: Optional project name. Takes preference over the GALILEO_PROJECT environment variable. Leave empty if using project_id
Raises
  • HTTPValidationError: If there’s a validation error in returning a list of ExperimentResponse
Returns
  • List of ExperimentResponse results:

​
run_experiment

def run_experiment(experiment_name: str,
                   *,
                   prompt_template: Optional[PromptTemplate]=None,
                   prompt_settings: Optional[PromptRunSettings]=None,
                   project: Optional[str]=None,
                   project_id: Optional[str]=None,
                   dataset: Optional[Union[Dataset, list[Union[dict[str, Any], str]], str]]=None,
                   dataset_id: Optional[str]=None,
                   dataset_name: Optional[str]=None,
                   metrics: Optional[list[Union[GalileoScorers, Metric, LocalMetricConfig, str]]]=None,
                   function: Optional[Callable]=None,
                   experiment_tags: Optional[dict[str, str]]=None) -> Any
Run an experiment with the specified parameters. There are two ways to run an experiment:
  1. Using a prompt template, prompt settings, and a dataset
  2. Using a runner function and a dataset
When using a runner function, you can also pass a list of dictionaries to the function to act as a dataset. The project can be specified by providing exactly one of the project name (via the ‘project’ parameter or the GALILEO_PROJECT environment variable) or the project ID (via the ‘project_id’ parameter or the GALILEO_PROJECT_ID environment variable). Arguments
  • experiment_name: Name of the experiment
  • prompt_template: Template for prompts
  • prompt_settings: Settings for prompt runs
  • project: Optional project name. Takes preference over the GALILEO_PROJECT environment variable. Leave empty if using project_id
  • project_id: Optional project Id. Takes preference over the GALILEO_PROJECT_ID environment variable. Leave empty if using project
  • dataset: Dataset object, list of records, or dataset name
  • dataset_id: ID of the dataset
  • dataset_name: Name of the dataset
  • metrics: List of metrics to evaluate
  • function: Optional function to run with the experiment
  • experiment_tags: Optional dictionary of key-value pairs to tag the experiment with
Raises
  • ValueError: If required parameters are missing or invalid
Returns
  • Experiment run results:
⌘I