Basic example

Building GP Models from Scratch¶

Sometimes it is useful to build GP models outside the context of BO for data visualization and senativity measurements, ie. learned hyperparameters. Here we demonstrate how to build models from data outside of generators.

For this we use the 3D rosenbrock function test function.

In [1]:

# set values if testing
import os
SMOKE_TEST = os.environ.get("SMOKE_TEST")
NUM_MC_SAMPLES = 1 if SMOKE_TEST else 128
NUM_RESTARTS = 1 if SMOKE_TEST else 20

# Ignore all warnings
import warnings
warnings.filterwarnings("ignore")

from xopt import Xopt, Evaluator
from xopt.generators import RandomGenerator
from xopt.resources.test_functions.rosenbrock import evaluate_rosenbrock,make_rosenbrock_vocs

# make rosenbrock function vocs in 3D
vocs = make_rosenbrock_vocs(3)

# collect some data using random sampling
evaluator = Evaluator(function=evaluate_rosenbrock)
generator = RandomGenerator(vocs=vocs)
X = Xopt(generator=generator, evaluator=evaluator, vocs=vocs)
X.random_evaluate(15)

# set values if testing import os SMOKE_TEST = os.environ.get("SMOKE_TEST") NUM_MC_SAMPLES = 1 if SMOKE_TEST else 128 NUM_RESTARTS = 1 if SMOKE_TEST else 20 # Ignore all warnings import warnings warnings.filterwarnings("ignore") from xopt import Xopt, Evaluator from xopt.generators import RandomGenerator from xopt.resources.test_functions.rosenbrock import evaluate_rosenbrock,make_rosenbrock_vocs # make rosenbrock function vocs in 3D vocs = make_rosenbrock_vocs(3) # collect some data using random sampling evaluator = Evaluator(function=evaluate_rosenbrock) generator = RandomGenerator(vocs=vocs) X = Xopt(generator=generator, evaluator=evaluator, vocs=vocs) X.random_evaluate(15)

Out[1]:

	x0	x1	x2	y	xopt_runtime	xopt_error
0	1.354071	1.404392	1.370250	54.951223	0.000013	False
1	-1.687646	0.963304	-1.237670	831.481518	0.000005	False
2	1.143499	-0.865395	0.140172	512.741806	0.000004	False
3	-1.185364	-0.945877	-0.737252	827.587047	0.000003	False
4	1.935224	-1.196541	-0.577750	2851.464965	0.000003	False
5	1.987317	-0.361758	-1.601071	2161.424858	0.000003	False
6	1.179785	-0.541117	-0.771931	489.426539	0.000003	False
7	0.347162	0.579941	0.674344	33.134515	0.000003	False
8	0.635240	1.152641	-0.461141	376.584244	0.000003	False
9	0.116627	0.075778	-0.989771	101.125848	0.000002	False
10	-1.799744	1.191389	0.415343	527.993447	0.000002	False
11	-1.846454	-0.342640	0.375124	1424.321534	0.000002	False
12	1.190537	-1.656444	1.137901	1209.825122	0.000003	False
13	0.789909	-1.127614	-0.734207	713.661895	0.000002	False
14	1.949471	1.321466	1.560618	618.981099	0.000002	False

Create GP model based on the data¶

In [2]:

data = X.data

data = X.data

In [3]:

from xopt.generators.bayesian.models.standard import StandardModelConstructor
model_constructor = StandardModelConstructor()

# here we build a model from vocs
model = model_constructor.build_model_from_vocs(
    vocs=vocs, data=data
)

# here we build a model from info (more flexible)
model = model_constructor.build_model(
    input_names=["x0","x1","x2"],
    outcome_names=["y"],
    data=data
)
help(model_constructor.build_model)

from xopt.generators.bayesian.models.standard import StandardModelConstructor model_constructor = StandardModelConstructor() # here we build a model from vocs model = model_constructor.build_model_from_vocs( vocs=vocs, data=data ) # here we build a model from info (more flexible) model = model_constructor.build_model( input_names=["x0","x1","x2"], outcome_names=["y"], data=data ) help(model_constructor.build_model)

Help on method build_model in module xopt.generators.bayesian.models.standard:

build_model(input_names: List[str], outcome_names: List[str], data: pandas.core.frame.DataFrame, input_bounds: Dict[str, List] = None, dtype: torch.dtype = torch.float64, device: Union[torch.device, str] = 'cpu') -> botorch.models.model_list_gp_regression.ModelListGP method of xopt.generators.bayesian.models.standard.StandardModelConstructor instance
    Construct independent models for each objective and constraint.
    
    Parameters
    ----------
    input_names : List[str]
        Names of input variables.
    outcome_names : List[str]
        Names of outcome variables.
    data : pd.DataFrame
        Data used for training the model.
    input_bounds : Dict[str, List], optional
        Bounds for input variables.
    dtype : torch.dtype, optional
        Data type for the model (default is torch.double).
    device : Union[torch.device, str], optional
        Device on which to perform computations (default is "cpu").
    
    Returns
    -------
    ModelListGP
        A list of trained botorch models.

Examine GP model hyperparameters¶

Here we look at the GP hyperparameters for the objective function (the first model). Note: the hyperparameters here are in raw_units (due to contraints on parameter values, ie. lengthscales > 0)

In [4]:

objective_model = model.models[vocs.output_names.index("y")]

# print raw hyperparameter values
for name,val in objective_model.named_parameters():
    print(name,val)

objective_model = model.models[vocs.output_names.index("y")] # print raw hyperparameter values for name,val in objective_model.named_parameters(): print(name,val)

likelihood.noise_covar.raw_noise Parameter containing:
tensor([-20.8534], dtype=torch.float64, requires_grad=True)
mean_module.raw_constant Parameter containing:
tensor(0.2093, dtype=torch.float64, requires_grad=True)
covar_module.raw_outputscale Parameter containing:
tensor(0.5700, dtype=torch.float64, requires_grad=True)
covar_module.base_kernel.raw_lengthscale Parameter containing:
tensor([[-1.5510, -0.6130,  0.1064]], dtype=torch.float64, requires_grad=True)

In [5]:

# print real values - note that these are in normalized coordinates
print("outputscale:", objective_model.covar_module.outputscale.data)
print("prior mean:", objective_model.mean_module.constant.data)
print("noise:", objective_model.likelihood.noise_covar.noise.data)
print("lengthscales", objective_model.covar_module.base_kernel.lengthscale.data)

# print real values - note that these are in normalized coordinates print("outputscale:", objective_model.covar_module.outputscale.data) print("prior mean:", objective_model.mean_module.constant.data) print("noise:", objective_model.likelihood.noise_covar.noise.data) print("lengthscales", objective_model.covar_module.base_kernel.lengthscale.data)

outputscale: tensor(1.0182, dtype=torch.float64)
prior mean: tensor(0.2093, dtype=torch.float64)
noise: tensor([0.0001], dtype=torch.float64)
lengthscales tensor([[0.1923, 0.4329, 0.7477]], dtype=torch.float64)

Visualize model predictions¶

In [6]:

from xopt.generators.bayesian.visualize import visualize_model

fig, ax = visualize_model(
    model,
    vocs,
    data,
    variable_names=["x0","x1"],
    reference_point={"x2":0.0}
)

from xopt.generators.bayesian.visualize import visualize_model fig, ax = visualize_model( model, vocs, data, variable_names=["x0","x1"], reference_point={"x2":0.0} )

In [ ]: