diff --git a/dev/.documenter-siteinfo.json b/dev/.documenter-siteinfo.json
index b6ca0be..a0862db 100644
--- a/dev/.documenter-siteinfo.json
+++ b/dev/.documenter-siteinfo.json
@@ -1 +1 @@
-{"documenter":{"julia_version":"1.10.4","generation_timestamp":"2024-07-08T12:50:17","documenter_version":"1.5.0"}}
\ No newline at end of file
+{"documenter":{"julia_version":"1.10.4","generation_timestamp":"2024-07-08T13:22:33","documenter_version":"1.5.0"}}
\ No newline at end of file
diff --git a/dev/docs/index.html b/dev/docs/index.html
index acd5308..384fb09 100644
--- a/dev/docs/index.html
+++ b/dev/docs/index.html
@@ -1,29 +1,29 @@
Documentation · BOSS.jl
The main function bo!(::BossProblem; kwargs...) performs the Bayesian optimization. It augments the dataset and updates the model parameters and/or hyperparameters.
Run the Bayesian optimization procedure to solve the given optimization problem or give a recommendation for the next evaluation point if problem.f == missing.
Arguments
problem::BossProblem: Defines the optimization problem.
Keywords
model_fitter::ModelFitter: Defines the algorithm used to estimate model parameters.
acq_maximizer::AcquisitionMaximizer: Defines the algorithm used to maximize the acquisition function.
acquisition::AcquisitionFunction: Defines the acquisition function maximized to select promising candidates for further evaluation.
term_cond::TermCond: Defines the termination condition.
options::BossOptions: Defines miscellaneous options and hyperparameters.
The BossProblem structure contains the whole problem definition, the model definition, and the data together with the current parameter and hyperparameter values.
Defines the whole optimization problem for the BOSS algorithm.
Problem Definition
There is some (noisy) blackbox function `y = f(x) = f_true(x) + ϵ` where `ϵ ~ Normal`.
+x = bo!(problem::BossProblem{Missing}; kwargs...)
Run the Bayesian optimization procedure to solve the given optimization problem or give a recommendation for the next evaluation point if problem.f == missing.
Arguments
problem::BossProblem: Defines the optimization problem.
Keywords
model_fitter::ModelFitter: Defines the algorithm used to estimate model parameters.
acq_maximizer::AcquisitionMaximizer: Defines the algorithm used to maximize the acquisition function.
acquisition::AcquisitionFunction: Defines the acquisition function maximized to select promising candidates for further evaluation.
term_cond::TermCond: Defines the termination condition.
options::BossOptions: Defines miscellaneous options and hyperparameters.
The BossProblem structure contains the whole problem definition, the model definition, and the data together with the current parameter and hyperparameter values.
Defines the whole optimization problem for the BOSS algorithm.
Problem Definition
There is some (noisy) blackbox function `y = f(x) = f_true(x) + ϵ` where `ϵ ~ Normal`.
We have some surrogate model `y = model(x) ≈ f_true(x)`
describing our knowledge (or lack of it) about the blackbox function.
We wish to find `x ∈ domain` such that `fitness(f(x))` is maximized
-while satisfying the constraints `f(x) <= y_max`.
Keywords
fitness::Fitness: The fitness function. See Fitness.
f::Union{Function, Missing}: The objective blackbox function.
The Fitness type is used to define the fitness function $\text{fit}(y) \rightarrow \mathbb{R}$.
The NoFitness can be used in problems without defined fitness (such as active learning problems). It is the default option used if no fitness is provided to BossProblem. The NoFitness can only be used with AcquisitionFunction that does not require fitness.
The LinFitness can be used to define a simple linear fitness function
\[\text{fit}(y) = \alpha^T y \;.\]
Using LinFitness instead of NonlinFitness may allow for simpler/faster computation of some acquisition functions.
The NonlinFitness can be used to define an arbitrary fitness function
Used to define a linear fitness function measuring the quality of an output y of the objective function.
May provide better performance than the more general NonlinFitness as some acquisition functions can be calculated analytically with linear fitness functions whereas this may not be possible with a nonlinear fitness function.
The Fitness type is used to define the fitness function $\text{fit}(y) \rightarrow \mathbb{R}$.
The NoFitness can be used in problems without defined fitness (such as active learning problems). It is the default option used if no fitness is provided to BossProblem. The NoFitness can only be used with AcquisitionFunction that does not require fitness.
The LinFitness can be used to define a simple linear fitness function
\[\text{fit}(y) = \alpha^T y \;.\]
Using LinFitness instead of NonlinFitness may allow for simpler/faster computation of some acquisition functions.
The NonlinFitness can be used to define an arbitrary fitness function
Used to define a linear fitness function measuring the quality of an output y of the objective function.
May provide better performance than the more general NonlinFitness as some acquisition functions can be calculated analytically with linear fitness functions whereas this may not be possible with a nonlinear fitness function.
bounds::AbstractBounds: The basic box-constraints on x. This field is mandatory.
discrete::AbstractVector{<:Bool}: Can be used to designate some dimensions of the domain as discrete.
cons::Union{Nothing, Function}: Used to define arbitrary nonlinear constraints on x. Feasible points x must satisfy all(cons(x) .> 0.). An appropriate acquisition maximizer which can handle nonlinear constraints must be used if cons is provided. (See AcquisitionMaximizer.)
Constraints on output vector y can be defined using the y_max field. Providing y_max to BossProblem defines the linear constraints y < y_max.
Arbitrary nonlinear constraints can be defined by augmenting the objective function. For example to define the constraint y[1] * y[2] < c, one can define an augmented objective function
function f_c(x)
+\end{aligned}\]BOSS.Domain — Type
Domain(; kwargs...)
Describes the optimization domain.
Keywords
bounds::AbstractBounds: The basic box-constraints on x. This field is mandatory.
discrete::AbstractVector{<:Bool}: Can be used to designate some dimensions of the domain as discrete.
cons::Union{Nothing, Function}: Used to define arbitrary nonlinear constraints on x. Feasible points x must satisfy all(cons(x) .> 0.). An appropriate acquisition maximizer which can handle nonlinear constraints must be used if cons is provided. (See AcquisitionMaximizer.)
Constraints on output vector y can be defined using the y_max field. Providing y_max to BossProblem defines the linear constraints y < y_max.
Arbitrary nonlinear constraints can be defined by augmenting the objective function. For example to define the constraint y[1] * y[2] < c, one can define an augmented objective function
function f_c(x)
y = f(x) # the original objective function
y_c = [y..., y[1] * y[2]]
return y_c
-end
and use
y_max = [fill(Inf, y_dim)..., c]
where y_dim is the output dimension of the original objective function. Note that defining nonlinear constraints this way increases the output dimension of the objective function and thus the model definition has to be modified accordingly.
model_posterior(model::CustomModel, data::ExperimentDataMAP; split::Bool) -> (x -> mean, std) <or> [(x -> mean_i, std_i) for i in 1:y_dim]
model_posterior(model::CustomModel, data::ExperimentDataBI; split::Bool) -> [(x -> mean, std) for each sample] <or> [[(x -> mean_i, std_i) for i in 1:y_dim] for each sample]
The LinModel and NonlinModel structures are used to define parametric models. (Some compuatations are simpler/faster with linear model, so the LinModel might provide better performance in the future. This functionality is not implemented yet, so using the NonlinModel is equiavalent for now.)
A parametric surrogate model linear in its parameters.
This model definition will provide better performance than the more general 'NonlinModel' in the future. This feature is not implemented yet so it is equivalent to using NonlinModel for now.
The linear model is defined as
ϕs = lift(x)
+end
and use
y_max = [fill(Inf, y_dim)..., c]
where y_dim is the output dimension of the original objective function. Note that defining nonlinear constraints this way increases the output dimension of the objective function and thus the model definition has to be modified accordingly.
model_posterior(model::CustomModel, data::ExperimentDataMAP; split::Bool) -> (x -> mean, std) <or> [(x -> mean_i, std_i) for i in 1:y_dim]
model_posterior(model::CustomModel, data::ExperimentDataBI; split::Bool) -> [(x -> mean, std) for each sample] <or> [[(x -> mean_i, std_i) for i in 1:y_dim] for each sample]
The LinModel and NonlinModel structures are used to define parametric models. (Some compuatations are simpler/faster with linear model, so the LinModel might provide better performance in the future. This functionality is not implemented yet, so using the NonlinModel is equiavalent for now.)
A parametric surrogate model linear in its parameters.
This model definition will provide better performance than the more general 'NonlinModel' in the future. This feature is not implemented yet so it is equivalent to using NonlinModel for now.
lift::Function: Defines the lift function (::Vector{<:Real}) -> (::Vector{Vector{<:Real}}) according to the definition above.
param_priors::AbstractVector{<:UnivariateDistribution}: The prior distributions for the parameters [θ₁₁, ..., θ₁ₚ, ..., θₘ₁, ..., θₘₚ] according to the definition above.
discrete::Union{Nothing, <:AbstractVector{<:Bool}}: Describes which dimensions are discrete. Typically, the discrete kwarg can be ignored by the end-user as it will be updated according to the Domain of the BossProblem during BOSS initialization.
If your model is linear, you can use LinModel which will provide better performance in the future. (Not yet implemented.)
Define the model as y = predict(x, θ) where θ are the model parameters.
Keywords
predict::Function: The predict function according to the definition above.
param_priors::AbstractVector{<:UnivariateDistribution}: The prior distributions for the model parameters.
discrete::Union{Nothing, <:AbstractVector{<:Bool}}: Describes which dimensions are discrete. Typically, the discrete kwarg can be ignored by the end-user as it will be updated according to the Domain of the BossProblem during BOSS initialization.
mean::Union{Nothing, Function}: Used as the mean function for the GP. Defaults to nothing equivalent to x -> [0.].
kernel::Kernel: The kernel used in the GP. Defaults to the Matern32Kernel().
amp_priors::AbstractVector{<:UnivariateDistribution}: The prior distributions for the amplitude hyperparameters of the GP. The amp_priors should be a vector of y_dim univariate distributions.
length_scale_priors::AbstractVector{<:MultivariateDistribution}: The prior distributions for the length scales of the GP. The length_scale_priors should be a vector of y_dimx_dim-variate distributions where x_dim and y_dim are the dimensions of the input and output of the model respectively.
The data from all past objective function evaluations as well as estimated parameter and/or hyperparameter values of the surrogate model are stored in the ExperimentData types.
The ExperimentDataPost types contain the estimated model (hyper)parameters in addition to the dataset. The ExperimentDataMAP structure contains the MAP estimate of the parameters in case a MAP model fitter is used, and the ExperimentDataBI structure contains samples of the parameters in case a Bayesian inference model fitter is used.
Stores the data matrices X,Y as well as the optimized model parameters and hyperparameters.
Fields
X::AbstractMatrix{<:Real}: Contains the objective function inputs as columns.
Y::AbstractMatrix{<:Real}: Contains the objective function outputs as columns.
θ::Union{Nothing, <:AbstractVector{<:Real}}: Contains the MAP parameters of the parametric model (or nothing if the model is nonparametric).
length_scales::Union{Nothing, <:AbstractMatrix{<:Real}}: Contains the MAP length scales of the GP as a x_dim×y_dim matrix (or nothing if the model is parametric).
amplitudes::Union{Nothing, <:AbstractVector{<:Real}}: Amplitudes of the GP.
noise_std::AbstractVector{<:Real}: The MAP noise standard deviations of each y dimension.
consistent::Bool: True iff the parameters (θ, length_scales, amplitudes, noise_std) have been fitted using the current dataset (X, Y). Is set to consistent = false after updating the dataset, and to consistent = true after re-fitting the parameters.
Stores the data matrices X,Y as well as the sampled model parameters and hyperparameters.
Fields
X::AbstractMatrix{<:Real}: Contains the objective function inputs as columns.
Y::AbstractMatrix{<:Real}: Contains the objective function outputs as columns.
θ::Union{Nothing, <:AbstractVector{<:AbstractVector{<:Real}}}: Samples of parameters of the parametric model stored column-wise in a matrix (or nothing if the model is nonparametric).
length_scales::Union{Nothing, <:AbstractVector{<:AbstractMatrix{<:Real}}}: Samples of the length scales of the GP as a vector of x_dim×y_dim matrices (or nothing if the model is parametric).
amplitudes::Union{Nothing, <:AbstractVector{<:AbstractVector{<:Real}}}: Samples of the amplitudes of the GP.
noise_std::AbstractVector{<:AbstractVector{<:Real}}: Samples of the noise standard deviations of each y dimension stored column-wise in a matrix.
consistent::Bool: True iff the parameters (θ, length_scales, amplitudes, noise_std) have been fitted using the current dataset (X, Y). Is set to consistent = false after updating the dataset, and to consistent = true after re-fitting the parameters.
Specifies the library/algorithm used for model parameter estimation. Inherit this type to define a custom model-fitting algorithms.
Example: struct CustomFitter <: ModelFitter{MAP} ... end or struct CustomFitter <: ModelFitter{BI} ... end
Structures derived from this type have to implement the following method: estimate_parameters(model_fitter::CustomFitter, problem::BossProblem; info::Bool).
This method should return a named tuple (θ = ..., length_scales = ..., noise_std = ...) with either MAP model parameters (if CustomAlg <: ModelFitter{MAP}) or model parameter samples (if CustomAlg <: ModelFitter{BI}).
The OptimizationMAP model fitter can be used to utilize any optimization algorithm from the Optimization.jl package in order to find the MAP estimate of the (hyper)parameters. (See the example usage.)
Finds the MAP estimate of the model parameters and hyperparameters using the Optimization.jl package.
Keywords
algorithm::Any: Defines the optimization algorithm.
multistart::Int: The number of optimization restarts.
parallel::Bool: If parallel=true then the individual restarts are run in parallel.
softplus_hyperparams::Bool: If softplus_hyperparams=true then the softplus function is applied to GP hyperparameters (length-scales & amplitudes) and noise deviations to ensure positive values during optimization.
softplus_params::Union{Bool, Vector{Bool}}: Defines to which parameters of the parametric model should the softplus function be applied to ensure positive values. Supplying a boolean instead of a binary vector turns the softplus on/off for all parameters. Defaults to false meaning the softplus is applied to no parameters.
The TuringBI model fitter can be used to utilize the Turing.jl library in order to sample the (hyper)parameters from the posterior given by the current dataset.
The SamplingMAP model fitter preforms MAP estimation by sampling the parameters from their priors and maximizing the posterior probability over the samples. This is a trivial model fitter suitable for simple experimentation with BOSS.jl and/or Bayesian optimization. A more sophisticated model fitter such as OptimizationMAP or TuringBI should be used to solve real problems.
The RandomMAP model fitter samples random parameter values from their priors. It does NOT optimize for the most probable parameters in any way. This model fitter is provided solely for easy experimentation with BOSS.jl and should not be used to solve problems.
Specifies the library/algorithm used for acquisition function optimization. Inherit this type to define a custom acquisition maximizer.
Example: struct CustomAlg <: AcquisitionMaximizer ... end
Structures derived from this type have to implement the following method: maximize_acquisition(acq_maximizer::CustomAlg, acq::AcquisitionFunction, problem::BossProblem, options::BossOptions) This method should return the point of the input domain which maximizes the given acquisition function acq (as a vector) or a batch of points (as a column-wise matrix).
The GridAM maximizes the acquisition function by evaluating all points on a fixed grid of points. This is a trivial acquisition maximizer suitable for simple experimentation with BOSS.jl and/or Bayesian optimization. More sophisticated acquisition maximizers such as OptimizationAM should be used to solve real problems.
The RandomAM simply returns a random point. It does NOT perform any optimization. This acquisition maximizer is provided solely for easy experimentation with BOSS.jl and should not be used to solve problems.
Selects a random interior point instead of maximizing the acquisition function. Can be used for method comparison.
Can handle constraints on x, but does so by generating random points in the box domain until a point satisfying the constraints is found. Therefore it can take a long time or even get stuck if the constraints are very tight.
The SequentialBatchAM can be used as a wrapper of any of the other acquisition maximizers. It returns a batch of promising points for future evaluations instead of a single point, and thus allows for evaluation of the objective function in batches.
Specifies the acquisition function describing the "quality" of a potential next evaluation point. Inherit this type to define a custom acquisition function.
Example: struct CustomAcq <: AcquisitionFunction ... end
Structures derived from this type have to implement the following method: (acquisition::CustomAcq)(problem::BossProblem, options::BossOptions)
This method should return a function acq(x::AbstractVector{<:Real}) = val::Real, which is maximized to select the next evaluation function of blackbox function in each iteration.
The expected improvement (EI) acquisition function.
Fitness function must be defined as a part of the problem definition in order to use EI. (See Fitness.)
Measures the quality of a potential evaluation point x as the expected improvement in best-so-far achieved fitness by evaluating the objective function at y = f(x).
In case of constrained problems, the expected improvement is additionally weighted by the probability of feasibility of y. I.e. the probability that all(cons(y) .> 0.).
If the problem is constrained on y and no feasible point has been observed yet, then the probability of feasibility alone is returned as the acquisition function.
Rather than using the actual evaluations (xᵢ,yᵢ) from the dataset, the best-so-far achieved fitness is calculated as the maximum fitness among the means ̂yᵢ of the posterior predictive distribution of the model evaluated at xᵢ. This is a simple way to handle evaluation noise which may not be suitable for problems with substantial noise. In case of Bayesian Inference, an averaged posterior of the model posterior samples is used for the prediction of ŷᵢ.
Keywords
ϵ_samples::Int: Controls how many samples are used to approximate EI. The ϵ_samples keyword is ignored unless MAP model fitter and NonlinFitness are used! In case of BI model fitter, the number of samples is instead set equal to the number of posterior samples. In case of LinearFitness, the expected improvement can be calculated analytically.
cons_safe::Bool: If set to true, the acquisition function acq(x) is made 'constraint-safe' by checking the bounds and constraints during each evaluation. Set cons_safe to true if the evaluation of the model at exterior points may cause errors or nonsensical values. You may set cons_safe to false if the evaluation of the model at exterior points can provide useful information to the acquisition maximizer and does not cause errors. Defaults to true.
Returns the point (x, y) from the dataset of the given problem such that y satisfies the constraints and fitness(y) is maximized. Returns nothing if the dataset is empty or if no feasible point is present.
Does not check whether x belongs to the domain as no exterior points should be present in the dataset.
Stores miscellaneous settings of the BOSS algorithm.
Keywords
info::Bool: Setting info=false silences the BOSS algorithm.
debug::Bool: Set debug=true to print stactraces of caught optimization errors.
parallel_evals::Symbol: Possible values: :serial, :parallel, :distributed. Defaults to :parallel. Determines whether to run multiple objective function evaluations within one batch in serial, parallel, or distributed fashion. (Only has an effect if batching AM is used.)
callback::BossCallback: If provided, callback(::BossProblem; kwargs...) will be called before the BO procedure starts and after every iteration.
If an object cb of type BossCallback is passed to BossOptions, the method shown below will be called before the BO procedure starts and after every iteration.
lift::Function: Defines the lift function (::Vector{<:Real}) -> (::Vector{Vector{<:Real}}) according to the definition above.
param_priors::AbstractVector{<:UnivariateDistribution}: The prior distributions for the parameters [θ₁₁, ..., θ₁ₚ, ..., θₘ₁, ..., θₘₚ] according to the definition above.
discrete::Union{Nothing, <:AbstractVector{<:Bool}}: Describes which dimensions are discrete. Typically, the discrete kwarg can be ignored by the end-user as it will be updated according to the Domain of the BossProblem during BOSS initialization.
If your model is linear, you can use LinModel which will provide better performance in the future. (Not yet implemented.)
Define the model as y = predict(x, θ) where θ are the model parameters.
Keywords
predict::Function: The predict function according to the definition above.
param_priors::AbstractVector{<:UnivariateDistribution}: The prior distributions for the model parameters.
discrete::Union{Nothing, <:AbstractVector{<:Bool}}: Describes which dimensions are discrete. Typically, the discrete kwarg can be ignored by the end-user as it will be updated according to the Domain of the BossProblem during BOSS initialization.
mean::Union{Nothing, Function}: Used as the mean function for the GP. Defaults to nothing equivalent to x -> [0.].
kernel::Kernel: The kernel used in the GP. Defaults to the Matern32Kernel().
amp_priors::AbstractVector{<:UnivariateDistribution}: The prior distributions for the amplitude hyperparameters of the GP. The amp_priors should be a vector of y_dim univariate distributions.
length_scale_priors::AbstractVector{<:MultivariateDistribution}: The prior distributions for the length scales of the GP. The length_scale_priors should be a vector of y_dimx_dim-variate distributions where x_dim and y_dim are the dimensions of the input and output of the model respectively.
The data from all past objective function evaluations as well as estimated parameter and/or hyperparameter values of the surrogate model are stored in the ExperimentData types.
The ExperimentDataPost types contain the estimated model (hyper)parameters in addition to the dataset. The ExperimentDataMAP structure contains the MAP estimate of the parameters in case a MAP model fitter is used, and the ExperimentDataBI structure contains samples of the parameters in case a Bayesian inference model fitter is used.
Stores the data matrices X,Y as well as the optimized model parameters and hyperparameters.
Fields
X::AbstractMatrix{<:Real}: Contains the objective function inputs as columns.
Y::AbstractMatrix{<:Real}: Contains the objective function outputs as columns.
θ::Union{Nothing, <:AbstractVector{<:Real}}: Contains the MAP parameters of the parametric model (or nothing if the model is nonparametric).
length_scales::Union{Nothing, <:AbstractMatrix{<:Real}}: Contains the MAP length scales of the GP as a x_dim×y_dim matrix (or nothing if the model is parametric).
amplitudes::Union{Nothing, <:AbstractVector{<:Real}}: Amplitudes of the GP.
noise_std::AbstractVector{<:Real}: The MAP noise standard deviations of each y dimension.
consistent::Bool: True iff the parameters (θ, length_scales, amplitudes, noise_std) have been fitted using the current dataset (X, Y). Is set to consistent = false after updating the dataset, and to consistent = true after re-fitting the parameters.
Stores the data matrices X,Y as well as the sampled model parameters and hyperparameters.
Fields
X::AbstractMatrix{<:Real}: Contains the objective function inputs as columns.
Y::AbstractMatrix{<:Real}: Contains the objective function outputs as columns.
θ::Union{Nothing, <:AbstractVector{<:AbstractVector{<:Real}}}: Samples of parameters of the parametric model stored column-wise in a matrix (or nothing if the model is nonparametric).
length_scales::Union{Nothing, <:AbstractVector{<:AbstractMatrix{<:Real}}}: Samples of the length scales of the GP as a vector of x_dim×y_dim matrices (or nothing if the model is parametric).
amplitudes::Union{Nothing, <:AbstractVector{<:AbstractVector{<:Real}}}: Samples of the amplitudes of the GP.
noise_std::AbstractVector{<:AbstractVector{<:Real}}: Samples of the noise standard deviations of each y dimension stored column-wise in a matrix.
consistent::Bool: True iff the parameters (θ, length_scales, amplitudes, noise_std) have been fitted using the current dataset (X, Y). Is set to consistent = false after updating the dataset, and to consistent = true after re-fitting the parameters.
Specifies the library/algorithm used for model parameter estimation. Inherit this type to define a custom model-fitting algorithms.
Example: struct CustomFitter <: ModelFitter{MAP} ... end or struct CustomFitter <: ModelFitter{BI} ... end
Structures derived from this type have to implement the following method: estimate_parameters(model_fitter::CustomFitter, problem::BossProblem; info::Bool).
This method should return a named tuple (θ = ..., length_scales = ..., noise_std = ...) with either MAP model parameters (if CustomAlg <: ModelFitter{MAP}) or model parameter samples (if CustomAlg <: ModelFitter{BI}).
The OptimizationMAP model fitter can be used to utilize any optimization algorithm from the Optimization.jl package in order to find the MAP estimate of the (hyper)parameters. (See the example usage.)
Finds the MAP estimate of the model parameters and hyperparameters using the Optimization.jl package.
Keywords
algorithm::Any: Defines the optimization algorithm.
multistart::Int: The number of optimization restarts.
parallel::Bool: If parallel=true then the individual restarts are run in parallel.
softplus_hyperparams::Bool: If softplus_hyperparams=true then the softplus function is applied to GP hyperparameters (length-scales & amplitudes) and noise deviations to ensure positive values during optimization.
softplus_params::Union{Bool, Vector{Bool}}: Defines to which parameters of the parametric model should the softplus function be applied to ensure positive values. Supplying a boolean instead of a binary vector turns the softplus on/off for all parameters. Defaults to false meaning the softplus is applied to no parameters.
The TuringBI model fitter can be used to utilize the Turing.jl library in order to sample the (hyper)parameters from the posterior given by the current dataset.
The SamplingMAP model fitter preforms MAP estimation by sampling the parameters from their priors and maximizing the posterior probability over the samples. This is a trivial model fitter suitable for simple experimentation with BOSS.jl and/or Bayesian optimization. A more sophisticated model fitter such as OptimizationMAP or TuringBI should be used to solve real problems.
The RandomMAP model fitter samples random parameter values from their priors. It does NOT optimize for the most probable parameters in any way. This model fitter is provided solely for easy experimentation with BOSS.jl and should not be used to solve problems.
Specifies the library/algorithm used for acquisition function optimization. Inherit this type to define a custom acquisition maximizer.
Example: struct CustomAlg <: AcquisitionMaximizer ... end
Structures derived from this type have to implement the following method: maximize_acquisition(acq_maximizer::CustomAlg, acq::AcquisitionFunction, problem::BossProblem, options::BossOptions) This method should return the point of the input domain which maximizes the given acquisition function acq (as a vector) or a batch of points (as a column-wise matrix).
The GridAM maximizes the acquisition function by evaluating all points on a fixed grid of points. This is a trivial acquisition maximizer suitable for simple experimentation with BOSS.jl and/or Bayesian optimization. More sophisticated acquisition maximizers such as OptimizationAM should be used to solve real problems.
The RandomAM simply returns a random point. It does NOT perform any optimization. This acquisition maximizer is provided solely for easy experimentation with BOSS.jl and should not be used to solve problems.
Selects a random interior point instead of maximizing the acquisition function. Can be used for method comparison.
Can handle constraints on x, but does so by generating random points in the box domain until a point satisfying the constraints is found. Therefore it can take a long time or even get stuck if the constraints are very tight.
The SequentialBatchAM can be used as a wrapper of any of the other acquisition maximizers. It returns a batch of promising points for future evaluations instead of a single point, and thus allows for evaluation of the objective function in batches.
Specifies the acquisition function describing the "quality" of a potential next evaluation point. Inherit this type to define a custom acquisition function.
Example: struct CustomAcq <: AcquisitionFunction ... end
Structures derived from this type have to implement the following method: (acquisition::CustomAcq)(problem::BossProblem, options::BossOptions)
This method should return a function acq(x::AbstractVector{<:Real}) = val::Real, which is maximized to select the next evaluation function of blackbox function in each iteration.
The expected improvement (EI) acquisition function.
Fitness function must be defined as a part of the problem definition in order to use EI. (See Fitness.)
Measures the quality of a potential evaluation point x as the expected improvement in best-so-far achieved fitness by evaluating the objective function at y = f(x).
In case of constrained problems, the expected improvement is additionally weighted by the probability of feasibility of y. I.e. the probability that all(cons(y) .> 0.).
If the problem is constrained on y and no feasible point has been observed yet, then the probability of feasibility alone is returned as the acquisition function.
Rather than using the actual evaluations (xᵢ,yᵢ) from the dataset, the best-so-far achieved fitness is calculated as the maximum fitness among the means ̂yᵢ of the posterior predictive distribution of the model evaluated at xᵢ. This is a simple way to handle evaluation noise which may not be suitable for problems with substantial noise. In case of Bayesian Inference, an averaged posterior of the model posterior samples is used for the prediction of ŷᵢ.
Keywords
ϵ_samples::Int: Controls how many samples are used to approximate EI. The ϵ_samples keyword is ignored unless MAP model fitter and NonlinFitness are used! In case of BI model fitter, the number of samples is instead set equal to the number of posterior samples. In case of LinearFitness, the expected improvement can be calculated analytically.
cons_safe::Bool: If set to true, the acquisition function acq(x) is made 'constraint-safe' by checking the bounds and constraints during each evaluation. Set cons_safe to true if the evaluation of the model at exterior points may cause errors or nonsensical values. You may set cons_safe to false if the evaluation of the model at exterior points can provide useful information to the acquisition maximizer and does not cause errors. Defaults to true.
Returns the point (x, y) from the dataset of the given problem such that y satisfies the constraints and fitness(y) is maximized. Returns nothing if the dataset is empty or if no feasible point is present.
Does not check whether x belongs to the domain as no exterior points should be present in the dataset.
Stores miscellaneous settings of the BOSS algorithm.
Keywords
info::Bool: Setting info=false silences the BOSS algorithm.
debug::Bool: Set debug=true to print stactraces of caught optimization errors.
parallel_evals::Symbol: Possible values: :serial, :parallel, :distributed. Defaults to :parallel. Determines whether to run multiple objective function evaluations within one batch in serial, parallel, or distributed fashion. (Only has an effect if batching AM is used.)
callback::BossCallback: If provided, callback(::BossProblem; kwargs...) will be called before the BO procedure starts and after every iteration.
If an object cb of type BossCallback is passed to BossOptions, the method shown below will be called before the BO procedure starts and after every iteration.
If PlotOptions is passed to BossOptions as callback, the state of the optimization problem is plotted in each iteration. Only works with one-dimensional x domains but supports multi-dimensional y.
Arguments
Plots::Module: Evaluate using Plots and pass the Plots module to PlotsOptions.
Keywords
f_true::Union{Nothing, Function}: The true objective function to be plotted.
points::Int: The number of points in each plotted function.
xaxis::Symbol: Used to change the x axis scale (:identity, :log).
yaxis::Symbol: Used to change the y axis scale (:identity, :log).
[1] Bobak Shahriari et al. “Taking the human out of the loop: A review of Bayesian optimization”. In: Proceedings of the IEEE 104.1 (2015), pp. 148–175
Settings
This document was generated with Documenter.jl version 1.5.0 on Monday 8 July 2024. Using Julia version 1.10.4.
+)
The kwargs first is true on the first callback before the BO procedure starts, and is false on all subsequent callbacks after each iteration.
See PlotCallback for an example usage of this feature for plotting.
If PlotOptions is passed to BossOptions as callback, the state of the optimization problem is plotted in each iteration. Only works with one-dimensional x domains but supports multi-dimensional y.
Arguments
Plots::Module: Evaluate using Plots and pass the Plots module to PlotsOptions.
Keywords
f_true::Union{Nothing, Function}: The true objective function to be plotted.
points::Int: The number of points in each plotted function.
xaxis::Symbol: Used to change the x axis scale (:identity, :log).
yaxis::Symbol: Used to change the y axis scale (:identity, :log).
[1] Bobak Shahriari et al. “Taking the human out of the loop: A review of Bayesian optimization”. In: Proceedings of the IEEE 104.1 (2015), pp. 148–175
Settings
This document was generated with Documenter.jl version 1.5.0 on Monday 8 July 2024. Using Julia version 1.10.4.