Bayesian optimization note series.

Last updated: June-15-2024

[TODO: include some discussions about different repositories, and notes of the BO book]

In this post, I plan to summarize some notes on Bayesian optimization (BO).

References:

• Book
  • Bayesian Optimization by Roman Garnett, 2023, link
  • Gaussian Processes for Machine Learning by Carl Edward Rasmussen and Christopher K. I. Williams, 2006, link
• Papers
  • A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning by Eric Brochu, Vlad M. Cora, Nando de Freitas, 2010, link
  • A Tutorial on Bayesian Optimization by Peter I. Frazier, 2018, link
• Code repository
  • TuRBO
    • Scalable Global Optimization via Local Bayesian Optimization
    • ``Implementation for the noise-free case and may not work well if observations are noisy as the center of the trust region should be chosen based on the posterior mean in this case.”
  • HEBO
  • BOTorch
    • Bayesian Optimization built on PyTorch
  • BayesianOptimization
  • Experimental design related
    • EDBO
    • GAUCHE
• Notes by me
  • Ch. 8 of BO book by Roman Garnett, 2023
    • PDF, Discussion about derivative information for EI and KG