Algorithms & Publications

Quasi-Newton Methods
Stochastic Optimization
Global Optimization


Philipp Hennig
Christian Schuler
Martin Kiefel


Entropy-Search: Information-Efficient Global Optimization

Paper: [preprint] [BibTeX]
Hennig, P. & Schuler, C.: Entropy Search for Information-Efficient Global Optimization
Journal of Machine Learning Research (JMLR) Vol. 13, 2012, in press. Please cite when you use this algorithm.
Code: After download, refer to the README.txt file

In numerical optimization, where the input space can have thousands or millions of dimensions, computational cost is the bottle-neck. But in experimental design, also known as global optimization, where the objective function itself is unknown, and often has a physical cost, sample cost dominates, and optimization algorithms should aim to make as much use of available information as possible. Entropy Search was specifically designed for this purpose. It is a global, black box optimization algorithm taking noisy evaluations as inputs, and actively guiding the experimental optimization process. Entropy Search is a way of turning the physical problem of finding good experimental parameters into a numerical problem of maximizing information gain. It is not numerically cheap - choosing each new evaluation point takes several seconds on a contemporary machine. But it is highly sample efficient, thus saving valuable experimentation time and resources.