Memory-based Stochastic Optimization (1996)

Andrew Moore, Jeff Schneider


Active Learning, Locally Weighted Learning, Memory-based Learning, Reinforcement Learning


In this paper we introduce new algorithms for optimizing noisy plants in which each experiment is very expensive. The algorithms build a global non-linear model of the expected output at the same time as using Bayesian linear regression analysis of locally weighted polynomial models. The local model answers queries about confidence, noise, gradient and Hessians, and use them to make automated decisions similar to those made by a practitioner of Response Surface Methodology. The global and local models are combined naturally as a locally weighted regression. We examine the question of whether the global model can really help optimization, and we extend it to the case of time-varying functions. We compare the new algorithms with a highly tuned higher-order stochastic optimization algorithm on randomly-generated functions and a simulated manufacturing task. We note significant improvements in total regret, time to converge, and final solution quality.

Full text

Download (application/pdf, 297.4 kB)

Approximate BibTeX Entry

    Year = {1996},
    Volume = {8},
    Pages = {1066--1072},
    Publisher = {MIT Press},
    Booktitle = {Neural Information Processing Systems 8},
    Editor = {D. Touretzky and M. Mozer and M. Hasselm},
    Author = { Andrew Moore, Jeff Schneider },
    Title = {Memory-based Stochastic Optimization}

Copyright 2010, Carnegie Mellon University, Auton Lab. All Rights Reserved.