Variable Resolution Dynamic Programming: Efficiently Learning Action Maps in Multivariate Real-valued State-spaces (1991)

Andrew Moore

Tags

Kd-trees and Ball-trees, Markov Decision Processes, Memory-based Learning, Reinforcement Learning

Abstract

An effective method to create an autonomous reactive controller is to learn a model of the environment and then use dynamic programming to derive a policy to maximize long term reward. Neither learning environmental models nor dynamic programming require parametric assumptions about the world, and so learning can proceed with no danger of becoming "stuck" by a mismatch between the parametric assumptions and reality. The paper discusses how such an approach can be realized in real valued multivariate state spaces in which straightforward discretization falls prey to the curse of dimensionality.

Full text

Download (application/pdf, 2.9 MB)

Approximate BibTeX Entry

@inproceedings{moore-variableresolution,
    Month = {June},
    Year = {1991},
    Publisher = {Morgan Kaufmann},
    Address = {340 Pine Street, 6th Fl., San Francisco, CA 94104},
    Booktitle = {Machine Learning: Proceedings of the Eighth International Conference},
    Editor = {Birnbaum, L. and Collins, G.},
    Author = {Andrew Moore},
    Title = {Variable Resolution Dynamic Programming: Efficiently Learning Action Maps in Multivariate Real-valued State-spaces}
}