Combining dynamic abstractions in large MDPs

Abstract

One of the reasons that it is difficult to plan and act in real-worlddomains is that they are very large. Existing research generallydeals with the large domain size using a static representation andexploiting a single type of domain structure. In this paper, wecreate a framework that encapsulates existing and new abstraction andapproximation methods into modules, and combines arbitrary modulesinto a system that allows for dynamic representation changes. We showthat the dynamic changes of representation allow our framework tosolve larger and more interesting domains than were previouslypossible, and while there are no optimality guarantees, suitablemodule choices gain tractability at little cost to optimality.