There is some previous work on modeling ambiguity in the transition probability and mitigating its effect on the optimal policy. Satia and Lave (1973); White and Eldieb (1994); Bagnell et al. (2001) investigate ambiguity in the context of infinite horizon DP with finite state and action spaces. They model ambiguity by constraining the transition probability matrix to lie in a pre-specified polytope. They do not discuss how one constructs this polytope. Moreover, the complexity of the resulting robust DP is at least an order of magnitude higher than DP. Shapiro and Kleywegt (2002) investigate ambiguity in the context of stochastic programming and propose a sampling based method for solving the maximin problem. However, they do not discuss how to choose and calibrate the set of ambiguous priors. None of this work discusses the dynamic structure of the ambiguity; in particular, there is no discussion of the central role of "Rectangularity". Our theoretical contributions are based on recent work on uncertain priors in the economics literature (Gilboa and Schmeidler, 1989; Epstein and Schneider, 2001, 2002; Hansen and Sargent, 2001). The focus of this body of work is on the axiomatic justi¯cation for uncertain priors in the context of multi-period utility maximization. It does not provide any means of selecting the set of uncertain priors nor does it focus on efficiently solving the resulting robust DP.