Bias and Variance Approximation in Value Function Estimates

We consider a finite-state, finite-action, infinite-horizon, discounted reward Markov decision process and study the bias and variance in the value function estimates that result from empirical estimates of the model parameters. We provide closed-form approximations for the bias and variance, which can then be used to derive confidence intervals around the value function estimates. We illustrate and validate our findings using a large database describing the transaction and mailing histories for customers of a mail-order catalog firm.

• 저널Management Science · 53(2) · 308–322
• 토픽Consumer Market Behavior and Pricing · Marketing
• DOI10.1287/mnsc.1060.0614
• 저자Shie Mannor, Duncan Simester, Peng Sun, John N. Tsitsiklis