Dynamic Optimization

Linear Programming Approaches

  1. Z. Wen, L. J. Durlofsky, B. Van Roy, and K. Aziz, ``Approximate Dynamic Programming for Optimizing Oil Production,'' Chapter 25 in Reinforcement Learning and Approximate Dynamic Programming for Feedback Control, edited by F. L. Lewis and D. Liu, Wiley-IEEE Press, 2012.

  2. Z. Wen, L. J. Durlofsky, B. Van Roy, and K. Aziz, ``Use of Approximate Dynamic Programming for Production Optimization,'' forthcoming in the SPE Proceedings.

  3. J. Han and B. Van Roy, ``Control of Diffusions via Linear Programming,'' in Stochastic Programming: The State of the Art, in Honor of George B. Dantzig, edited by Gerd Infanger, pp. 329-354, Springer, 2011.

  4. V. F. Farias and B. Van Roy, ``An Approximate Dynamic Programming Approach to Network Revenue Management,'' 2007.

  5. D. P. de Farias and B. Van Roy, ``A Cost-Shaping Linear Program for Average-Cost Approximate Dynamic Programming with Performance Guarantees,'' Mathematics of Operations Research, Vol. 31, No. 3, pp. 597-620, 2006.

  6. R. Cogill, M. Rotkowitz, B. Van Roy, S. Lall, ``An Approximate Dynamic Programming Approach to Decentralized Control of Stochastic Systems,'' Lecture Notes in Control and Information Sciences, Springer, Berlin, 2006, Vol. 329, pp. 243-256.

  7. V. F. Farias and B. Van Roy, ``Tetris: A Study of Randomized Constraint Sampling,'' in Probabilistic and Randomized Methods for Design Under Uncertainty, G. Calafiore and F. Dabbene, eds., Springer-Verlag, 2006.

  8. D. P. de Farias and B. Van Roy, `` On Constraint Sampling in the Linear Programming Approach to Approximate Dynamic Programming,'' Mathematics of Operations Research, Vol. 29, No. 3, August 2004, pp. 462-478.

  9. D. P. de Farias and B. Van Roy, ``The Linear Programming Approach to Approximate Dynamic Programming,'' Operations Research, Vol. 51, No. 6, November-December 2003, pp. 850-865.

Approximate Value Iteration and Temporal-Difference Methods

  1. B. Van Roy, ``On Regression-Based Stopping Times,'' Discrete Event Dynamic Systems, Vol. 20, No. 3, pp. 307-324, 2010.

  2. C. C. Moallemi, S. Kumar, and B. Van Roy, ``Approximate and Data-Driven Dynamic Programming for Queueing Networks,'' 2008.

  3. B. Van Roy ``Performance Loss Bounds for Approximate Value Iteration with State Aggregation,'' Mathematics of Operations Research, Vol. 31, No. 2, pp. 234-244, 2006.

  4. D. S. Choi and B. Van Roy, ``A Generalized Kalman Filter for Fixed Point Approximation and Efficient Temporal-Difference Learning,'' Discrete Event Dynamic Systems, Vol. 16, No. 2, April 2006.

  5. B. Van Roy, `` Neuro-Dynamic Programming: Overview and Recent Trends,'' in Handbook of Markov Decision Processes: Methods and Applications, edited by E. Feinberg and A. Shwartz, Kluwer, 2001.

  6. J. N. Tsitsiklis and B. Van Roy, `` On Average Versus Discounted Reward Temporal-Difference Learning,'' Machine Learning, Vol. 49, No. 2-3, 2002, pp. 179-191.

  7. J. N. Tsitsiklis and B. Van Roy, ``Regression Methods for Pricing Complex American-Style Options,'' IEEE Transactions on Neural Networks, Vol. 12, No. 4 (special issue on computational finance), July 2001, pp. 694-703.

  8. D. P. de Farias and B. Van Roy, `` On the Existence of Fixed Points for Approximate Value Iteration and Temporal-Difference Learning,'' Journal of Optimization Theory and Applications, Vol. 105, No. 3, June, 2000.

  9. J. N. Tsitsiklis and B. Van Roy, ``Average Cost Temporal-Difference Learning,'' Automatica, Vol. 35, No. 11, November 1999, pp. 1799-1808.

  10. J. N. Tsitsiklis and B. Van Roy, ``Optimal Stopping of Markov Processes: Hilbert Space Theory, Approximation Algorithms, and an Application to Pricing High-Dimensional Financial Derivatives,'' IEEE Transactions on Automatic Control, Vol. 44, No. 10, October 1999, pp. 1840-1851.

  11. J. N. Tsitsiklis and B. Van Roy, ``An Analysis of Temporal-Difference Learning with Function Approximation,'' IEEE Transactions on Automatic Control, Vol. 42, No. 5, May 1997, pp. 674-690.

  12. J. N. Tsitsiklis and B. Van Roy, ``Feature-Based Methods for Large Scale Dynamic Programming,'' Machine Learning, Vol. 22, 1996, pp. 59-94.

  13. B. Van Roy, D. P. Bertsekas, Y. Lee, and J. N. Tsitsiklis, ``A Neuro-Dynamic Programming Approach to Retailer Inventory Management,'' Proceedings of the IEEE Conference on Decision and Control, 1997. (full length version)

Miscellaneous

  1. H. Permuter, P. Cuff, B. Van Roy, and T. Weissman, ``Capacity of the Trapdoor Channel with Feedback,'' IEEE Transactions on Information Theory, Vol. 54, No. 7, pp. 3150-3165, 2008.

  2. V. F. Farias and B. Van Roy, ``Dynamic Pricing with a Prior on Market Response,'' Operations Research, Vol. 58, No. 1, pp. 16-29, 2010.

  3. B. Van Roy, ``A Short Proof of Optimality for the MIN Cache Replacement Algorithm,'' Information Processing Letters, Vol. 102, No. 2, pp. 72-73, 2007.

  4. G. Y. Weintraub, C. L. Benkard, and B. Van Roy, ``Computational Methods for Oblivious Equilibrium,'' Operations Research, Vol. 58, No. 4, pp. 1247-1265, 2010. [Matlab code (updated July 2012)]

  5. G. Y. Weintraub, L. C. Benkard, and B. Van Roy, ``Markov Perfect Industry Dynamics with Many Firms,'' Econometrica, Vol. 76, No. 6, 2008, pp. 1375-1411. [Technical Appendix]

  6. V. F. Farias and B. Van Roy ``Approximation Algorithms for Dynamic Resource Allocation,'' Operations Research Letters, Vol. 34, No. 2, March 2006, pp. 180-190.

  7. X. Yan, P. Diaconis, P. Rusmevichientong, and B. Van Roy, ``Solitaire: Man Versus Machine,'' Advances in Neural Information Processing Systems 17, MIT Press, 2005.

  8. C. C. Moallemi and B. Van Roy ``Distributed Optimization in Adaptive Networks,'' Advances in Neural Information Processing Systems 16, MIT Press, 2004. [appendix]

  9. P. Rusmevichientong and B. Van Roy, ``A Tractable POMDP for a Class of Sequencing Problems,'' Proceedings of the Conference on Uncertainty in Artificial Intelligence, 2001.

  10. N. O. Keohane, B. Van Roy, and R. J. Zeckhauser, ``Managing the Quality of a Resource with Stock and Flow Controls,'' Journal of Public Economics, Vol. 91, 2007, pp. 541-569.