Title: Continuous-time mean field Markov decision models

Abstract: We consider a finite number of $N$ statistically equal individuals, each moving on a finite set of states according to a continuous-time Markov Decision Process (MDP). Transition intensities of the individuals and generated rewards depend not only on the state and action of the individual itself, but also on the states of the other individuals as well as the chosen action. Interactions like this are typical for a wide range of models in e.g.\ biology, epidemics, finance, social science and queueing systems among others. The aim is to maximize the expected discounted reward of the system, i.e. the individuals have to cooperate as a team. Computationally this is a difficult task when $N$ is large. Thus, we consider the limit for $N\to\infty.$ In contrast to other papers we treat this problem from an MDP perspective and use Pontryagin's maximum principle to solve the limiting problem. This has the advantage that we need less assumptions in order to construct asymptotically optimal strategies than using viscosity solutions of HJB equations. We show how to apply our results using two examples: a machine replacement problem and a problem from epidemics. We also show that optimal feedback policies are not necessarily asymptotically optimal.