The Annals of Statistics, Vol. 4, No. 6 (Nov., 1976), pp. 1219-1235 (17 pages) The paper deals with continuous time Markov decision processes on a fairly general state space. The rewards are ...

Markov processes form a fundamental class of stochastic models in which the evolution of a system is delineated by the memoryless property. In such processes, the future state depends solely on the ...

Content: In this lecture we analyse the class of Markov processes in continuous time. We start with a focus on processes processes in discrete state spaces. For this we introduce point processes as a ...

Bunches of individual customers approach a single servicing facility according to a stationary compound Poisson process. The resulting waiting line process is studied in continuous time by the method ...