This book has two principal aims. In the first half of the book, the aim is the study of discrete time and continuous time Markov chains. The first part of the text is very well written and easily accessible to the advanced undergraduate engineering or mathematics student.
My only complaint in the first half of the text regards the definition of continuous time Markov chains. The definition is introduced using the technical concepts of jump chain/holding time properties. This doesn't tie out well with the treatment of the discrete time case and may seem counter-intuitive to readers initially. However, the author does establish the equivalence of the jump chain/holding time definition to the usual transition probability definition towards the end of Chapter 2.
The second half of the text deals with the relationship of Markov chains to other aspects of stochastic analysis and the application of Markov chains to applied settings.
In Chapter 4, the material takes a serious jump (explosion?) in sophistication level. In this chapter, the author introduces filtrations, martingales, optional sampling/optional stopping and Brownian motion. This is entirely too ambitious a reading list to squeeze into the 40 or so pages allocated for all of this, in the opinion of this reviewer. The author places some prerequisite material in the appendix chapter.
Chapter 5 is a much more down-to-earth treatment of genuine applications of Markov chains. Birth/Death processes in biology, queuing networks in information theory, inventory management in operations research, and Markov decision processes are introduced via a series of very nice toy examples. This chapter wraps up with a nice discussion of simulation and the method of Markov chain Monte Carlo.
If the next edition of this book removes chapter 4 and replaces it with treatment of an actual real-world problem (or two) using genuine data sets, this reviewer would be happy to rate that edition 5 stars.
Markov chains are central to the understanding of random processes. This is not only because they pervade the applications of random processes, but also because one can calculate explicitly many quantities of interest. This textbook, aimed at advanced undergraduate or MSc students with some background in basic probability theory, focuses on Markov chains and quickly develops a coherent and rigorous theory whilst showing also how actually to apply it. Both discrete-time and continuous-time chains are studied. A distinguishing feature is an introduction to more advanced topics such as martingales and potentials in the established context of Markov chains. There are applications to simulation, economics, optimal control, genetics, queues and many other topics, and exercises and examples drawn both from theory and practice. It will therefore be an ideal text either for elementary courses on random processes or those that are more oriented towards applications.