This book provides a rigorous but elementary introduction to the theory of Markov Processes on a countable state space. It should be accessible to students with a solid undergraduate background in mathematics, including students from engineering, economics, physics, and biology. Topics covered are: Doeblin's theory, general ergodic properties, and continuous time processes.Applications are dispersed throughout the book. In addition, a whole chapter is devoted to reversible processes and the use of their associated Dirichlet forms to estimate the rate of convergence to equilibrium. These results are then applied to the analysis of the Metropolis (a.k.a simulated annealing) algorithm.

The corrected and enlarged 2nd edition contains a new chapter in which the author develops computational methods for Markov chains on a finite state space. Most intriguing isthe section with a new technique for computing stationary measures, which is applied to derivations of Wilson's algorithm and Kirchoff's formula for spanning trees in a connected graph.

Review: This book is an excellent presentation of the application of martingale theory to the theory of Markov processes, especially multidimensional diffusions. This approach was initiated by Stroock and Varadhan in their famous papers. (...) The proofs and techniques are presented in such a way that an adaptation in other contexts can be easily done. (...) The reader must be familiar with standard probability theory and measure theory which are summarized at the beginning of the book. This monograph can be recommended to graduate students and research workers but also to all interested in Markov processes from a more theoretical point of view. Mathematische Operationsforschung und Statistik, 1981

When the first edition of this textbook published in 2011, it constituted a substantial revision of the best-selling Birkh user title by the same author, A Concise Introduction to the Theory of Integration. Appropriate as a primary text for a one-semester graduate course in integration theory, this GTM is also useful for independent study. A complete solutions manual is available for instructors who adopt the text for their courses. This second edition has been revised as follows: 2.2.5 and 8.3 have been substantially reworked. New topics have been added. As an application of the material about Hermite functions in 7.3.2, the author has added a brief introduction to Schwartz's theory of tempered distributions in 7.3.4. Section 7.4 is entirely new and contains applications, including the Central Limit Theorem, of Fourier analysis to measures. Related to this are subsections 8.2.5 and 8.2.6, where L vy's Continuity Theorem and Bochner's characterization of the Fourier transforms of Borel probability on ℝ^N are proven. Subsection 8.1.2 is new and contains a proof of the Hahn Decomposition Theorem. Finally, there are several new exercises, some covering material from the original edition and others based on newly added material.

When the first edition of this textbook published in 2011, it constituted a substantial revision of the best-selling Birkhäuser title by the same author, A Concise Introduction to the Theory of Integration. Appropriate as a primary text for a one-semester graduate course in integration theory, this GTM is also useful for independent study. A complete solutions manual is available for instructors who adopt the text for their courses. This second edition has been revised as follows: 2.2.5 and 8.3 have been substantially reworked. New topics have been added. As an application of the material about Hermite functions in 7.3.2, the author has added a brief introduction to Schwartz's theory of tempered distributions in 7.3.4. Section 7.4 is entirely new and contains applications, including the Central Limit Theorem, of Fourier analysis to measures. Related to this are subsections 8.2.5 and 8.2.6, where Lévy's Continuity Theorem and Bochner's characterization of the Fourier transforms of Borel probability on ℝ^N are proven. Subsection 8.1.2 is new and contains a proof of the Hahn Decomposition Theorem. Finally, there are several new exercises, some covering material from the original edition and others based on newly added material.

This book provides a rigorous but elementary introduction to the theory of Markov Processes on a countable state space. It should be accessible to students with a solid undergraduate background in mathematics, including students from engineering, economics, physics, and biology. Topics covered are: Doeblin's theory, general ergodic properties, and continuous time processes. A whole chapter is devoted to reversible processes and the use of their associated Dirichlet forms to estimate the rate of convergence to equilibrium.

Kiyosi It 's greatest contribution to probability theory may be his introduction of stochastic differential equations to explain the Kolmogorov-Feller theory of Markov processes. Starting with the geometric ideas that guided him, this book gives an account of It 's program.

The modern theory of Markov processes was initiated by A. N. Kolmogorov. However, Kolmogorov's approach was too analytic to reveal the probabilistic foundations on which it rests. In particular, it hides the central role played by the simplest Markov processes: those with independent, identically distributed increments. To remedy this defect, It interpreted Kolmogorov's famous forward equation as an equation that describes the integral curve of a vector field on the space of probability measures. Thus, in order to show how It 's thinking leads to his theory of stochastic integral equations, Stroock begins with an account of integral curves on the space of probability measures and then arrives at stochastic integral equations when he moves to a pathspace setting. In the first half of the book, everything is done in the context of general independent increment processes and without explicit use of It 's stochastic integral calculus. In the second half, the author provides a systematic development of It 's theory of stochastic integration: first for Brownian motion and then for continuous martingales. The final chapter presents Stratonovich's variation on It 's theme and ends with an application to the characterization of the paths on which a diffusion is supported.

The book should be accessible to readers who have mastered the essentials of modern probability theory and should provide such readers with a reasonably thorough introduction to continuous-time, stochastic processes.