Semi-Markov processes are a generalization of Markov processes since the exponential distribution of time intervals is replaced with an arbitrary distribution. This paper provides an integro-differential form of the Kolmogorovs backward equations for a large class of homogeneous semi-Markov processes, having the form of an abstract Volterra integro-differential equation. An equivalent evolutionary (differential) form of the equations is also provided. Fractional equations in the time variable are a particular case of our analysis. Weak limits of semi-Markov processes are also considered and their corresponding integro-differential Kolmogorovs equations are identified.
This paper describes the structure of solutions to Kolmogorovs equations for nonhomogeneous jump Markov processes and applications of these results to control of jump stochastic systems. These equations were studied by Feller (1940), who clarified in 1945 in the errata to that paper that some of its results covered only nonexplosive Markov processes. We present the results for possibly explosive Markov processes. The paper is based on the invited talk presented by the authors at the International Conference dedicated to the 200th anniversary of the birth of P. L.~Chebyshev.
We consider backward stochastic differential equations (BSDEs) related to finite state, continuous time Markov chains. We show that appropriate solutions exist for arbitrary terminal conditions, and are unique up to sets of measure zero. We do not require the generating functions to be monotonic, instead using only an appropriate Lipschitz continuity condition.
In this paper, we consider the optimal stopping problem on semi-Markov processes (SMPs) with finite horizon, and aim to establish the existence and computation of optimal stopping times. To achieve the goal, we first develop the main results of finite horizon semi-Markov decision processes (SMDPs) to the case with additional terminal costs, introduce an explicit construction of SMDPs, and prove the equivalence between the optimal stopping problems on SMPs and SMDPs. Then, using the equivalence and the results on SMDPs developed here, we not only show the existence of optimal stopping time of SMPs, but also provide an algorithm for computing optimal stopping time on SMPs. Moreover, we show that the optimal and -optimal stopping time can be characterized by the hitting time of some special sets, respectively.
Suppose that $(X_t)_{t ge 0}$ is a one-dimensional Brownian motion with negative drift $-mu$. It is possible to make sense of conditioning this process to be in the state $0$ at an independent exponential random time and if we kill the conditioned process at the exponential time the resulting process is Markov. If we let the rate parameter of the random time go to $0$, then the limit of the killed Markov process evolves like $X$ conditioned to hit $0$, after which time it behaves as $X$ killed at the last time $X$ visits $0$. Equivalently, the limit process has the dynamics of the killed bang--bang Brownian motion that evolves like Brownian motion with positive drift $+mu$ when it is negative, like Brownian motion with negative drift $-mu$ when it is positive, and is killed according to the local time spent at $0$. An extension of this result holds in great generality for Borel right processes conditioned to be in some state $a$ at an exponential random time, at which time they are killed. Our proofs involve understanding the Campbell measures associated with local times, the use of excursion theory, and the development of a suitable analogue of the bang--bang construction for general Markov processes. As examples, we consider the special case when the transient Borel right process is a one-dimensional diffusion. Characterizing the limiting conditioned and killed process via its infinitesimal generator leads to an investigation of the $h$-transforms of transient one-dimensional diffusion processes that goes beyond what is known and is of independent interest.
We consider plain vanilla European options written on an underlying asset that follows a continuous time semi-Markov multiplicative process. We derive a formula and a renewal type equation for the martingale option price. In the case in which intertrade times follow the Mittag-Leffler distribution, under appropriate scaling, we prove that these option prices converge to the price of an option written on geometric Brownian motion time-changed with the inverse stable subordinator. For geometric Brownian motion time changed with an inverse subordinator, in the more general case when the subordinators Laplace exponent is a special Bernstein function, we derive a time-fractional generalization of the equation of Black and Scholes.