From the perspective of probability, the stability of growing network is studied in the present paper. Using the DMS model as an example, we establish a relation between the growing network and Markov process. Based on the concept and technique of first-passage probability in Markov theory, we provide a rigorous proof for existence of the steady-state degree distribution, mathematically re-deriving the exact formula of the distribution. The approach based on Markov chain theory is universal and performs well in a large class of growing networks.