Perturbation theory (PT) is a powerful and commonly used tool in the investigation of closed quantum systems. In the context of open quantum systems, PT based on the Markovian quantum master equation is much less developed. The investigation of open systems mostly relies on exact diagonalization of the Liouville superoperator or quantum trajectories. In this approach, the system size is rather limited by current computational capabilities. Analogous to closed-system PT, we develop a PT suitable for open quantum systems. This proposed method is useful in the analytical understanding of open systems as well as in the numerical calculation of system properties, which would otherwise be impractical.