We construct a family of genealogy-valued Markov processes that are induced by a continuous-time Markov population process. We derive exact expressions for the likelihood of a given genealogy conditional on the history of the underlying population process. These lead to a version of the nonlinear filtering equation, which can be used to design efficient Monte Carlo inference algorithms. Existing full-information approaches for phylodynamic inference are special cases of the theory.