Studying the development of malignant tumours, it is important to know and predict the proportions of different cell types in tissue samples. Knowing the expected temporal evolution of the proportion of normal tissue cells, compared to stem-like and non-stem like cancer cells, gives an indication about the progression of the disease and indicates the expected response to interventions with drugs. Such processes have been modeled using Markov processes. An essential step for the simulation of such models is then the determination of state transition probabilities. We here consider the experimentally more realistic scenario in which the measurement of cell population sizes is noisy, leading to a particular hidden Markov model. In this context, randomness in measurement is related to noisy measurements, which are used for the estimation of the transition probability matrix. Randomness in sampling, on the other hand, is here related to the error in estimating the state probability from small cell populations. Using aggregated data of fluorescence-activated cell sorting (FACS) measurement, we develop a minimum mean square error estimator (MMSE) and maximum likelihood (ML) estimator and formulate two problems to find the minimum number of required samples and measurements to guarantee the accuracy of predicted population sizes using a transition probability matrix estimated from noisy data. We analyze the properties of two estimators for different noise distributions and prove an optimal solution for Gaussian distributions with the MMSE. Our numerical results show, that for noisy measurements the convergence mechanism of transition probabilities and steady states differ widely from the real values if one uses the standard deterministic approach in which measurements are assumed to be noise free.
Download