An urgent problem in controlling COVID-19 spreading is to understand the role of undocumented infection. We develop a five-state model for COVID-19, taking into account the unique features of the novel coronavirus, with key parameters determined by the government reports and mathematical optimization. Tests using data from China, South Korea, Italy, and Iran indicate that the model is capable of generating accurate prediction of the daily accumulated number of confirmed cases and is entirely suitable for real-time prediction. The drastically disparate testing and diagnostic standards/policies among different countries lead to large variations in the estimated parameter values such as the duration of the outbreak, but such uncertainties have little effect on the occurrence time of the inflection point as predicted by the model, indicating its reliability and robustness. Model prediction for Italy suggests that insufficient government action leading to a large fraction of undocumented infection plays an important role in the abnormally high mortality in that country. With the data currently available from United Kingdom, our model predicts catastrophic epidemic scenarios in the country if the government did not impose strict travel and social distancing restrictions. A key finding is that, if the percentage of undocumented infection exceeds a threshold, a non-negligible hidden population can exist even after the the epidemic has been deemed over, implying the likelihood of future outbreaks should the currently imposed strict government actions be relaxed. This could make COVID-19 evolving into a long-term epidemic or a community disease a real possibility, suggesting the necessity to conduct universal testing and monitoring to identify the hidden individuals.