Recent outbreak of the novel coronavirus COVID-19 has affected all of our lives in one way or the other. While medical researchers are working hard to find a cure and doctors/nurses to attend the affected individuals, measures such as `lockdown, `stay-at-home, `social distancing are being implemented in different parts of the world to curb its further spread. To model the non-stationary spread, we propose a novel time-varying semiparametric AR$(p)$ model for the count valued time-series of newly affected cases, collected every day and also extend it to propose a novel time-varying INGARCH model. Our proposed structures of the models are amenable to Hamiltonian Monte Carlo (HMC) sampling for efficient computation. We substantiate our methods by simulations that show superiority compared to some of the close existing methods. Finally we analyze the daily time series data of newly confirmed cases to study its spread through different government interventions.