Understanding influencing factors is essential for the surveillance and prevention of infectious diseases, and the factors are likely to vary spatially and temporally as the disease progresses. Taking daily cases and deaths data during the coronavirus disease 2019 (COVID-19) outbreak in the U.S. as a case study, we develop a mobility-augmented geographically and temporally weighted regression (M-GTWR) model to quantify the spatiotemporal impacts of social-demographic factors and human activities on the COVID-19 dynamics. Different from the base GTWR model, we incorporate a mobility-adjusted distance weight matrix where travel mobility is used in addition to the spatial adjacency to capture the correlations among local observations. The model residuals suggest that the proposed model achieves a substantial improvement over other benchmark methods in addressing the spatiotemporal nonstationarity. Our results reveal that the impacts of social-demographic and human activity variables present significant spatiotemporal heterogeneity. In particular, a 1% increase in population density may lead to 0.63% and 0.71% more daily cases and deaths, and a 1% increase in the mean commuting time may result in 0.22% and 0.95% increases in daily cases and deaths. Although increased human activities will, in general, intensify the disease outbreak, we report that the effects of grocery and pharmacy-related activities are insignificant in areas with high population density. And activities at the workplace and public transit are found to either increase or decrease the number of cases and deaths, depending on particular locations. The results of our study establish a quantitative framework for identifying influencing factors during a disease outbreak, and the obtained insights may have significant implications in guiding the policy-making against infectious diseases.