Imperfect channel state information degrades the performance of multiple-input multiple-output (MIMO) communications; its effect on single-user (SU) and multi-user (MU) MIMO transmissions are quite different. In particular, MU-MIMO suffers from residual inter-user interference due to imperfect channel state information while SU-MIMO only suffers from a power loss. This paper compares the throughput loss of both SU and MU MIMO on the downlink due to delay and channel quantization. Accurate closed-form approximations are derived for the achievable rates for both SU and MU MIMO. It is shown that SU-MIMO is relatively robust to delayed and quantized channel information, while MU MIMO with zero-forcing precoding loses spatial multiplexing gain with a fixed delay or fixed codebook size. Based on derived achievable rates, a mode switching algorithm is proposed that switches between SU and MU MIMO modes to improve the spectral efficiency, based on the average signal-to-noise ratio (SNR), the normalized Doppler frequency, and the channel quantization codebook size. The operating regions for SU and MU modes with different delays and codebook sizes are determined, which can be used to select the preferred mode. It is shown that the MU mode is active only when the normalized Doppler frequency is very small and the codebook size is large.