We analyze the ground-state energy, local spin correlation, impurity spin polarization, impurity-induced magnetization, and corresponding zero-field susceptibilities of the symmetric single-impurity Kondo model on a tight-binding chain with bandwidth $W=2{cal D}$ and coupling strength $J_{rm K}$. We compare perturbative results and variational upper bounds from Yosida, Gutzwiller, and first-order Lanczos wave functions to the numerically exact data obtained from the Density-Matrix Renormalization Group (DMRG) and from the Numerical Renormalization Group (NRG) methods. The Gutzwiller variational approach becomes exact in the strong-coupling limit and reproduces the ground-state properties from DMRG and NRG for large couplings. We calculate the impurity spin polarization and its susceptibility in the presence of magnetic fields that are applied globally/locally to the impurity spin. The Yosida wave function provides qualitatively correct results in the weak-coupling limit. In DMRG, chains with about $10^3$ sites are large enough to describe the susceptibilities down to $J_{rm K}/{cal D}approx 0.5$. For smaller Kondo couplings, only the NRG provides reliable results for a general host-electron density of states $rho_0(epsilon)$. To compare with results from Bethe Ansatz, we study the impurity-induced magnetization and zero-field susceptibility. For small Kondo couplings, the zero-field susceptibilities at zero temperature approach $chi_0(J_{rm K}ll {cal D})/(gmu_{rm B})^2approx exp[1/(rho_0(0)J_{rm K})]/(2C{cal D}sqrt{pi e rho_0(0)J_{rm K}})$, where $ln(C)$ is the regularized first inverse moment of the density of states. Using NRG, we determine the universal sub-leading corrections up to second order in $rho_0(0)J_{rm K}$.