Vacancy centers in diamond have proven to be a viable solid-state platform for quantum coherent opto-electronic applications. Among the variety of vacancy centers, silicon-vacancy (SiV) centers have recently attracted much attention as an inversion-symmetric system that is less susceptible to electron-phonon interactions. Nevertheless, phonon-mediated processes still degrade the coherent properties of SiV centers, however characterizing their electron-phonon coupling is extremely challenging due to their weak spectroscopic signatures and remains an open experimental problem. In this paper we theoretically investigate signatures of electron-phonon coupling in simulated linear and nonlinear spectra of SiV centers. We demonstrate how even extremely weak electron-phonon interactions, such as in SiV centers, may be completely characterized via nonlinear spectroscopic techniques and even resolved between different fine-structure transitions.