In countries with a moderate seismic hazard, the classical methods developed for strong motion prone countries to estimate the seismic behaviour and subsequent vulnerability of existing buildings are often inadequate and not financially realistic. The main goals of this paper are to show how the modal analysis can contribute to the understanding of the seismic building response and the good relevancy of a modal model based on ambient vibrations for estimating the structural deformation under moderate earthquakes. We describe the application of an enhanced modal analysis technique (Frequency Domain Decomposition) to process ambient vibration recordings taken at the Grenoble City Hall building (France). The frequencies of ambient vibrations are compared with those of weak earthquakes recorded by the French permanent accelerometric network (RAP) that was installed to monitor the building. The frequency variations of the building under moderate earthquakes are shown to be slight (~2%) and therefore ambient vibration frequencies are relevant over the elastic domain of the building. The modal parameters extracted from ambient vibrations are then used to determine the 1D lumped-mass model in order to reproduce the inter-storey drift under weak earthquakes and to fix a 3D numerical model that could be used for strong earthquakes. The correlation coefficients between data and synthetic motion are close to 80% and 90% in horizontal directions, for the 1D and 3D modelling, respectively.