We analyze the stellar populations of a sample of 62 massive (log Mstar/Msun > 10.7) galaxies in the redshift range 1 < z < 1.6, with the main goal of investigating the role of recent quenching in the size growth of quiescent galaxies. We demonstrate that our sample is not biased toward bright, compact, or young galaxies, and thus is representative of the overall quiescent population. Our high signal-to-noise ratio Keck LRIS spectra probe the rest-frame Balmer break region which contains important absorption line diagnostics of recent star formation activity. We obtain improved measures of the various stellar population parameters, including the star-formation timescale tau, age and dust extinction, by fitting templates jointly to both our spectroscopic and broad-band photometric data. We identify which quiescent galaxies were recently quenched and backtrack their individual evolving trajectories on the UVJ color-color plane finding evidence for two distinct quenching routes. By using sizes measured in the previous paper of this series, we confirm that the largest galaxies are indeed among the youngest at a given redshift. This is consistent with some contribution to the apparent growth from recent arrivals, an effect often called progenitor bias. However, we calculate that recently-quenched objects can only be responsible for about half the increase in average size of quiescent galaxies over a 1.5 Gyr period, corresponding to the redshift interval 1.25 < z < 2. The remainder of the observed size evolution arises from a genuine growth of long-standing quiescent galaxies.