We investigate the spectral characteristics of 51 candidate High Frequency Peakers (HFPs), from the ``bright HFP sample, in order to determine the nature of each object, and to obtain a smaller sample of genuine young radio sources. Simultaneous multi-frequency VLA observations carried out at various epochs have been used to detect flux density and spectral shape variability in order to pinpoint contaminant objects, since young radio sources are not expected to be significantly variable on such a short time-scale. From the analysis of the spectral variability we find 13 contaminant objects, 11 quasars, 1 BL Lac, and 1 unidentified object, which we have rejected from the sample of candidate young radio sources. The 6 years elapsed between the first and latest observing run are not enough to detect any substantial evolution of the overall spectrum of genuine, non variable, young radio sources. If we also consider the pc-scale information, we find that the total radio spectrum we observe is the result of the superposition of the spectra of different regions (lobes, hot-spots, core, jets), instead of a single homogeneous radio component. This indicates that the radio source structure plays a relevant role in determining the spectral shape also in the rather common case in which the morphology appears unresolved even on high-resolution scales.