We present a combined experimental and theoretical study of the effects of pressure on T_c of the hexagonal layered superconductors nH-CaAlSi (n = 1, 5, 6), where nH labels the different stacking variants that were recently discovered. Experimentally, the pressure dependence of T_c has been investigated by measuring the magnetic susceptibility of single crystals up to 10 kbar. In contrast to previous results on polycrystalline samples, single crystals with different stacking sequences display different pressure dependences of T_c. 1H-CaAlSi shows a decrease of T_c with pressure, whereas 5H and 6H-CaAlSi exhibit an increase of T_c with pressure. Ab-initio calculations for 1H, 5H and 6H -CaAlSi reveal that an ultrasoft phonon branch associated to out-of-plane vibrations of the Al-Si layers softens with pressure, leading to a structural instability at high pressures. For 1H-CaAlSi the softening is not sufficient to cause an increase of T_c, which is consistent with the present experiments, but adverse to previous reports. For 5H and 6H the softening provides the mechanism to understand the observed increase of T_c with pressure. Calculations for hypothetical 2H and 3H stacking variants reveal qualitative and quantitative differences.