Polarized Raman scattering measurements have been performed on Na0.5CoO2 single crystal from 8 to 305 K. Both the A1g and E1g phonon modes show a softening below Tc1 ~ 83 K. Additionally, the A1g phonon mode, which is forbidden in the scattering geometry of cross polarization for the triangular CoO2 layers, appears below Tc1. In contrast, the metal-insulator transition at Tc2 ~ 46 K has only secondary effect on the Raman spectra. The phonon softening and the ``forbidden Raman intensity follow closely magnetic order parameter and the gap function at the Fermi surface, indicating that the distortion of CoO6 octahedra at Tc1, instead of the Na ordering at ~350 K, is the relevant structural component of the 83 K phase transition.