Structural evolution of a prospective hydrogen storage material, ammonia borane NH3BH3, has been studied at high pressures up to 12 GPa and at low temperatures by synchrotron powder diffraction. At 293 K and above 1.1 GPa a disordered I4mm structure reversibly transforms into a new ordered phase. Its Cmc21 structure was solved from the diffraction data, the positions of N and B atoms and the orientation of NH3 and BH3 groups were finally assigned with the help of density functional theory calculations. Group-theoretical analysis identifies a single two-component order parameter, combining ordering and atomic displacement mechanisms, which link an orientationally disordered parent phase I4mm with ordered distorted Cmc21, Pmn21 and P21 structures. We propose a generic phase diagram for NH3BH3, mapping three experimentally found and one predicted (P21) phases as a function of temperature and pressure, along with the evolution of the corresponding structural distortions. Ammonia borane belongs to the class of improper ferroelastics and we show that both temperature- and pressure-induced phase transitions can be driven to be of the second order. The role of N-H...H-B dihydrogen bonds and other intermolecular interactions in the stability of NH3BH3 polymorphs is examined.