BaFe2Se3 (Pnma, CsAg2I3-type structure), recently assumed to show superconductivity at ~ 11 K, exhibits a pressure-dependent structural transition to the CsCu2Cl3-type structure (Cmcm space group) around 60 kbar, as evidenced from pressure-dependent synchrotron powder diffraction data. Temperature-dependent synchrotron powder diffraction data indicate an evolution of the room-temperature BaFe2Se3 structure towards a high symmetry CsCu2Cl3 form upon heating. Around 425 K BaFe2Se3 undergoes a reversible, first order isostructural transition, that is supported by the differential scanning calorimetry data. The temperature-dependent structural changes occur in two stages, as determined by the alignment of the FeSe4 tetrahedra and corresponding adjustments of the positions of Ba atoms. On further heating, a second order phase transformation into the Cmcm structure is observed at 660 K. A rather unusual combination of isostructural and second-order phase transformations is parameterized within phenomenological theory assuming high-order expansion of Landau potential. A generic phase diagram mapping observed structures is proposed on the basis of the parameterization.