In this work, we prove that any symplectic matrix can be factored into no more than 9 unit triangular symplectic matrices. This structure-preserving factorization of the symplectic matrices immediately reveals two well-known features that, (i) the determinant of any symplectic matrix is one, (ii) the matrix symplectic group is path connected, as well as a new feature that (iii) all the unit triangular symplectic matrices form a set of generators of the matrix symplectic group. Furthermore, this factorization yields effective methods for the unconstrained parametrization of the matrix symplectic group as well as its structured subsets. The unconstrained parametrization enables us to apply faster and more efficient unconstrained optimization algorithms to the problems with symplectic constraints under certain circumstances.