Quantum key distribution (QKD) guarantees the secure communication between legitimate parties with quantum mechanics. High-dimensional QKD (HDQKD) not only increases the secret key rate but also tolerates higher quantum bit error rate (QBER). Many HDQKD experiments have been realized by utilizing orbital-angular-momentum (OAM) photons as the degree of freedom (DOF) of OAM of the photon is a prospective resource for HD quantum information. In this work we proposed and characterized that a high-quality HDQKD based on polarization-OAM hybrid states can be realized by utilizing state mapping between different DOFs. Both the preparation and measurement procedures of the proof-of-principle verification experiment are simple and stable. Our experiment verified that $(0.60pm 0.06)%$ QBER and $1.849pm 0.008$ bits secret key rate per sifted signal can be achieved for a four-dimensional QKD with the weak coherent light source and decoy state method.