The electronic states near the Fermi level of recently discovered superconductor Ba$_2$CuO$_{4-delta}$ consist primarily of the Cu $d_{x^2-y^2}$ and $d_{3z^2-r^2}$ orbitals. We investigate the electronic correlation effect and the orbital polarization of an effective two-orbital Hubbard model mimicking the low-energy physics of Ba$_2$CuO$_{4-delta}$ in the hole-rich regime by utilizing the dynamical mean-field theory with the Lanczos method as the impurity solver. We find that the hole-overdoped Ba$_2$CuO$_{4-delta}$ with $3d^8$ (Cu$^{3+}$) is in the orbital-selective Mott phase (OSMP) at half-filling, and the typical two-orbital feature remains in Ba$_2$CuO$_{4-delta}$ when the electron filling approaches $n_esim 2.5$, which closely approximates to the experimental hole doping for the emergence of the high-$T_c$ superconductivity. We also obtain that the orbital polarization is very stable in the OSMP, and the multiorbital correlation can drive orbital polarization transitions. These results indicate that in hole-overdoped Ba$_2$CuO$_{4-delta}$ the OSMP physics and orbital polarization, local magnetic moment, and spin or orbital fluctuations still exist. We propose that our present results are also applicable to Sr$_2$CuO$_{4-delta}$ and other two-orbital cuprates, demanding an unconventional multiorbital superconducting scenario in hole-overdoped high-$T_c$ cuprates.