The recently discovered cuprate superconductor Ba$_2$CuO$_{3+delta}$ exhibits a high $T_csimeq73$K at $deltasimeq0.2$. The polycrystal grown under high pressure has a structure similar to La$_2$CuO$_4$, but with dramatically different lattice parameters due to the CuO$_6$ octahedron compression. The crystal field in the compressed Ba$_2$CuO$_4$ leads to an inverted Cu $3d$ $e_g$ complex with the $d_{x^2-y^2}$ orbital sitting below the $d_{3z^2-r^2}$ and an electronic structure highly unusual compared to the conventional cuprates. We construct a two-orbital Hubbard model for the Cu $d^9$ state at hole doping $x=2delta$ and study the orbital-dependent strong correlation and superconductivity. For the undoped case at $x=0$, we found that strong correlation drives an orbital-polarized Mott insulating state with the spin-$1/2$ moment of the localized $d_{3z^2-r^2}$ orbital. In contrast to the single-band cuprates where superconductivity is suppressed in the overdoped regime, hole doping the two-orbital Mott insulator leads to orbital-dependent correlations and the robust spin and orbital exchange interactions produce a high-$T_c$ antiphase $d$-wave superconductor even in the heavily doped regime at $x=0.4$. We conjecture that Ba$_2$CuO$_{3+delta}$ realizes mixtures of such heavily hole-doped superconducting Ba$_2$CuO$_4$ and disordered Ba$_2$CuO$_{3}$ chains in a single-layer or predominately separated bilayer structure. Our findings suggest that unconventional cuprates with liberated orbitals as doped two-band Mott insulators can be a direction for realizing high-T$_c$ superconductivity with enhanced transition temperature $T_c$.