A new layered perovskite-type oxide Ba$_2$RhO$_4$ was synthesized by a high-pressure technique with the support of convex-hull calculations. The crystal and electronic structure were studied by both experimental and computational tools. Structural refinements for powder x-ray diffraction data showed that Ba$_2$RhO$_4$ crystallizes in a K$_2$NiF$_4$-type structure, isostructural to Sr$_2$RuO$_4$ and Ba$_2$IrO$_4$. Magnetic, resistivity, and specific heat measurements for polycrystalline samples of Ba$_2$RhO$_4$ indicate that the system can be characterized as a correlated metal. Despite the close similarity to its Sr$_2$RuO$_4$ counterpart in the electronic specific heat coefficient and the Wilson ratio, Ba$_2$RhO$_4$ shows no signature of superconductivity down to 0.16 K. Whereas the Fermi surface topology has reminiscent pieces of Sr$_2$RuO$_4$, an electron-like e$_g$-($d_{x^2-y^2}$) band descends below the Fermi level, making of this compound unique also as a metallic counterpart of the spin-orbit-coupled Mott insulator Ba$_2$IrO$_4$.