We investigate the anisotropic superconducting and magnetic properties of single-crystal RbEuFe$_4$As$_4$ using magnetotransport and magnetization measurements. We determine a magnetic ordering temperature of the Eu-moments of $T_m$ = 15 K and a superconducting transition temperature of $T_c$ = 36.8 K. The superconducting phase diagram is characterized by high upper critical field slopes of -70 kG/K and -42 kG/K for in-plane and out-of-plane fields, respectively, and a surprisingly low superconducting anisotropy of $Gamma$ = 1.7. Ginzburg-Landau parameters of $kappa_c sim 67$ and $kappa_{ab} sim 108$ indicate extreme type-II behavior. These superconducting properties are in line with those commonly seen in optimally doped Fe-based superconductors. In contrast, Eu-magnetism is quasi-two dimensional as evidenced by highly anisotropic in-plane and out-of-plane exchange constants of 0.6 K and $<$ 0.04 K. A consequence of the quasi-2D nature of the Eu-magnetism are strong magnetic fluctuation effects, a large suppression of the magnetic ordering temperature as compared to the Curie-Weiss temperature, and a cusp-like anomaly in the specific heat devoid of any singularity. Magnetization curves reveal a clear magnetic easy-plane anisotropy with in-plane and out-of-plane saturation fields of 2 kG and 4 kG.