Unconventional superconductors are of high interest due to their rich physics, a topical example being topological edge-states associated with $p$-wave superconductivity. A practical obstacle in studying such systems is the very low critical temperature $T_text{c}$ that is required to realize a $p$-wave superconducting phase in a material. We predict that the $T_text{c}$ of an intrinsic $p$-wave superconductor can be significantly enhanced by coupling it via an atomically thin ferromagnetic layer (F) to a conventional $s$-wave or a $d$-wave superconductor with a higher critical temperature. We show that this $T_text{c}$-boost is tunable via the direction of the magnetization in F. Moreover, we show that the enhancement in $T_text{c}$ can also be achieved using the Zeeman-effect of an external magnetic field. Our findings provide a way to increase $T_text{c}$ in $p$-wave superconductors in a controllable way and make the exotic physics associated with such materials more easily accessible experimentally.