The complex magnetism and transport properties of tetragonal Ce$_3$Cu$_4$As$_4$O$_2$ were examined through neutron scattering and physical properties measurements on polycrystalline samples. The lamellar structure consists of alternating layers of $rm CeCu_4As_4$ with a single square Ce lattice and oxygen-linked Ce bi-layer $rm Ce_2O_2$. Extending along $bf c$, a tube-like Fermi surface from DFT calculations points to a quasi-two-dimensional electronic system. Peaks in the specific heat at the Ne{e}l temperature $T_{N}=24$ $rm K$, $T_{2}~=~16 $ $rm K$ and $T_{3}~=~1.9$ $ rm K$ indicate three magnetic phase transitions or distinct cross-over phenomena. For $T<T_{N}$ neutron diffraction indicates the development of ferromagnetic ab sheets for both Ce sites, with alternating polarization along $bf{c}$, a wave vector ${bf k}_{1}={bf c}^*$. For $T<T_{2}$, quasi-two-dimensional low-energy spin fluctuations with ${bf k}_{2}=frac{1}{2}{bf a}^*$ and polarized perpendicular to ${bf k}_{2}$ are suppressed. The data are consistent with quasi-two-dimensional antiferromagnetic order in the $rm CeCu_4As_4$ planes polarized along the ${bf k}_{2}$ wave vector. $T_{3}$ marks a spin-flop transition where the ${bf k}_{1}$ staggered magnetization switches to in-plane polarization. While the narrow 4f bands lie deep below the Fermi surface, there are significant transport anomalies associated with the transitions; in particular a substantial reduction in resistivity for $T<T_{N}$. At $T=100$ $ rm mK$ the ${bf k}_1$ modulated staggered moment is $0.85~mu_B$, which matches the $0.8~mu_B$ saturation magnetization achieved for H $~=~7$ $ rm T$ at $T~=~2$ $ rm K$. From low T Lorentzian fits the correlation length is in excess of 75 AA. We argue the unusual sequence of magnetic transitions results from competing interactions and anisotropies for the two Ce sites.