Strongly correlated electrons in layered perovskite structures have been the birthplace of high-temperature superconductivity, spin liquid, and quantum criticality. Specifically, the cuprate materials with layered structures made of corner sharing square planar CuO$_4$ units have been intensely studied due to their Mott insulating grounds state which leads to high-temperature superconductivity upon doping. Identifying new compounds with similar lattice and electronic structures has become a challenge in solid state chemistry. Here, we report the hydrothermal crystal growth of a new copper tellurite sulfate Cu$_3$(TeO$_4$)(SO$_4$)$cdot$H$_2$O, a promising alternative to layered perovskites. The orthorhombic phase (space group $Pnma$) is made of corrugated layers of corner-sharing CuO$_4$ square-planar units that are edge-shared with TeO$_4$ units. The layers are linked by slabs of corner-sharing CuO$_4$ and SO$_4$. Using both the bond valence sum analysis and magnetization data, we find purely Cu$^{2+}$ ions within the layers, but a mixed valence of Cu$^{2+}$/Cu${^+}$ between the layers. Cu$_3$(TeO$_4$)(SO$_4$)$cdot$H$_2$O undergoes an antiferromagnetic transition at $T_N$=67 K marked by a peak in the magnetic susceptibility. Upon further cooling, a spin-canting transition occurs at $T^{star}$=12 K evidenced by a kink in the heat capacity. The spin-canting transition is explained based on a $J_1$-$J_2$ model of magnetic interactions, which is consistent with the slightly different in-plane super-exchange paths. We present Cu$_3$(TeO$_4$)(SO$_4$)$cdot$H$_2$O as a promising platform for the future doping and strain experiments that could tune the Mott insulating ground state into superconducting or spin liquid states.