CsCa$_2$Fe$_4$As$_4$F$_2$ is a newly discovered iron-based superconductor with $T_mathrm{c}sim$ 30 K containing double Fe$_2$As$_2$ layers that are separated by insulating Ca$_2$F$_2$ spacer layers. Here we report the transport and magnetization measurements on CsCa$_2$Fe$_4$As$_4$F$_2$ single crystals grown for the first time using the self flux of CsAs. We observed a huge resistivity anisotropy $rho_c(T)/rho_{ab}(T)$, which increases with decreasing temperature, from 750 at 300 K to 3150 at 32 K. The $rho_c(T)$ data exhibit a non-metallic behavior above $sim$140 K, suggesting an incoherent electronic state at high temperatures due to the dimension crossover. The superconducting onset transition temperature in $rho_{ab}$ is 0.7 K higher than that in $rho_c$, suggesting two-dimensional (2D) superconducting fluctuations. The lower and upper critical fields also show an exceptional anisotropy among iron-based superconductors. The $H_{c1}^bot(T)$ data are well fitted using the model with two $s$-wave-like superconducting gaps, $Delta_1(0)=6.75$ meV and $Delta_2(0)=2.32$ meV. The inter-plane coherence length $xi_c(0)$ is $3.6$ AA, remarkably smaller than the distance between conducting layers (8.6 AA), consolidating the 2D nature in the title material.