We report the magnetization ($chi$, $M$), specific heat ($C_{text{P}}$), and neutron powder diffraction results on a quasi-two-dimensional $S$ = 2 square lattice antiferromagnet Ba$_2$FeSi$_2$O$_7$ consisting of FeO$_4$ tetragons with a large compressive distortion (27%). Despite of the quasi-two-dimensional lattice structure, both $chi$ and $C_{text{P}}$ present three dimensional magnetic long-range order below the Neel temperature $T_{text{N}}$ = 5.2 K. Neutron diffraction data shows a collinear $Q_{m}$ = (1,0,0.5) antiferromagnetic (AFM) structure with the in-plane ordered magnetic moment suppressed by 26% below $T_{text{N}}$. Both the AFM structure and the suppressed moments are well explained by the Monte Carlo simulation with a large single-ion ab-plane anisotropy $D$ = 1.4 meV and a rather small in-plane Heisenberg exchange $J_{text{intra}}$ = 0.15 meV. The characteristic two dimensional spin fluctuations can be recognized in the magnetic entropy release and diffuse scattering above $T_{text{N}}$. This new quasi-2D magnetic system also displays unusual non-monotonic dependence of the $T_{text{N}}$ as a function of magnetic field $H$.