We present the first spatially resolved map of stellar proper motions within the central ($sim$3.1 $times$ 2.4 kpc) regions of the Small Magellanic Cloud (SMC). The data used for this study encompasses four tiles from the ongoing near-infrared VISTA survey of the Magellanic Clouds system and covers a total contiguous area on the sky of $sim$6.81 deg$^2$. Proper motions have been calculated independently in two dimensions from the spatial offsets in the $K_s$ filter over time baselines between 22 and 27 months. The reflex motions of $sim$33~000 background galaxies are used to calibrate the stellar motions to an absolute scale. The resulting catalog is composed of more than 690 000 stars which have been selected based on their position in the $(J-K_s, K_s)$ color-magnitude diagram. For the median absolute proper motion of the SMC, we find ($mu_{alpha}mathrm{cos}(delta)$, $mu_{delta}$) = (1.087 $pm$ 0.192 (sys.) $pm$ 0.003 (stat.), $-$1.187 $pm$ 0.008 (sys.) $pm$ 0.003 (stat.)) mas yr$^{-1}$, consistent with previous studies. Mapping the proper motions as a function of position within the SMC reveals a non uniform velocity pattern indicative of a tidal feature behind the main body of the SMC and a flow of stars in the South-East moving predominantly along the line-of-sight.