Motivated by the widespread experimental observations of nematicity in strongly underdoped cuprate superconductors, we investigate the possibility of enhanced nematic fluctuations in the vicinity of a Mott insulator that displays Neel-type antiferromagnetic order. By performing a strong-coupling expansion of an effective model that contains both Cu-$d$ and O-$p$ orbitals on the square lattice, we demonstrate that quadrupolar fluctuations in the $p$-orbitals inevitably generate a biquadratic coupling between the spins of the $d$-orbitals. The key point revealed by our classical Monte Carlo simulations and large-$N$ calculations is that the biquadratic term favors local stripe-like magnetic fluctuations, which result in an enhanced nematic susceptibility that onsets at a temperature scale determined by the effective Heisenberg exchange $J$. We discuss the impact of this type of nematic order on the magnetic spectrum and outline possible implications on our understanding of nematicity in the cuprates.