Effect of the electric quadrupole moment, $Q$, is studied for positron-atom bound systems. It is demonstrated that for $Q >50$ a.u. the electric quadrupole potential is sufficiently strong to bind positron (or electron) even in the absence of the dipole polarization potential. Such large values of $Q$ are not known for atomic ground states, however, they exist in molecules and excited atoms. In the state $2s2p~^3P^o_2$ of beryllium, the quadrupole contribution makes difference between stable bound state and decay to Be$^+$ ion and positronium. In a majority of atoms the quadrupole contribution is small and can be neglected.