We theoretically demonstrate a strong dependence of the annihilation rate between (singlet) excitons on the sign of dipole-dipole couplings between molecules. For molecular H-aggregates, where this sign is positive, the phase relation of the delocalized two-exciton wavefunctions causes a destructive interference in the annihilation probability. For J-aggregates, where this sign is negative, the interference is constructive instead, as a result of which no such coherent suppression of the annihilation rate occurs. As a consequence, room temperature annihilation rates of typical H- and J-aggregates differ by a factor of ~3, while an order of magnitude difference is found for low-temperature aggregates with a low degree of disorder. These findings, which explain experimental observations, reveal a fundamental principle underlying exciton-exciton annihilation, with major implications for technological devices and experimental studies involving high excitation densities.