Some recent works have introduced a quantum twist to the concept of complementarity, exemplified by a setup in which the which-way detector is in a superposition of being present and absent. It has been argued that such experiments allow measurement of particle-like and wave-like behavior at the same time. Here we derive an inequality which puts a bound on the visibility of interference and the amount of which-way information that one can obtain, in the context of such modified experiments. As the wave-aspect can only be revealed by an ensemble of detections, we argue that in such experiments, a single detection can contribute only to one subensemble, corresponding to either wave-aspect or particle aspect. This way, each detected particle behaves either as particle or as wave, never both, and Bohrs complementarity is fully respected.