It is usually considered that the spectrum of an optical cavity coupled to an atomic medium does not exhibit a normal-mode splitting unless the system satisfies the strong coupling condition, meaning the Rabi frequency of the coherent coupling exceeds the decay rates of atom and cavity excitations. Here we show that this need not be the case, but depends on the way in which the coupled system is probed. Measurements of the reflection of a probe laser from the input mirror of an overdamped cavity reveal an avoided crossing in the spectrum which is not observed when driving the atoms directly and measuring the Purcell-enhanced cavity emission. We understand these observations by noting a formal correspondence with electromagnetically-induced transparency of a three-level atom in free space, where our cavity acts as the absorbing medium and the coupled atoms play the role of the control field.