BaCuSi$_2$O$_6$, a $S=1/2$ quantum antiferromagnet with a double-layer structure of Cu$^{2+}$ ions in a distorted planar-rectangular coordination and with a dimerized spin singlet ground state, is studied by means of the electron paramagnetic resonance technique. It is argued that multiple absorptions observed at low temperatures are intimately related to a thermally-activated spin-triplet exciton superstructure. Analysis of the angular dependence of exciton modes in BaCuSi$_2$O$_6$ allows us to accurately estimate anisotropy parameters. In addition, the temperature dependence of EPR intensity and linewidth is discussed.