X-rays from massive stars are ubiquitous yet not clearly understood. In an XMM-Newton observation devoted to observe the first site of star formation in the $rho$ Ophiuchi dark cloud, we detect smoothly variable X-ray emission from the B2IV+B2V system of $rho$ Ophiuchi. Tentatively we assign the emission to the primary component. The light curve of the pn camera shows a first phase of low, almost steady rate, then a rise phase of duration of 10 ks, followed by a high rate phase. The variability is seen primarily in the band 1.0-8.0 keV while little variability is detected below 1 keV. The spectral analysis of the three phases reveals the presence of a hot component at 3.0 keV that adds up to two relatively cold components at 0.9 keV and 2.2 keV. We explain the smooth variability with the emergence of an extended active region on the surface of the primary star due to its fast rotation (v $sin~i sim315$ km/s). We estimate that the region has diameter in the range $0.5-0.6$ R$_*$. The hard X-ray emission and its variability hint a magnetic origin, as suggested for few other late-O$-$early-B type stars. We also discuss an alternative explanation based on the emergence from occultation of a young (5-10 Myr) low mass companion bright and hot in X-rays.