We present a search for dark photon dark matter that could couple to gravitational-wave interferometers using data from Advanced LIGO and Virgos third observing run. To perform this analysis, we use two methods, one based on cross-correlation of the strain channels in the two nearly aligned LIGO detectors, and one that looks for excess power in the strain channels of the LIGO and Virgo detectors. The excess power method optimizes the Fourier Transform coherence time as a function of frequency, to account for the expected signal width due to Doppler modulations. We do not find any evidence of dark photon dark matter with a mass between $m_{rm A} sim 10^{-14}-10^{-11}$ eV/$c^2$, which corresponds to frequencies between 10-2000 Hz, and therefore provide upper limits on the square of the minimum coupling of dark photons to baryons, i.e. $U(1)_{rm B}$ dark matter. For the cross-correlation method, the best median constraint on the squared coupling is $sim1.31times10^{-47}$ at $m_{rm A}sim4.2times10^{-13}$ eV/$c^2$; for the other analysis, the best constraint is $sim 1.2times 10^{-47}$ at $m_{rm A}sim 5.7times 10^{-13}$ eV/$c^2$. These limits improve upon those obtained in direct dark matter detection experiments by a factor of $sim100$ for $m_{rm A}sim [2-4]times 10^{-13}$ eV/$c^2$.