Quantum vortices naturally emerge in rotating Bose-Einstein condensates (BECs) and, similarly to their classical counterparts, allow the study of a range of interesting out-of-equilibrium phenomena like turbulence and chaos. However, the study of such phenomena requires to determine the precise location of each vortex within a BEC, which becomes challenging when either only the condensate density is available or sources of noise are present, as is typically the case in experimental settings. Here, we introduce a machine learning based vortex detector motivated by state-of-the-art object detection methods that can accurately locate vortices in simulated BEC density images. Our model allows for robust and real-time detection in noisy and non-equilibrium configurations. Furthermore, the network can distinguish between vortices and anti-vortices if the condensate phase profile is also available. We anticipate that our vortex detector will be advantageous both for experimental and theoretical studies of the static and dynamical properties of vortex configurations in BECs.