What is the most efficient way to generate random numbers device-independently using a photon pair source based on spontaneous parametric down conversion (SPDC)? We consider this question by comparing two implementations of a detection-loophole-free Bell test. In particular, we study in detail a scenario where a heralded single photon source (HSPS) is used to herald path-entangled states, i.e. entanglement between two spatial modes sharing a single photon and where non-locality is revealed using photon counting preceded by small displacement operations. We start by giving a theoretical description of such a measurement. We then show how to optimize the Bell-CHSH violation through a non-perturbative calculation, taking the main experimental imperfections into account. We finally bound the amount of randomness that can be extracted and compare it to the one obtained with the conventional scenario using photon pairs entangled e.g. in polarization and analyzed through photon counting. While the former requires higher overall detection efficiencies, it is far more efficient in terms of both the entropy per experimental run and the rate of random bit generation.