The bulk photovoltaic effect (BPVE) has attracted an increasing interest due to its potential to overcome the efficiency limit of traditional photovoltaics, and much effort has been devoted to understanding its underlying physics. However, previous work has shown that theoretical models of the shift current and the phonon-assisted ballistic current in real materials do not fully account for the experimental BPVE photocurrent, and so other mechanisms should be investigated in order to obtain a complete picture of BPVE. In this Letter, we demonstrate two approaches that enable the ab initio calculation of the ballistic current originating from the electron-hole interaction in semiconductors. Using BaTiO$_3$ and MoS$_2$ as two examples, we show clearly that for them the asymmetric scattering from electron-hole interaction is less appreciable than that from electron-phonon interaction, indicating more scattering processes need to be included to further improve the BPVE theory. Moreover, our approaches build up a venue for predicting and designing materials with larger ballistic current due to electron-hole interactions.