The Oort cloud (OC) probably formed more than 4$,$Gyr ago and has been moving with the Sun in the Galaxy since, exposed to external influences, most prominently to the Galactic tide and passing field stars. Theories suggest that other stars might posses exocomets distributed similarly to our OC. We study the erosion of the OC and the possibility for capturing exocomets during the encounters with such field stars. We carry out simulations of flybys, where both stars are surrounded by a cloud of comets. We measure how many exocomets are transferred to the OC, how many OCs comets are lost, and how this depends on the other stars mass, velocity and impact parameter. Exocomets are transferred to the OC only during relatively slow ($lesssim0.5,$km$,$s$^{-1}$) and close ($lesssim10^5,$AU) flybys and these are expected to be extremely rare. Assuming that all passing stars are surrounded by a cloud of exocomets, we derive that the fraction of exocomets in the OC has been about $10^{-5}$--$10^{-4}$. Finally we simulate the OC for the whole lifetime of the Sun, taking into account the encounters and the tidal effects. The OC has lost 25--65% of its mass, mainly due to stellar encounters, and at most 10% (and usually much less) of its mass can be captured. However, exocomets are often lost shortly after the encounter that delivers them, due to the Galactic tide and consecutive encounters.