We perform $N$-body simulations of the early phases of open cluster evolution including a large population of planetesimals, initially arranged in Kuiper-belt like discs around each star. Using a new, 4th-order and time-reversible $N$-body code on Graphics Processing Units (GPUs), we evolve the whole system under the stellar gravity, i.e. treating planetesimals as test particles, and consider two types of initial cluster models, similar to IC348 and the Hyades, respectively. In both cases, planetesimals can be dynamically excited, transferred between stars or liberated to become free-floating (such as A/2017 U1 or Oumuamua) during the early cluster evolution. We find that planetesimals captured from another star are not necessarily dynamically distinct from those native to a star. After an encounter both native and captured planetesimals can exhibit aligned periastrons, qualitatively similar to that seen in the Solar system and commonly thought to be the signature of Planet 9. We discuss the implications of our results for both our Solar system and exoplanetary systems.