The discovery of planetary systems outside of the solar system has challenged some of the tenets of planetary formation. Among the difficult-to-explain observations, are systems with a giant planet orbiting a very-low mass star, such as the recently discovered GJ~3512b planetary system, where a Jupiter-like planet orbits an $M$-star in a tight and eccentric orbit. Systems such as this one are not predicted by the core accretion theory of planet formation. Here we suggest a novel mechanism, in which the giant planet is born around a more typical Sun-like star ($M_{*,1}$), but is subsequently exchanged during a dynamical interaction with a flyby low-mass star ($M_{*,2}$). We perform state-of-the-art $N$-body simulations with $M_{*,1}=1M_odot$ and $M_{*,2}=0.1M_odot$ to study the statistical outcomes of this interaction, and show that exchanges result in high eccentricities for the new orbit around the low-mass star, while about half of the outcomes result in tighter orbits than the planet had around its birth star. We numerically compute the cross section for planet exchange, and show that an upper limit for the probability per planetary system to have undergone such an event is $Gammasim 4.4(M_{rm c}/100M_odot)^{-2}(a_{rm p}/{rm AU}) (sigma/1,{rm km},{rm s}^{-1})^{5}$Gyr$^{-1}$, where $a_{rm p}$ is the planet semi-major axis around the birth star, $sigma$ the velocity dispersion of the star cluster, and $M_{rm c}$ the total mass of the star cluster. Hence these planet exchanges could be relatively common for stars born in open clusters and groups, should already be observed in the exoplanet database, and provide new avenues to create unexpected planetary architectures.