Gravitational waves from binary black holes that are gravitationally lensed can be distorted by small microlenses along the line of sight. Microlenses with masses of a few tens of solar masses, and that are close to a critical curve in the lens plane, can introduce a time delay of a few millisecond. Such time delay would result in distinctive interference patterns in the gravitational wave that can be measured with current experiments such as LIGO/Virgo. We consider the particular case of primordial black holes with masses between 5 and 50 solar masses acting as microlenses. We study the effect of a population of primordial black holes constituting a fraction of the dark matter, and contained in a macrolens (galaxy or cluster), over gravitational waves that are being lensed by the combined effect of the macrolens plus microlenses. We find that at the typical magnifications expected for observed GW events, the fraction of dark matter in the form of compact microlenses, such as primordial black holes, can be constrained to percent level. Similarly, if a small percentage of the dark matter is in the form of microlenses with a few tens of solar masses, at sufficiently large magnification factors, all gravitational waves will show interference effects. These effects could have an impact on the inferred parameters. The effect is more important for macroimages with negative parity, which usually arrive after the macroimages with positive parity.