(Abridged) The behaviour of mass loss across bi-stability jump is a key uncertainty in models of massive stars. While an increase in mass loss is theoretically predicted, this has so far not been observationally confirmed. However, radiation-driven winds of massive stars are known to exhibit clumpy structures triggered by the line-deshadowing instability (LDI). Wind clumping affects empirical mass-loss rates inferred from density square-dependent spectral diagnostics. If clumping properties differ significantly for O and B supergiants across the bi-stability jump, this may help alleviate discrepancies between theory and observations. We investigate with analytical and numerical tools how the onset of clumpy structures behaves in the winds of O supergiants (OSG) and B supergiants (BSG) across the bi-stability jump. We derive a scaling relation for the linear growth rate of the LDI for a single optically thick line and apply it in both regimes. We run 1D time-dependent line-driven instability simulations to study the non-linear evolution of the LDI in clumpy OSG and BSG winds. Linear perturbation analysis for a single line shows that the LDI linear growth rate scales strongly with stellar effective temperature and terminal wind speed. This implies significantly lower growth rates for (cooler, slower) BSG winds than for OSG winds. This is confirmed by the non-linear simulations, which show significant differences in OSG and BSG wind structure formation, with the latter characterized by significantly weaker clumping factors and lower velocity dispersions. This suggests that lower correction factors due to clumping should be employed when deriving empirical mass-loss rates for BSGs on the cool side of the bi-stability jump. Moreover, the non-linear simulations provide a theoretical background toward explaining the general lack of observed intrinsic X-ray emission in (single) B star winds.