Understanding and dealing with inference biases in gravitational-wave (GW) parameter estimation when a plethora of signals are present in the data is one of the key challenges for the analysis of data from future GW detectors. Working within the linear signal approximation, we describe generic metrics to predict inference biases on GW source parameters in the presence of confusion noise from unfitted foregrounds, from overlapping signals that coalesce close in time to one another, and from residuals of other signals that have been incorrectly fitted out. We illustrate the formalism with simplified, yet realistic, scenarios appropriate to third-generation ground-based (Einstein Telescope) and space-based (LISA) detectors, and demonstrate its validity against Monte-Carlo simulations. We find it to be a reliable tool to cheaply predict the extent and direction of the biases. Finally, we show how this formalism can be used to correct for biases that arise in the sequential characterisation of multiple sources in a single data set, which could be a valuable tool to use within a global-fit analysis pipeline.