Experimentally adjudicating between different causal accounts of Bell inequality violations via statistical model selection

Bell inequalities follow from a set of seemingly natural assumptions about how to provide a causal model of a Bell experiment. In the face of their violation, two types of causal models that modify some of these assumptions have been proposed: (i) those that are parametrically conservative and structurally radical, such as models where the parameters are conditional probability distributions (termed classical causal models) but where one posits inter-lab causal influences or superdeterminism, and (ii) those that are parametrically radical and structurally conservative, such as models where the labs are taken to be connected only by a common cause but where conditional probabilities are replaced by conditional density operators (these are termed quantum causal models). We here seek to adjudicate between these alternatives based on their predictive power. The data from a Bell experiment is divided into a training set and a test set, and for each causal model, the parameters that yield the best fit for the training set are estimated and then used to make predictions about the test set. Our main result is that the structurally radical classical causal models are disfavoured relative to the structurally conservative quantum causal model. Their lower predictive power seems to be due to the fact that, unlike the quantum causal model, they are prone to a certain type of overfitting wherein statistical fluctuations away from the no-signalling condition are mistaken for real features. Our technique shows that it is possible to witness quantumness even in a Bell experiment that does not close the locality loophole. It also overturns the notion that it is impossible to experimentally test the plausibility of superdeterminist models of Bell inequality violations.