The observation of coherent elastic neutrino nucleus scattering (CE$ u$NS) by the COHERENT collaboration in 2017 has opened a new window to both test Standard Model predictions at relatively low energies and probe new physics scenarios. Our investigations show, however, that a careful treatment of the statistical methods used to analyze the data is essential to derive correct constraints and bounds on new physics parameters. In this manuscript we perform a detailed analysis of the publicly available COHERENT CsI data making use of all available background data. We point out that Wilks theorem is not fulfilled in general and a calculation of the confidence regions via Monte Carlo simulations following a Feldman-Cousins procedure is necessary. As an example for the necessity of this approach to test new physics scenarios we quantify the allowed ranges for several scenarios with neutrino non-standard interactions. Furthermore, we provide accompanying code to enable an easy implementation of other new physics scenarios as well as data files of our results.
Download