Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userFitting very flexible models: Linear regression with large numbers of parameters
85
0
0.0
(
0
)
Authors
David W. Hogg
Ask ChatGPT about the research
No Arabic abstract
There are many uses for linear fitting; the context here is interpolation and denoising of data, as when you have calibration data and you want to fit a smooth, flexible function to those data. Or you want to fit a flexible function to de-trend a time series or normalize a spectrum. In these contexts, investigators often choose a polynomial basis, or a Fourier basis, or wavelets, or something equally general. They also choose an order, or number of basis functions to fit, and (often) some kind of regularization. We discuss how this basis-function fitting is done, with ordinary least squares and extensions thereof. We emphasize that it is often valuable to choose far more parameters than data points, despite folk rules to the contrary: Suitably regularized models with enormous numbers of parameters generalize well and make good predictions for held-out data; over-fitting is not (mainly) a problem of having too many parameters. It is even possible to take the limit of infinite parameters, at which, if the basis and regularization are chosen correctly, the least-squares fit becomes the mean of a Gaussian process. We recommend cross-validation as a good empirical method for model selection (for example, setting the number of parameters and the form of the regularization), and jackknife resampling as a good empirical method for estimating the uncertainties of the predictions made by the model. We also give advice for building stable computational implementations.
rate research
Read More
In a statistical analysis in Particle Physics, nuisance parameters can be introduced to take into account various types of systematic uncertainties. The best estimate of such a parameter is often modeled as a Gaussian distributed variable with a given standard deviation (the corresponding systematic error). Although the assigned systematic errors are usually treated as constants, in general they are themselves uncertain. A type of model is presented where the uncertainty in the assigned systematic errors is taken into account. Estimates of the systematic variances are modeled as gamma distributed random variables. The resulting confidence intervals show interesting and useful properties. For example, when averaging measurements to estimate their mean, the size of the confidence interval increases for decreasing goodness-of-fit, and averages have reduced sensitivity to outliers. The basic properties of the model are presented and several examples relevant for Particle Physics are explored.
Fitting a simplifying model with several parameters to real data of complex objects is a highly nontrivial task, but enables the possibility to get insights into the objects physics. Here, we present a method to infer the parameters of the model, the model error as well as the statistics of the model error. This method relies on the usage of many data sets in a simultaneous analysis in order to overcome the problems caused by the degeneracy between model parameters and model error. Errors in the modeling of the measurement instrument can be absorbed in the model error allowing for applications with complex instruments.
We give an example of a formula involving the sinc function that holds for every N = 0, 1, 2, ..., up to about 10^102832732165, then fails for all larger N. We give another example that begins to fail after about N ~ exp(exp(exp(exp(exp(exp(e)))))). This number is larger than the Skewes numbers.
In a previous article we developed an approach to the optimal (minimum variance, unbiased) statistical estimation technique for the equilibrium displacement of a damped, harmonic oscillator in the presence of thermal noise. Here, we expand that work to include the optimal estimation of several linear parameters from a continuous time series. We show that working in the basis of the thermal driving force both simplifies the calculations and provides additional insight to why various approximate (not optimal) estimation techniques perform as they do. To illustrate this point, we compare the variance in the optimal estimator that we derive for thermal noise with those of two approximate methods which, like the optimal estimator, suppress the contribution to the variance that would come from the irrelevant, resonant motion of the oscillator. We discuss how these methods fare when the dominant noise process is either white displacement noise or noise with power spectral density that is inversely proportional to the frequency ($1/f$ noise). We also construct, in the basis of the driving force, an estimator that performs well for a mixture of white noise and thermal noise. To find the optimal multi-parameter estimators for thermal noise, we derive and illustrate a generalization of traditional matrix methods for parameter estimation that can accommodate continuous data. We discuss how this approach may help refine the design of experiments as they allow an exact, quantitative comparison of the precision of estimated parameters under various data acquisition and data analysis strategies.
We derive a Bayesian framework for incorporating selection effects into population analyses. We allow for both measurement uncertainty in individual measurements and, crucially, for selection biases on the population of measurements, and show how to extract the parameters of the underlying distribution based on a set of observations sampled from this distribution. We illustrate the performance of this framework with an example from gravitational-wave astrophysics, demonstrating that the mass ratio distribution of merging compact-object binaries can be extracted from Malmquist-biased observations with substantial measurement uncertainty.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
