Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userFast simulated annealing in $R^d$ and an application to maximum likelihood estimation
105
0
0.0
(
0
)
Authors
Sylvain Rubenthaler
Ask ChatGPT about the research
No Arabic abstract
Using classical simulated annealing to maximise a function $psi$ defined on a subset of $R^d$, the probability $p(psi(theta_n)leq psi_{max}-epsilon)$ tends to zero at a logarithmic rate as $n$ increases; here $theta_n$ is the state in the $n$-th stage of the simulated annealing algorithm and $psi_{max}$ is the maximal value of $psi$. We propose a modified scheme for which this probability is of order $n^{-1/3}log n$, and hence vanishes at an algebraic rate. To obtain this faster rate, the exponentially decaying acceptance probability of classical simulated annealing is replaced by a more heavy-tailed function, and the system is cooled faster. We also show how the algorithm may be applied to functions that cannot be computed exactly but only approximated, and give an example of maximising the log-likelihood function for a state-space model.
rate research
Read More
Maximum simulated likelihood estimation of mixed multinomial logit (MMNL) or probit models requires evaluation of a multidimensional integral. Quasi-Monte Carlo (QMC) methods such as shuffled and scrambled Halton sequences and modified Latin hypercube sampling (MLHS) are workhorse methods for integral approximation. A few earlier studies explored the potential of sparse grid quadrature (SGQ), but this approximation suffers from negative weights. As an alternative to QMC and SGQ, we looked into the recently developed designed quadrature (DQ) method. DQ requires fewer nodes to get the same level of accuracy as of QMC and SGQ, is as easy to implement, ensures positivity of weights, and can be created on any general polynomial spaces. We benchmarked DQ against QMC in a Monte Carlo study under different data generating processes with a varying number of random parameters (3, 5, and 10) and variance-covariance structures (diagonal and full). Whereas DQ significantly outperformed QMC in the diagonal variance-covariance scenario, it could also achieve a better model fit and recover true parameters with fewer nodes (i.e., relatively lower computation time) in the full variance-covariance scenario. Finally, we evaluated the performance of DQ in a case study to understand preferences for mobility-on-demand services in New York City. In estimating MMNL with five random parameters, DQ achieved better fit and statistical significance of parameters with just 200 nodes as compared to 1000 QMC draws, making DQ around five times faster than QMC methods.
Given a target function $U$ to minimize on a finite state space $mathcal{X}$, a proposal chain with generator $Q$ and a cooling schedule $T(t)$ that depends on time $t$, in this paper we study two types of simulated annealing (SA) algorithms with generators $M_{1,t}(Q,U,T(t))$ and $M_{2,t}(Q,U,T(t))$ respectively. While $M_{1,t}$ is the classical SA algorithm, we introduce a simple and improved variant that we call $M_{2,t}$ which provably converges faster. When $T(t) > c_{M_2}/log(t+1)$ follows the logarithmic cooling schedule, our proposed algorithm is strongly ergodic both in total variation and in relative entropy, and converges to the set of global minima, where $c_{M_2}$ is a constant that we explicitly identify. If $c_{M_1}$ is the optimal hill-climbing constant that appears in logarithmic cooling of $M_{1,t}$, we show that $c_{M_1} geq c_{M_2}$ and give simple conditions under which $c_{M_1} > c_{M_2}$. Our proposed $M_{2,t}$ thus converges under a faster logarithmic cooling in this regime. The other situation that we investigate corresponds to $c_{M_1} > c_{M_2} = 0$, where we give a class of fast and non-logarithmic cooling schedule that works for $M_{2,t}$ (but not for $M_{1,t}$). In addition to these asymptotic convergence results, we compare and analyze finite-time behaviour between these two annealing algorithms as well. Finally, we present two algorithms to simulate $M_{2,t}$.
The asymptotic variance of the maximum likelihood estimate is proved to decrease when the maximization is restricted to a subspace that contains the true parameter value. Maximum likelihood estimation allows a systematic fitting of covariance models to the sample, which is important in data assimilation. The hierarchical maximum likelihood approach is applied to the spectral diagonal covariance model with different parameterizations of eigenvalue decay, and to the sparse inverse covariance model with specified parameter values on different sets of nonzero entries. It is shown computationally that using smaller sets of parameters can decrease the sampling noise in high dimension substantially.
The mixed fractional Vasicek model, which is an extended model of the traditional Vasicek model, has been widely used in modelling volatility, interest rate and exchange rate. Obviously, if some phenomenon are modeled by the mixed fractional Vasicek model, statistical inference for this process is of great interest. Based on continuous time observations, this paper considers the problem of estimating the drift parameters in the mixed fractional Vasicek model. We will propose the maximum likelihood estimators of the drift parameters in the mixed fractional Vasicek model with the Radon-Nikodym derivative for a mixed fractional Brownian motion. Using the fundamental martingale and the Laplace transform, both the strong consistency and the asymptotic normality of the maximum likelihood estimators have been established for all $Hin(0,1)$, $H eq 1/2$.
This paper considers a new method for the binary asteroid orbit determination problem. The method is based on the Bayesian approach with a global optimisation algorithm. The orbital parameters to be determined are modelled through an a posteriori distribution made of a priori and likelihood terms. The first term constrains the parameters space and it allows the introduction of available knowledge about the orbit. The second term is based on given observations and it allows us to use and compare different observational error models. Once the a posteriori model is built, the estimator of the orbital parameters is computed using a global optimisation procedure: the simulated annealing algorithm. The maximum a posteriori (MAP) techniques are verified using simulated and real data. The obtained results validate the proposed method. The new approach guarantees independence of the initial parameters estimation and theoretical convergence towards the global optimisation solution. It is particularly useful in these situations, whenever a good initial orbit estimation is difficult to get, whenever observations are not well-sampled, and whenever the statistical behaviour of the observational errors cannot be stated Gaussian like.
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
