Efficient Nearest Neighbor Language Models

Non-parametric neural language models (NLMs) learn predictive distributions of text utilizing an external datastore, which allows them to learn through explicitly memorizing the training datapoints. While effective, these models often require retrieval from a large datastore at test time, significantly increasing the inference overhead and thus limiting the deployment of non-parametric NLMs in practical applications. In this paper, we take the recently proposed k-nearest neighbors language model as an example, exploring methods to improve its efficiency along various dimensions. Experiments on the standard WikiText-103 benchmark and domain-adaptation datasets show that our methods are able to achieve up to a 6x speed-up in inference speed while retaining comparable performance. The empirical analysis we present may provide guidelines for future research seeking to develop or deploy more efficient non-parametric NLMs.

Applied Nonparametric Regression Methods as Alternatives to the Linear Regression Methods (Comparative Study)

Linear regression methods impose strong constraints on regression models, especially on the error terms where it assumes that it is independent and follows normal distribution, and this may not be satisfied in many studies, leading to bias that cannot be ignored from the actual model, which affects the credibility of the study. We present in this paper the problem of estimating the regression function using the Nadarya Watson kernel and k- nearest neighbor estimators as alternatives to the parametric linear regression estimators through a simulation study on an imposed model, where we conducted a comparative study between these methods using the statistical programming language R in order to know the best of these estimations. Where the mean squares errors (MSE) was used to determine the best estimate. The results of the simulation study also indicate the effectiveness and efficiency of the nonparametric in the representation of the regression function as compared to linear regression estimators, and indicate the convergence of the performance of these two estimates.

Comparative Study Of Several Mathematical Methods for building Continuous surface Based On 3D Laser Scanning Clouds

In this paper, comparisons between several mathematical interpolation methods applied on high accuracy and huge laser clouds which only represents the DTM. In order to implicate the aforementioned, a group of a variety of Laser Scanned Areas has been chosen to represent different types of terrain including complex and flat terrain, taking into account that man-made features are not involved in this study and that different Laser Clouds density is used to make the study more general. A different set of algorithms were applied to conclude which one is more suitable. This step was followed by the comparison between different interpolation results. The results have shown that the Points' Density has a great impact on a manner in which the optimal interpolation method is applied. Moreover, it has proven that the Nearest Neighbor Algorithm is the best applied method compared with the other alternatives