Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userDensity estimation from an individual numerical sequence
181
0
0.0
(
0
)
Added by
Gusztav Morvai
Publication date
2007
fields
Informatics Engineering
and research's language is
English
Ask ChatGPT about the research
No Arabic abstract
This paper considers estimation of a univariate density from an individual numerical sequence. It is assumed that (i) the limiting relative frequencies of the numerical sequence are governed by an unknown density, and (ii) there is a known upper bound for the variation of the density on an increasing sequence of intervals. A simple estimation scheme is proposed, and is shown to be $L_1$ consistent when (i) and (ii) apply. In addition it is shown that there is no consistent estimation scheme for the set of individual sequences satisfying only condition (i).
rate research
Read More
We consider univariate regression estimation from an individual (non-random) sequence $(x_1,y_1),(x_2,y_2), ... in real times real$, which is stable in the sense that for each interval $A subseteq real$, (i) the limiting relative frequency of $A$ under $x_1, x_2, ...$ is governed by an unknown probability distribution $mu$, and (ii) the limiting average of those $y_i$ with $x_i in A$ is governed by an unknown regression function $m(cdot)$. A computationally simple scheme for estimating $m(cdot)$ is exhibited, and is shown to be $L_2$ consistent for stable sequences ${(x_i,y_i)}$ such that ${y_i}$ is bounded and there is a known upper bound for the variation of $m(cdot)$ on intervals of the form $(-i,i]$, $i geq 1$. Complementing this positive result, it is shown that there is no consistent estimation scheme for the family of stable sequences whose regression functions have finite variation, even under the restriction that $x_i in [0,1]$ and $y_i$ is binary-valued.
A new approach to $L_2$-consistent estimation of a general density functional using $k$-nearest neighbor distances is proposed, where the functional under consideration is in the form of the expectation of some function $f$ of the densities at each point. The estimator is designed to be asymptotically unbiased, using the convergence of the normalized volume of a $k$-nearest neighbor ball to a Gamma distribution in the large-sample limit, and naturally involves the inverse Laplace transform of a scaled version of the function $f.$ Some instantiations of the proposed estimator recover existing $k$-nearest neighbor based estimators of Shannon and Renyi entropies and Kullback--Leibler and Renyi divergences, and discover new consistent estimators for many other functionals such as logarithmic entropies and divergences. The $L_2$-consistency of the proposed estimator is established for a broad class of densities for general functionals, and the convergence rate in mean squared error is established as a function of the sample size for smooth, bounded densities.
We describe estimators $chi_n(X_0,X_1,...,X_n)$, which when applied to an unknown stationary process taking values from a countable alphabet ${cal X}$, converge almost surely to $k$ in case the process is a $k$-th order Markov chain and to infinity otherwise.
Let ${X_n}_{n=0}^{infty}$ be a stationary real-valued time series with unknown distribution. Our goal is to estimate the conditional expectation of $X_{n+1}$ based on the observations $X_i$, $0le ile n$ in a strongly consistent way. Bailey and Ryabko proved that this is not possible even for ergodic binary time series if one estimates at all values of $n$. We propose a very simple algorithm which will make prediction infinitely often at carefully selected stopping times chosen by our rule. We show that under certain conditions our procedure is strongly (pointwise) consistent, and $L_2$ consistent without any condition. An upper bound on the growth of the stopping times is also presented in this paper.
The forward estimation problem for stationary and ergodic time series ${X_n}_{n=0}^{infty}$ taking values from a finite alphabet ${cal X}$ is to estimate the probability that $X_{n+1}=x$ based on the observations $X_i$, $0le ile n$ without prior knowledge of the distribution of the process ${X_n}$. We present a simple procedure $g_n$ which is evaluated on the data segment $(X_0,...,X_n)$ and for which, ${rm error}(n) = |g_{n}(x)-P(X_{n+1}=x |X_0,...,X_n)|to 0$ almost surely for a subclass of all stationary and ergodic time series, while for the full class the Cesaro average of the error tends to zero almost surely and moreover, the error tends to zero in probability.
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
