Donoho and Stark have shown that a precise deterministic recovery of missing information contained in a time interval shorter than the time-frequency uncertainty limit is possible. We analyze this signal recovery mechanism from a physics point of vie
w and show that the well-known Shannon-Nyquist sampling theorem, which is fundamental in signal processing, also uses essentially the same mechanism. The uncertainty relation in the context of information theory, which is based on Fourier analysis, provides a criterion to distinguish Shannon-Nyquist sampling from compressed sensing. A new signal recovery formula, which is analogous to Donoho-Stark formula, is given using the idea of Shannon-Nyquist sampling; in this formulation, the smearing of information below the uncertainty limit as well as the recovery of information with specified bandwidth take place. We also discuss the recovery of states from the domain below the uncertainty limit of coordinate and momentum in quantum mechanics and show that in principle the state recovery works by assuming ideal measurement procedures. The recovery of the lost information in the sub-uncertainty domain means that the loss of information in such a small domain is not fatal, which is in accord with our common understanding of the uncertainty principle, although its precise recovery is something we are not used to in quantum mechanics. The uncertainty principle provides a universal sampling criterion covering both the classical Shannon-Nyquist sampling theorem and the quantum mechanical measurement.
Sparse coding, which represents a data point as a sparse reconstruction code with regard to a dictionary, has been a popular data representation method. Meanwhile, in database retrieval problems, learning the ranking scores from data points plays an
important role. Up to now, these two problems have always been considered separately, assuming that data coding and ranking are two independent and irrelevant problems. However, is there any internal relationship between sparse coding and ranking score learning? If yes, how to explore and make use of this internal relationship? In this paper, we try to answer these questions by developing the first joint sparse coding and ranking score learning algorithm. To explore the local distribution in the sparse code space, and also to bridge coding and ranking problems, we assume that in the neighborhood of each data point, the ranking scores can be approximated from the corresponding sparse codes by a local linear function. By considering the local approximation error of ranking scores, the reconstruction error and sparsity of sparse coding, and the query information provided by the user, we construct a unified objective function for learning of sparse codes, the dictionary and ranking scores. We further develop an iterative algorithm to solve this optimization problem.
