We aim at formulating a higher-spin gravity theory around AdS$_2$ relevant for holography. As a first step, we investigate its kinematics by identifying the low-dimensional cousins of the standard higher-dimensional structures in higher-spin gravity such as the singleton, the higher-spin symmetry algebra, the higher-rank gauge and matter fields, etc. In particular, the higher-spin algebra is given here by $hs[lambda]$ and parameterized by a real parameter $lambda$. The singleton is defined to be a Verma module of the AdS$_2$ isometry subalgebra $so(2,1) subset hs[lambda]$ with conformal weight $Delta = frac{1pmlambda}{2},$. On the one hand, the spectrum of local modes is determined by the Flato-Fronsdal theorem for the tensor product of two such singletons. It is given by an infinite tower of massive scalar fields in AdS$_2$ with ascending masses expressed in terms of $lambda$. On the other hand, the higher-spin fields arising through the gauging of $hs[lambda]$ algebra do not propagate local degrees of freedom. Our analysis of the spectrum suggests that AdS$_2$ higher-spin gravity is a theory of an infinite collection of massive scalars with fine-tuned masses, interacting with infinitely many topological gauge fields. Finally, we discuss the holographic CFT$_1$ duals of the kinematical structures identified in the bulk.