Critical behavior developed near a quantum phase transition, interesting in its own right, offers exciting opportunities to explore the universality of strongly-correlated systems near the ground state. Cold atoms in optical lattices, in particular, represent a paradigmatic system, for which the quantum phase transition between the superfluid and Mott insulator states can be externally induced by tuning the microscopic parameters. In this paper, we describe our approach to study quantum criticality of cesium atoms in a two-dimensional lattice based on in situ density measurements. Our research agenda involves testing critical scaling of thermodynamic observables and extracting transport properties in the quantum critical regime. We present and discuss experimental progress on both fronts. In particular, the thermodynamic measurement suggests that the equation of state near the critical point follows the predicted scaling law at low temperatures.