Off-shell Higgs at the high mass tail may shed light on the underlying mechanism of the electroweak symmetry breaking. Due to the large cancellation in the standard model (SM) between the box and Higgs-mediated triangle diagrams, the $ggto WW(ZZ)$ process in the SM is dominated by the $V_T V_T$ transverse-mode at the high mass tail. The cancellation does not necessarily hold, when there is a sufficiently large new physics contribution resulting in $V_LV_L$ longitudinal mode, which is commonly the case when the Higgs sector is modified. Thus the $V_LV_L$ final states in the high mass tail can be utilized as a sensitive probe for such models. In the paper we focus on a study of the $gg to ZZ$ process in the fully leptonic decay modes, proposing to utilize the polarization modes of the off-shell Higgs to probe new physics, whose contribution mainly shows in the longitudinal mode. As examples, we analyze three different Higgs sector new physics cases (Higgs portal with a light scalar, a broad-width scalar that mixes with the Higgs, and quantum critical Higgs models), and demonstrate that the angular information relating to the polarization serves as very sensitive probe for such new physics.