We propose a multi-level type operator that can be used in the framework of operator (or Cald{e}ron) preconditioning to construct uniform preconditioners for negative order operators discretized by piecewise polynomials on a family of possibly locally refined partitions. The cost of applying this multi-level operator scales linearly in the number of mesh cells. Therefore, it provides a uniform preconditioner that can be applied in linear complexity when used within the preconditioning framework from our earlier work [Math. of Comp., 322(89) (2020), pp. 645-674].