We study the electronic structure of the nodal line semimetal ZrSiTe both experimentally and theoretically. We find two different surface states in ZrSiTe - topological drumhead surface states and trivial floating band surface states. Using the spectra of Wilson loops, we show that a non-trivial Berry phase that exists in a confined region within the Brillouin Zone gives rise to the topological drumhead-type surface states. The $mathbb{Z}_2$ structure of the Berry phase induces a $mathbb{Z}_2$ modular arithmetic of the surface states, allowing surface states deriving from different nodal lines to hybridize and gap out, which can be probed by a set of Wilson loops. Our findings are confirmed by textit{ab-initio} calculations and angle-resolved photoemission experiments, which are in excellent agreement with each other and the topological analysis. This is the first complete characterization of topological surface states in the family of square-net based nodal line semimetals and thus fundamentally increases the understanding of the topological nature of this growing class of topological semimetals.