We reveal that the phenomenon of full transmission without phase accumulation commonly associated with epsilon-near-zero (ENZ) materials for a plane-wave does not require vanishing of permittivity. We theoretically connect the phenomenon with condition of the Brewster effect satisfied at the edges of stop bands (so called exceptional points of degeneracy) and show that the full transmission without phase accumulation can be observed in various one-dimensional periodic structures. Particularly, exploiting the manifold of exceptional points of degeneracy in one-dimensional all-dielectric periodic lattices, we demonstrate that these structures not only offer a lossless and extremely simple, CMOS compatible alternative for some applications of ENZ media, but exhibit new properties of all-angle full transmission with zero phase delay.