Wormholes are hypothetical tunnels that connect remote parts of spacetime. In General Relativity, wormholes are threaded by exotic matter that violates the energy conditions. In this work, we consider wormholes threaded by nonexotic matter in nonminimal torsion-matter coupling $f(T)$ gravity. We find that the nonminimal torsion-matter coupling can indeed hold the wormhole open. However, from geometric point of view, for the wormhole to have asymptotic flatness, the coupling matter density must falloff rapidly at large radius, otherwise the physical wormhole must be finite due to either change of metric signature or lack of valid embedding. On the other hand, the matter source supporting the wormhole can satisfy the null energy condition only in the neighborhood of the throat of the wormhole. Therefore, the wormhole in the underlying model has finite sizes and cannot stretch to the entire spacetime.