In this paper and its sequel, we construct topologically invariant defects in two-dimensional classical lattice models and quantum spin chains. We show how defect lines commute with the transfer matrix/Hamiltonian when they obey the defect commutation relations, cousins of the Yang-Baxter equation. These relations and their solutions can be extended to allow defect lines to branch and fuse, again with properties depending only on topology. In this part I, we focus on the simplest example, the Ising model. We define lattice spin-flip and duality defects and their branching, and prove they are topological. One useful consequence is a simple implementation of Kramers-Wannier duality on the torus and higher genus surfaces by using the fusion of duality defects. We use these topological defects to do simple calculations that yield exact properties of the conformal field theory describing the continuum limit. For example, the shift in momentum quantization with duality-twisted boundary conditions yields the conformal spin 1/16 of the chiral spin field. Even more strikingly, we derive the modular transformation matrices explicitly and exactly.