We reveal a class of three-dimensional $d$-orbital topological materials in the antifluorite Cu$_2$S family. Derived from the unique properties of low-energy $t_{2g}$ states, their phases are solely determined by the sign of spin-orbit coupling (SOC): topological insulator for negative SOC, whereas topological semimetal for positive SOC; both having Dirac-cone surface states but with contrasting helicities. With broken inversion symmetry, the semimetal becomes one with a nodal box consisting of butterfly-shaped nodal lines that are robust against SOC. Further breaking the tetrahedral symmetry by strain leads to an ideal Weyl semimetal with four pairs of Weyl points. Interestingly, the Fermi arcs coexist with a surface Dirac cone on the (010) surface, as required by a $Z_2$-invariant.