We propose to realize one-dimensional topological phases protected by SU($N$) symmetry using alkali or alkaline-earth atoms loaded into a bichromatic optical lattice. We derive a realistic model for this system and investigate it theoretically. Depending on the parity of $N$, two different classes of symmetry-protected topological (SPT) phases are stabilized at half-filling for physical parameters of the model. For even $N$, the celebrated spin-1 Haldane phase and its generalization to SU($N$) are obtained with no local symmetry breaking. In stark contrast, at least for $N=3$, a new class of SPT phases, dubbed chiral Haldane phases, that spontaneously break inversion symmetry, emerge with a two-fold ground-state degeneracy. The latter ground states with open-boundary conditions are characterized by different left and right boundary spins which are related by conjugation. Our results show that topological phases are within close reach of the latest experiments on cold fermions in optical lattices.