Iron telluride (FeTe), a relative of the iron based high temperature superconductors, displays unusual magnetic order and structural transitions. Here we explore the idea that strong correlations may play an important role in these materials. We argue that the unusual orders observed in FeTe can be understood from a picture of correlated local moments with orbital degeneracy, coupled to a small density of itinerant electrons. A component of the structural transition is attributed to orbital, rather than magnetic ordering, introducing a strongly anisotropic character to the system along the diagonal directions of the iron lattice. Double exchange interactions couple the diagonal chains leading to the observed ordering wavevector. The incommensurate order in samples with excess iron arises from electron doping in this scenario. The strong anisotropy of physical properties in the ordered phase should be detectable by transport in single domains. Predictions for ARPES, inelastic neutron scattering and hole/electron doping studies are also made.