Behavioural economists have shown that people are often averse to inequality and will make choices to avoid unequal outcomes. In this paper, we consider how to allocate indivisible goods fairly so as to minimize inequality. We consider how this interacts with axiomatic properties such as envy-freeness, Pareto efficiency and strategy-proofness. We also consider the computational complexity of computing allocations minimizing inequality. Unfortunately, this is computationally intractable in general so we consider several tractable greedy online mechanisms that minimize inequality. Finally, we run experiments to explore the performance of these methods.