Non-abelian discrete symmetries are of particular importance in model building. They are mainly invoked to explain the various fermion mass hierarchies and forbid dangerous superpotential terms. In string models they are usually associated to the geometry of the compactification manifold and more particularly to the magnetised branes in toroidal compactifications. Motivated by these facts, in this note we propose a unified framework to construct representations of finite discrete family groups based on the automorphisms of the discrete and finite Heisenberg group. We focus in particular in the $PSL_2(p)$ groups which contain the phenomenologically interesting cases.