It is believed that fermions in adjoint representation on single site lattice will restore the center symmetry, which is a crucial requirement for the volume independence of large-N lattice gauge theories. We present a perturbative analysis which supports the assumption for overlap fermions, but shows that center symmetry is broken for naive fermions.