We study modular transformation of holomorphic Yukawa couplings in magnetized D-brane models. It is found that their products are modular forms, which are non-trivial representations of finite modular subgroups, e.g. $S_3$, $S_4$, $Delta(96)$ and $Delta(384)$.