In this paper, based on directional modulation (DM), robust beamforming matrix design for sum secrecy rate maximization is investigated in multi-user systems. The base station (BS) is assumed to have the imperfect knowledge of the direction angle toward each eavesdropper, with the estimation error following the Von Mises distribution. To this end, a Von Mises distribution-Sum Secrecy Rate Maximization (VMD-SSRM) method is proposed to maximize the sum secrecy rate by employing semi-definite relaxation and first-order approximation based on Taylor expansion to solve the optimization problem. Then in order to optimize the sum secrecy rate in the case of the worst estimation error of direction angle toward each eavesdropper, we propose a maximum angle estimation error-SSRM (MAEE-SSRM) method. The optimization problem is constructed based on the upper and lower bounds of the estimated eavesdropping channel related coefficient and then solved by the change of the variable method. Simulation results show that our two proposed methods have better sum secrecy rate than zero-forcing (ZF) method and signal-to-leakage-and-noise ratio (SLNR) method. Furthermore, the sum secrecy rate performance of our VMD-SSRM method is better than that of our MAEE-SSRM method.