We investigate fundamental processes that govern dynamics of vortex nucleation in sub-100 nm mesoscopic magnets. We focus on a structure with broken symmetry - Pacman-like nanomagnet shape - in which we study micromagnetic behavior both by means of a simple model and numerically. We show that it is possible to establish desired vortex chirality and polarity by applying only quasi-static in-plane magnetic field along specific directions. We identify the modes of vortex nucleation that are very robust against external magnetic field noise. These vortex nucleation modes are common among wide range of sub-100 nm magnets with broken rotational symmetry.