HERMES (High Energy Rapid Modular Ensemble of Satellites) is an innovative mission aiming to observe transient high-energy events such as gamma-ray bursts (GRBs) through a constellation of CubeSats hosting a broadband X and gamma-ray detector. The detector is based on a solid-state Silicon Drift Detector (SDD) coupled to a scintillator crystal, and is sensitive in the 2 keV to 2 MeV band. An accurate evaluation of the foreseen in-orbit instrumental background is essential to assess the scientific performance of the experiment. An outline of the Monte Carlo simulations of the HERMES payload will be provided, describing the various contributions on the total background and the optimization strategies followed in the instrument design. Moreover, the simulations were used in order to derive the effective area and response matrices of the instrument, also as a function of the source location with respect to the detector frame of reference.