The knowledge of the structure of the magnetic field inside a blazar jet, as deduced from polarization observations at radio to optical wavelengths, is closely related to the formation and propagation of relativistic jets that result from accretion onto supermassive black holes. However, a largely unexplored aspect of the theoretical understanding of radiation transfer physics in blazar jets has been the magnetic field geometry as revealed by the polarized emission and the connection between the variability in polarization and flux across the spectrum. Here, we explore the effects of various magnetic geometries that can exist inside a blazar jet: parallel, oblique, toroidal, and tangled. We investigate the effects of changing the orientation of the magnetic field, according to the above-mentioned geometries, on the resulting high-energy spectral energy distributions (SEDs) and spectral variability patterns (SVPs) of a typical blazar. We use the MUlti-ZOne Radiation Feedback (MUZORF) model of Joshi et al. (2014) to carry out this study and to relate the geometry of the field to the observed SEDs at X-ray and gamma-ray energies. One of the goals of the study is to understand the relationship between synchrotron and inverse Compton peaks in blazar SEDs and the reason for the appearance of gamma-ray orphan flares observed in some blazars. This can be associated with the directionality of the magnetic field, which creates a difference in the radiation field as seen by an observer versus that seen by the electrons in the emission region.