Recent studies of early-type galaxies have suggested that the initial mass function (IMF) slope is bottom-heavy, i.e. they contain a larger fraction of low-mass stars than the Milky Way. However, measurements of the IMF remain challenging in unresolved galaxies because features in their observed spectra are sensitive to a number of factors including the stellar age, metallicity, and elemental abundances, in addition to the IMF. In this paper, we use new high signal-to-noise IMACS (Magellan) spectra to study the elliptical shell galaxy NGC 3923 at optical (3700-6600 Angstrom), and near-infrared (7900-8500 Angstrom) wavelengths, as a function of radius. We have undertaken a number of independent approaches to better understand the uncertainties in our results. 1) We compare two different stellar population model libraries; 2) we undertake spectral index fitting as well as full spectral fitting; 3) we have performed simulations for which we a priori know the input IMF, and which closely match our data; 4) we also investigate the effects of including a two-component, rather than a single stellar population. We show that our results are sensitive to the assumptions we make and to the methods we use. In addition, we evaluate the accuracy and precision of our results based on simulated mock data. We find some indication (although assumption-dependent) for a bottom-heavy IMF in the mass-range 0.5-1.0 M_sun, while the IMF in the mass-range 0.08-0.5 M_sun appears to be Milky-Way like and constant. Including near-infrared data to our analysis gives consistent results, and improves the precision.