We show how observations of gravitational waves from binary neutron star (BNS) mergers over the next few years can be combined with insights from nuclear physics to obtain useful constraints on the equation of state (EoS) of dense matter, in particular, constraining the neutron-matter EoS to within 20% between one and two times the nuclear saturation density $n_0approx 0.16 {text{fm}^{-3}}$. Using Fisher information methods, we combine observational constraints from simulated BNS merger events drawn from various population models with independent measurements of the neutron star radii expected from x-ray astronomy (the Neutron Star Interior Composition Explorer (NICER) observations in particular) to directly constrain nuclear physics parameters. To parameterize the nuclear EoS, we use a different approach, expanding from pure nuclear matter rather than from symmetric nuclear matter to make use of recent quantum Monte Carlo (QMC) calculations. This method eschews the need to invoke the so-called parabolic approximation to extrapolate from symmetric nuclear matter, allowing us to directly constrain the neutron-matter EoS. Using a principal component analysis, we identify the combination of parameters most tightly constrained by observational data. We discuss sensitivity to various effects such as different component masses through population-model sensitivity, phase transitions in the core EoS, and large deviations from the central parameter values.