A method is developed that allows analysis of quantum Monte Carlo simulations to identify errors in trial wave functions. The purpose of this method is to allow for the systematic improvement of variational wave functions by identifying degrees of freedom that are not well-described by an initial trial state. We provide proof of concept implementations of this method by identifying the need for a Jastrow correlation factor, and implementing a selected multi-determinant wave function algorithm for small dimers that systematically decreases the variational energy. Selection of the two-particle excitations is done using quantum Monte Carlo within the presence of a Jastrow correlation factor, and without the need to explicitly construct the determinants. We also show how this technique can be used to design compact wave functions for transition metal systems. This method may provide a route to analyze and systematically improve descriptions of complex quantum systems in a scalable way.