Very light particles with CP-violating couplings to ordinary matter, such as axions or axion-like particles, can mediate long-range forces between polarized and unpolarized fermions. We describe a new experimental search for such forces between unpolarized nucleons in two 250 kg Pb weights and polarized neutrons and electrons in a $^3$He-K co-magnetometer located about 15 cm away. We place improved constrains on the products of scalar and pseudoscalar coupling constants, $g^n_p g^N_s < 4.2times10^{-30}$ and $g^e_p g^N_s < 1.7times10^{-30}$ (95% CL) for axion-like particle masses less than $10^{-6}$ eV, which represents an order of magnitude improvement over the best previous neutron laboratory limit.