We obtain R-band photometry for galaxies in six nearby poor groups for which we have spectroscopic data, including 328 new galaxy velocities. For the five groups with luminous X-ray halos, the composite group galaxy luminosity function (GLF) is fit adequately by a Schechter function with Mstar = -21.6 +/- 0.4 + 5log h and alpha = -1.3 +/- 0.1. We also find that (1) the ratio of dwarfs to giants is significantly larger for the five groups with luminous X-ray halos than for the one marginally X-ray detected group, (2) the composite GLF for the luminous X-ray groups is consistent in shape with that for rich clusters, (3) the composite group GLF rises more steeply at the faint end than that of the field, (4) the shape difference between the field and composite group GLFs results mostly from the population of non-emission line galaxies, whose dwarf-to-giant ratio is larger in the denser group environment than in the field, and (5) the non-emission line dwarfs are more concentrated about the group center than the non-emission line giants. This last result indicates that the dwarfs and giants occupy different orbits (i.e., have not mixed completely) and suggests that the populations formed at a different times. Our results show that the shape of the GLF varies with environment and that this variation is due primarily to an increase in the dwarf-to-giant ratio of quiescent galaxies in higher density regions, at least up to the densities characteristic of X-ray luminous poor groups. This behavior suggests that, in some environments, dwarfs are more biased than giants with respect to dark matter. This trend conflicts with the prediction of standard biased galaxy formation models. (Abridged)