We construct nuclear matter based on an extended parity doublet model including four light nucleons $N(939)$, $N(1440)$, $N(1535)$, and $N(1650)$. We exclude some values of the chiral invariant masses by requiring the saturation properties of normal nuclear matter; saturation density, binding energy, incompressibility, and symmetry energy. We find further constraint to the chiral invariant masses from the tidal deformability determined by the observation of the gravitational waves from neutron star merger GW170817. Our result shows that the chiral invariant masses are larger than about $600,$MeV. We also give some predictions on the symmetry energy and the slope parameters in the high density region, which will be measured in future experiments.