We have carried out detailed magnetic and transport studies of the new Sr0.5Ce0.5FBiS2-xSex (x = 0.5, 1) superconductors derived by doping Se in Sr0.5Ce0.5FBiS2. Se-doping produces several effects: it suppresses semiconducting like behavior observed in the undoped Sr0.5Ce0.5FBiS2, ferromagnetic ordering temperature, TFM, decreases considerably from 7.5 K (in Sr0.5Ce0.5FBiS2) to 3.5 K and superconducting transition temperature, Tc, gets enhanced slightly to 2.9 - 3.3 K. Thus in these Se-doped materials, TFM is just marginally higher than Tc. Magnetization studies provide an evidence of bulk superconductivity in Sr0.5Ce0.5FBiS2-xSex. Quite remarkably, as compared with the effective paramagnetic Ce-moment (~ 2.2 muB), the ferromagnetically ordered Ce-moment in the superconducting state is rather small (~ 0.1 muB). To the best of our knowledge, the title compounds are the first Ce-based superconducting itinerant ferromagnetic materials (Tc < TFM). We stress that Ce-4f electrons are responsible for both superconductivity and ferromagnetism just as U-5f electrons are in UCoGe. Furthermore, a novel feature of these materials is a dual hysteresis loop corresponding to both the ferromagnetism and the coexisting superconductivity. Such features of Sr0.5Ce0.5FBiS2-xSex put these materials apart from the well known U-containing superconducting ferromagnets reported so far.