YBaCuFeO5 is one of the interesting multiferroic compounds, which exhibits magnetic ordering and dielectric anomaly above 200 K. Partial substitution of Fe with other magnetic and non-magnetic ion affects the magnetic and the structural properties of the system. We report detailed investigation of structural, magnetic and dielectric properties of YBaCuFe0.85M0.15O5 (M=Co, Ni and Ga). We observed that the partial replacement of Ni and Co in place of Fe, results in magnetic dilution and broadening of the magnetic transition and shifting towards lower temperature. The replacement of Fe with non-magnetic Ga also results in shifting of the magnetic transition to the lower temperature side. The observed dielectric relaxation behavior in these compounds is due to the charge carrier hoping. This study highlights the impacts of magnetic and non-magnetic doping at the magnetic site on magnetic and dielectric properties in layered perovskite compound YBaCuFeO5.