Magnetoelectric coupling in the polycrystalline antiferromagnets CuFe0.95Rh0.05O2 and CuFeO2 has been investigated. For both samples, electric polarization was observed in the absence of an applied external magnetic field demonstrating that for multiferroic research ceramics are worth to be studied. The observed magnetodielectric effect for CuFe0.95Rh0.05O2 in the electrically polar phase supports the existence of a noncollinear antiferromagnetic state. Interestingly, the electric polarization of this sample can be suppressed by a magnetic field. The temperature dependence of the relative magnitude of the magnetodielectric effect shows a discontinuity, clearly indicating different mechanisms of the magnetodielectric couplings in polar and paraelectric antiferromagnetic states.