The recent quest for improved functional materials like high permittivity dielectrics and/or multiferroics has triggered an intense wave of research. Many materials have been checked for their dielectric permittivity or their polarization state. In this report, we call for caution when samples are simultaneously displaying insulating behavior and defect-related conductivity. Many oxides containing mixed valent cations or oxygen vacancies fall in this category. In such cases, most of standard experiments may result in effective high dielectric permittivity which cannot be related to ferroelectric polarization. Here we list few examples of possible discrepancies between measured parameters and their expected microscopic origin.