Recent observations from NASAs Kepler mission detected the first planets in circumbinary orbits. The question we try to answer is where these planets formed in the circumbinary disk and how far inside they migrated to reach their present location. We investigate the first and more delicate phase of planet formation when planetesimals accumulate to form planetary embryos. We use the hydrodynamical code FARGO to study the evolution of the disk and of a test population of planetesimals embedded in it. With this hybrid hydrodynamical--N--body code we can properly account for the gas drag force on the planetesimals and for the gravitational force of the disk on them. The numerical simulations show that the gravity of the eccentric disk on the planetesimal swarm excites their eccentricities to values much larger than those induced by the binary perturbations only within 10 AU from the stars. Moreover, the disk gravity prevents a full alignment of the planetesimal pericenters. Both these effects lead to large impact velocities, beyond the critical value for erosion. Planetesimals accumulation in circumbinary disks appears to be prevented close to the stellar pair by the gravitational perturbations of the circumbinary disk. The observed planets possibly formed in the outer regions of the disk and then migrated inside by tidal interaction with the disk.