We investigate the stochastic gravitational wave background produced by primordial black hole binaries during their early inspiral stage while accreting high-density radiation surrounding them in the early universe. We first show that the gravitational wave amplitude produced from a primordial black hole binary has correction terms because of the rapid rate of increase in masses of the primordial black holes. These correction terms arise due to non-vanishing first and second time derivatives of the masses and their contribution to the overall second time derivative of quadrupole moment tensor. We find that some of these correction terms are not only significant in comparison with the main term but even dominant over the main term for certain ranges of time in the early Universe. The significance of these correction terms is not only for the gravitational wave amplitude produced from an individual PBH-binary, but persists for the overall stochastic gravitational wave background produced from them. We show that the spectral density produced from such accreting primordial black hole binaries lie within the detectability range of some present and future gravitational wave detectors.