Strong electronic nematic fluctuations have been discovered near optimal doping for several families of Fe-based superconductors, motivating the search for a possible link between these fluctuations, nematic quantum criticality, and high temperature superconductivity. Here we probe a key prediction of quantum criticality, namely power law dependence of the associated nematic susceptibility as a function of composition and temperature approaching the compositionally-tuned putative quantum critical point. To probe the bare quantum critical point requires suppression of the superconducting state, which we achieve by using large magnetic fields, up to 45 T, while performing elastoresistivity measurements to follow the nematic susceptibility. We performed these measurements for the prototypical electron-doped pnictide, Ba(Fe$_{1-x}$Co$_x$)$_2$As$_2$, over a dense comb of dopings. We find that close to the putative quantum critical point, the nematic susceptibility appears to obey power law behavior over almost a decade of variation in composition, consistent with basic notions of nematic quantum criticality. Paradoxically, however, we also find that the temperature dependence for compositions close to the critical value cannot be described by a single power law. This is surprising as power law scaling in both doping and temperature is expected close to a quantum critical point.