A device-independent randomness expansion protocol aims to take an initial random string and generate a longer one, where the security of the protocol does not rely on knowing the inner workings of the devices used to run it. In order to do so, the protocol tests that the devices violate a Bell inequality and one then needs to bound the amount of extractable randomness in terms of the observed violation. The entropy accumulation theorem gives a bound in terms of the single-round von Neumann entropy of any strategy achieving the observed score. Tight bounds on this are known for the one-sided randomness when using the Clauser-Horne-Shimony-Holt (CHSH) game. Here we find the minimum von Neumann entropies for a given CHSH score relevant for one and two sided randomness that can be applied to various protocols. In particular, we show the gain that can be made by using the two-sided randomness and by using a protocol without spot-checking where the input randomness is recycled. We also discuss protocols that fully close the locality loophole while expanding randomness. Although our bounds are mostly numerical, we conjecture analytic formulae for the curves in two cases.