In this note, we prove a tight lower bound on the joint entropy of $n$ unbiased Bernoulli random variables which are $n/2$-wise independent. For general $k$-wise independence, we give new lower bounds by adapting Navon and Samorodnitskys Fourier proof of the `LP bound on error correcting codes. This counts as partial progress on a problem asked by Gavinsky and Pudlak.