We introduce particle systems in one or more dimensions in which particles perform branching Brownian motion and the population size is kept constant equal to $N > 1$, through the following selection mechanism: at all times only the $N$ fittest particles survive, while all the other particles are removed. Fitness is measured with respect to some given score function $s:R^d to R$. For some choices of the function $s$, it is proved that the cloud of particles travels at positive speed in some possibly random direction. In the case where $s$ is linear, we show under some assumptions on the initial configuration that the shape of the cloud scales like $log N$ in the direction parallel to motion but at least $c(log N)^{3/2}$ in the orthogonal direction for some $c > 0$. We conjecture that the exponent 3/2 is sharp. This result is equivalent to the following result of independent interest: in one-dimensional systems, the genealogical time is greater than $c(log N)^3$, thereby contributing a step towards the original predictions of Brunet and Derrida. We discuss several open problems and also explain how our results can be viewed as a rigorous justification of Weismanns arguments for the role of recombination in population genetics.