This paper studies the relative spatial distribution of red-sequence and blue-cloud galaxies, and their relation to the dark matter distribution in the COMBO-17 survey as function of scale down to z~1. We measure the 2nd-order auto- and cross-correlation functions of galaxy clustering and express the relative biasing by using aperture statistics. Also estimated is the relation between the galaxies and the dark matter distribution exploiting galaxy-galaxy lensing (GGL). All observables are further interpreted in terms of a halo model. To fully explain the galaxy clustering cross-correlation function with a halo model, we need to introduce a new parameter,R, that describes the statistical relation between numbers of red and blue galaxies within the same halo. We find that red and blue galaxies are clearly differently clustered, a significant evolution of the relative clustering with redshift was not found. There is evidence for a scale-dependence of relative biasing. The relative clustering, the GGL and, with some tension, the galaxy numbers can be explained consistently within a halo model. For the cross-correlation function one requires a HOD variance that becomes Poisson even for relatively small occupancy numbers. For our sample, this rules out with high confidence a Poisson satellite scenario as found in semi-analytical models. Red galaxies have to be concentrated towards the halo centre, either by a central red galaxy or by a concentration parameter above that for dark matter.The value of R depends on the presence or absence of central galaxies: If no central galaxies or only red central galaxies are allowed, R is consistent with zero, whereas a positive correlation $R=+0.5pm0.2$ is needed if both blue and red galaxies can have central galaxies.[ABRIDGED]