Dynamic excitatory-inhibitory (E-I) balance is a paradigmatic mechanism invoked to explain the irregular low firing activity observed in the cortex. However, we will show that the E-I balance can be at the origin of other regimes observable in the brain. The analysis is performed by combining simulations of sparse E-I networks composed of N spiking neurons with analytical investigations of low dimensional neural mass models. The bifurcation diagrams, derived for the neural mass model, allow to classify the asynchronous and coherent behaviours emerging any finite in-degree K. In the limit N >> K >> 1 both supra and sub-threshold balanced asynchronous regimes can be observed. Due to structural heterogeneity the asynchronous states are characterized by the splitting of the neurons in three groups: silent, fluctuation and mean driven. The coherent rhythms are characterized by regular or irregular temporal fluctuations joined to spatial coherence similar to coherent fluctuations observed in the cortex over multiple spatial scales. Collective Oscillations (COs) can emerge due to two different mechanisms. A first mechanism similar to the pyramidal-interneuron gamma (PING) one. The second mechanism is intimately related to the presence of current fluctuations, which sustain COs characterized by an essentially simultaneous bursting of the two populations. We observe period-doubling cascades involving the PING-like COs finally leading to the appearance of coherent chaos. For sufficiently strong current fluctuations we report a novel mechanism of frequency locking among collective rhythms promoted by these intrinsic fluctuations. Our analysis suggest that despite PING-like or fluctuation driven COS are observable for any finite in-degree K, in the limit N >> K >> 1 these solutions result in two coexisting balanced regimes: an asynchronous and a fully synchronized one.