There is consensus in the current literature that stable states of asynchronous irregular spiking activity require (i) large networks of 10 000 or more neurons and (ii) external background activity or pacemaker neurons. Yet already in 1963, Griffith showed that networks of simple threshold elements can be persistently active at intermediate rates. Here, we extend Griffiths work and demonstrate that sparse networks of integrate-and-fire neurons assume stable states of self-sustained asynchronous and irregular firing without external input or pacemaker neurons. These states can be robustly induced by a brief pulse to a small fraction of the neurons, or by short a period of irregular input, and last for several minutes. Self-sustained activity states emerge when a small fraction of the synapses is strong enough to significantly influence the firing probability of a neuron, consistent with the recently proposed long-tailed distribution of synaptic weights. During self-sustained activity, each neuron exhibits highly irregular firing patterns, similar to experimentally observed activity. Moreover, the interspike interval distribution reveals that neurons switch between discrete states of high and low firing rates. We find that self-sustained activity states can exist even in small networks of only a thousand neurons. We investigated networks up to 100 000 neurons. Finally, we discuss the implications of self-sustained activity for learning, memory and signal propagation.