We present a detailed transmission electron microscopy and electron diffraction study of the thinnest possible membrane, a single layer of carbon atoms suspended in vacuum and attached only at its edges. Membranes consisting of two graphene layers are also reported. We find that the membranes exhibit an apparently random spontaneous curvature that is strongest in single-layer membranes. A direct visualization of the roughness is presented for two-layer membranes where we used the variation of diffracted intensities with the local orientation of the membrane.