We investigate the capacity, convexity and characterization of a general family of norm-constrained feed-forward networks.