This paper introduces a system to recognize labels of time plans, where labels are extracted from time plan. This labels are images, so spatial segmentation is used to extract images of labels only. Size of images of labels are made same using median's algorithm for two purposes. The first one is to create database training for used neural networks. The second is to recognizing's processing. Two methods of recognizing are dependent on using neural networks technic: classification using perceptron network and recognizing using back propagation network. Perceptron network is built to take image as input and to give classification index as output for label. Then label is recognize dependent on stored table of ASCII for label. Back propagation network is designed to recognize images for all letters of English alphabet that are used in time plan. Results of research appear efficiency of designed system to recognize labels of time plan from their images for both methods after system had been applied on three time plans.