People live in various environments, although they can understand scenes around them with just a glance. To do this, they depend on their high ability to effectively process visual data and connect it to wide pre-knowledge about what they are expected to see. This is not the case for computers, which can’t reach high levels of scene understanding until now. Most researches treat scene understanding as a usual classification problem, where they have just to classify scenes in predefined limited categories (forest, city, garden). They normally used classification or machine learning algorithms, which limit their ability to understand scenes and reduces their chances to be used in a practical way because of a required training phase of these algorithms. Some researches try to make use of knowledge in Ontologies to reach a high level scene understanding, but these researches are still limited to specific domains only. In this thesis we are trying to understand scene images without any pre-knowledge about their domain. We will not treat this problem as a normal classification problem; however we will extract high level concepts from scene images. These concepts will not only represent objects in the scene, but they will also reflect the places and events in the scene. To do this, we develop a novel algorithm named SMHITS. It depends on a semantically rich common sense knowledge base to extract associated concepts with a primitive group of concepts. To use SMHITS in scene understanding, we also develop a system named ICES. Instead of using a classification or machine learning algorithm, ICES depends on a large dataset of images that is independent of any scene domain. Results show the superiority of SMHITS comparing to current ConceptNet associated concepts extraction algorithm, as it has higher precision and can take advantage of expansion of its knowledge base. Results also show that ICES output concepts are semantically rich.