This paper proposes to improve our previous work on the concept-based video shot indexing, by considering an ontological concept construction in the TRECVid 2007 video retrieval, based on two steps. First, each single concept is modeled independently. Second, an ontology-based concept is introduced via the representation of the influence relations between concepts and the ontological readjustment of the confidence values. The main contribution of this paper is in the exploitation manner of the inter-concepts similarity in our indexing system, where three measures are represented: co-occurrence, visual similarity and LSCOM-lite ontology path length contribution. The experimental results report the efficiency and the significant improvement provided by the proposed scheme.