A two-fold quality assurance approach for dynamic knowledge bases: The 3cixty use case

The 3cixty platform relies on a continuous integration workflow for the generation and maintenance of evolving knowledge bases in the domain of culture and tourism. This approach is inspired by common practices in the software engineering industry in which continuous integration is widely-used for quality assurance purposes. The objective of this paper is to present a similar approach for knowledge base population and publishing. The proposed approach consists of two main steps: (i) exploratory testing, and (ii) fine-grained analysis. In the exploratory testing step, the knowledge base is tested for patterns that may reveal erroneous data or outliers that could indicate inaccuracies. This phase is knowledge-base agnostic and provides inputs for the second phase. In the finegrained analysis step, specific tests are developed for a particular knowledge base according to the data model and pre-defined constraints that shape the data. More precisely, a set of predefined queries are executed and their results are compared to the expected answers (similar to unit testing in software engineering) in order to automatically validate that the knowledge base fulfills a set of requirements.

The main objective of this approach is to detect and to flag potential defects as

early as possible in the data publishing process and to eliminate or minimize the

undesirable outcomes in the applications that depend on the knowledge base, typically,

user interfaces that enable to explore the data but rely on a particular shape of the data. This two-fold approach proves to be critical when the knowledge base is continuously evolving, not necessarily in a monotonic way, and when real-world applications highly depend on it such as the 3cixty multi-device application.