Publications

2023

1. Descriptive Comparison of Visual Ontology Change Summarisation Methods

   Chung, Kornpol, Pernisch, Romana, and Schlobach, Stefan

   In Proceedings of the 20th Extended Semantic Web Conference: Posters and Demos, Jun, 2023

   [URL]

2022

1. Visualising the effects of ontology changes and studying their understanding with ChImp

   Pernisch, Romana, Dell’Aglio, Daniele, Serbak, Mirko, Gonçalves, Rafael S., and Bernstein, Abraham

   Journal of Web Semantics, vol. 74, pp. 100715, Oct, 2022

   [Abs] [URL] [DOI]

   Due to the Semantic Web’s decentralised nature, ontology engineers rarely know all applications that leverage their ontology. Consequently, they are unaware of the full extent of possible consequences that changes might cause to the ontology. Our goal is to lessen the gap between ontology engineers and users by investigating ontology engineers’ understanding of ontology changes’ impact at editing time. Hence, this paper introduces the Protégé plugin ChImp which we use to reach our goal. We elicited requirements for ChImp through a questionnaire with ontology engineers. We then developed ChImp according to these requirements and it displays all changes of a given session and provides selected information on said changes and their effects. For each change, it computes a number of metrics on both the ontology and its materialisation. It displays those metrics on both the originally loaded ontology at the beginning of the editing session and the current state to help ontology engineers understand the impact of their changes. We investigated the informativeness of materialisation impact measures, the meaning of severe impact, and also the usefulness of ChImp in an online user study with 36 ontology engineers. We asked the participants to solve two ontology engineering tasks – with and without ChImp (assigned in random order) – and answer in-depth questions about the applied changes as well as the materialisation impact measures. We found that ChImp increased the participants’ understanding of change effects and that they felt better informed. Answers also suggest that the proposed measures were useful and informative. We also learned that the participants consider different outcomes of changes severe, but most would define severity based on the amount of changes to the materialisation compared to its size. The participants also acknowledged the importance of quantifying the impact of changes and that the study will affect their approach of editing ontologies.

2021

1. VideoGraph – Towards Using Knowledge Graphs for Interactive Video Retrieval

   Rossetto, Luca, Baumgartner, Matthias, Ashena, Narges, Ruosch, Florian, Pernisch, Romana, Heitz, Lucien, and Bernstein, Abraham

   In MultiMedia Modeling, pp. 417–422, 2021

   [Abs] [URL] [DOI]

   Video is a very expressive medium, able to capture a wide variety of information in different ways. While there have been many advances in the recent past, which enable the annotation of semantic concepts as well as individual objects within video, their larger context has so far not extensively been used for the purpose of retrieval. In this paper, we introduce the first iteration of VideoGraph, a knowledge graph-based video retrieval system. VideoGraph combines information extracted from multiple video modalities with external knowledge bases to produce a semantically enriched representation of the content in a video collection, which can then be retrieved using graph traversal. For the 2021 Video Browser Showdown, we show the first proof-of-concept of such a graph-based video retrieval approach.

2. Multi-Domain and Explainable Prediction of Changes in Web Vocabularies

   Meroño-Peñuela, Albert, Pernisch, Romana, Guéret, Christophe, and Schlobach, Stefan

   In Proceedings of the 11th on Knowledge Capture Conference, pp. 193–200, 2021

   [Abs] [URL] [DOI]

   Web vocabularies (WV) have become a fundamental tool for structuring Web data: over 10 million sites use structured data formats and ontologies to markup content. Maintaining these vocabularies and keeping up with their changes are manual tasks with very limited automated support, impacting both publishers and users. Existing work shows that machine learning can be used to reliably predict vocabulary changes, but on specific domains (e.g. biomedicine) and with limited explanations on the impact of changes (e.g. their type, frequency, etc.). In this paper, we describe a framework that uses various supervised learning models to learn and predict changes in versioned vocabularies, independent of their domain. Using well-established results in ontology evolution we extract domain-agnostic and human-interpretable features and explain their influence on change predictability. Applying our method on 139 WV from 9 different domains, we find that ontology structural and instance data, the number of versions, and the release frequency highly correlate with predictability of change. These results can pave the way towards integrating predictive models into knowledge engineering practices and methods.

3. Beware of the Hierarchy — An Analysis of Ontology Evolution and the Materialisation Impact for Biomedical Ontologies

   Pernisch, Romana, Dell’Aglio, Daniele, and Bernstein, Abraham

   Journal of Web Semantics, 2021

   [Abs] [URL] [DOI]

   Ontologies are becoming a key component of numerous applications and research fields. But knowledge captured within ontologies is not static. Some ontology updates potentially have a wide ranging impact; others only affect very localised parts of the ontology and their applications. Investigating the impact of the evolution gives us insight into the editing behaviour but also signals ontology engineers and users how the ontology evolution is affecting other applications. However, such research is in its infancy. Hence, we need to investigate the evolution itself and its impact on the simplest of applications: the materialisation. In this work, we define impact measures that capture the effect of changes on the materialisation. In the future, the impact measures introduced in this work can be used to investigate how aware the ontology editors are about consequences of changes. By introducing five different measures, which focus either on the change in the materialisation with respect to the size or on the number of changes applied, we are able to quantify the consequences of ontology changes. To see these measures in action, we investigate the evolution and its impact on materialisation for nine open biomedical ontologies, most of which adhere to the EL++ description logic. Our results show that these ontologies evolve at varying paces but no statistically significant difference between the ontologies with respect to their evolution could be identified. We identify three types of ontologies based on the types of complex changes which are applied to them throughout their evolution. The impact on the materialisation is the same for the investigated ontologies, bringing us to the conclusion that the effect of changes on the materialisation can be generalised to other similar ontologies. Further, we found that the materialised concept inclusion axioms experience most of the impact induced by changes to the class inheritance of the ontology and other changes only marginally touch the materialisation.

4. Toward Measuring the Resemblance of Embedding Models for Evolving Ontologies

   Pernisch, Romana, Dell’Aglio, Daniele, and Bernstein, Abraham

   In Proceedings of the 11th on Knowledge Capture Conference, pp. 177–184, Dec, 2021

   [Abs] [URL] [DOI]

   Updates on ontologies affect the operations built on top of them. But not all changes are equal: some updates drastically change the result of operations; others lead to minor variations, if any. Hence, estimating the impact of a change ex-ante is highly important, as it might make ontology engineers aware of the consequences of their action during editing. However, in order to estimate the impact of changes, we need to understand how to measure them. To address this gap for embeddings, we propose a new measure called Embedding Resemblance Indicator (ERI), which takes into account both the stochasticity of learning embeddings as well as the shortcomings of established comparison methods. We base ERI on (i) a similarity score, (ii) a robustness factor \hatμ (based on the embedding method, similarity measure, and dataset), and (iii) the number of added or deleted entities to the embedding computed with the Jaccard index. To evaluate ERI, we investigate its usage in the context of two biomedical ontologies and three embedding methods—GraRep, LINE, and DeepWalk—as well as the two standard benchmark datasets—FB15k-237 and Wordnet-18-RR—with TransE and RESCAL embeddings. To study different aspects of ERI, we introduce synthetic changes in the knowledge graphs, generating two test-cases with five versions each and compare their impact with the expected behaviour. Our studies suggests that ERI behaves as expected and captures the similarity of embeddings based on the severity of changes. ERI is crucial for enabling further studies into impact of changes on embeddings.

2020

1. ChImp: Visualizing Ontology Changes and their Impact in Protégé

   Pernisch, Romana, Serbak, Mirko, Dell’ Aglio, Daniele, and Bernstein, Abraham

   In Visualization and interaction for ontologies and linked data, co-located with ISWC2020, vol. 2778, pp. 47–60, 2020

   [Abs] [URL]

   Today, ontologies are an established part of many applications and research. However, ontologies evolve over time, and ontology editors—engineers and domain experts—need to be aware of the con-sequences of changes while editing. Ontology editors might not be fully aware of how they are influencing consistency, quality, or the structure of the ontology, possibly causing applications to fail. To support editors and in crease their sensitivity towards the consequences of their actions, we conducted a user survey to elicit preferences for representing changes, eg, with ontology metrics such as number of classes and properties. Based on the survey, we developed ChImp—a Protégé plug-in to display information about the impact of changes in real-time. During editing of the ontology, ChImp lists the applied changes, checks and displays the consistency status, and reports measures describing the effect on the structure of the ontology. Akin to software IDEs and integrated testing approaches, we hope that displaying such metrics will help to improve ontology evolution processes in the long run.

2. LifeGraph: A Knowledge Graph for Lifelogs

   Rossetto, Luca, Baumgartner, Matthias, Ashena, Narges, Ruosch, Florian, Pernisch, Romana, and Bernstein, Abraham

   In LSC@ICMR, pp. 13–17, 2020

   [Abs] [URL] [DOI]

   The data produced by efforts such as life logging is commonly multi modal and can have manifold interrelations with itself as well as external information. Representing this data in such a way that these rich relations as well as all the different sources can be leveraged is a non-trivial undertaking. In this paper, we present the first iteration of LifeGraph, a Knowledge Graph for lifelogging data. LifeGraph aims at not only capturing all aspects of the data contained in a lifelog but also linking them to external, static knowledge bases in order to put the log as a whole as well as its individual entries into a broader context. In the Lifelog Search Challenge 2020, we show a first proof-of-concept implementation of LifeGraph as well as a retrieval system prototype which utilizes it to search the log for specific events.

2019

1. Toward Predicting Impact of Changes in Evolving Knowledge Graphs

   Pernischová, Romana, Dell’Aglio, Daniele, Horridge, Matthew, Baumgartner, Matthias, and Bernstein, Abraham

   In Proceedings of the ISWC 2019 Satellite Tracks (Posters & Demonstrations, Industry, and Outrageous Ideas) co-located with 18th International Semantic Web Conference (ISWC 2019), Auckland, New Zealand, October 26-30, 2019, vol. 2456, pp. 137–140, 2019

   [URL]
2. The Butterfly Effect in Knowledge Graphs: Predicting the Impact of Changes in the Evolving Web of Data

   Pernischová, Romana

   In Doctoral Consortium, Oct, 2019

2018

1. Stream Processing: the Matrix Revolutions

   Pernischová, Romana, Ruosch, Florian, Dell’Aglio, Daniele, and Bernstein, Abraham

   In Proceedings of the 12th International Workshop on Scalable Semantic Web Knowledge Base Systems co-located with 17th International Semantic Web Conference, SSWS@ISWC 2018, Monterey, California, USA, October 9, 2018, vol. 2179, pp. 15–27, 2018

   [URL]

2017

1. Abstracting .torrent Content Consumption into Two-mode Graphs and their Projection to Content Networks (ConNet)

   Lareida, Andri, Pernischová, Romana, Rodrigues, Bruno Bastos, and Stiller, Burkhard

   In 2017 IFIP/IEEE Symposium on Integrated Network and Service Management (IM), Lisbon, Portugal, May 8-12, 2017, pp. 151–159, 2017

   [Abs] [URL] [DOI]

   Video-on-demand and live streaming services are about to take over video discs. Video streaming services typically cannot compete with the content available in Peer-to-Peer (P2P) file sharing networks. Thus, content providers can use P2P systems to identify content to include in their offer. This work defines a novel method to apply Social Network Analysis (SNA) on video streaming or download traces. Those traces are abstracted int a two-mode graph, which is projected to a content-centric one mode graph (ConNet). SNA measures are used on a ConNet to classify a content-centric graph and provide a general interpretation and insights into the system the traces were collected from. To evaluate the proposed method, real world traces acquired from BitTorrent (BT) swarms sharing movies and television (TV) shows are used to construct 48 hourly graphs to show the evolution of the graph. The results show that the video network can be classified as scale-free, that SNA measures can be used as an alternative popularity indicator, and that the network evolves over time and exhibits diurnal patterns. Finally, this work shows that the proposed method can be applied to real world traces and provides a novel perspective on video consumption.