Ontology Embeddings with Pretrained Language Models and Schema Information for Ontology-Related Tasks

In the Semantic Web, an ontology is a framework that organizes concepts in a hierarchical structure while adding relationships, rules, and axioms between them. Contrary to relational databases, ontologies offer a more flexible and expressive way to represent knowledge, and unlike the rigidity of relational databases, which define tables and relations strictly, ontologies allow for the representation of complex, interconnected concepts with the ability to infer new knowledge based on the defined relationships and rules. Many real-world applications benefit from the sophistication of this framework. From biomedicine to biodiversity, nutrition, and material science, the expressiveness of ontologies is an essential tool for representing and describing the world around us. Emerging from the vast developments of ontologies across multiple domains and applications, numerous ontology-related tasks have seen the light. Amongst them are, ontology integration, ontology completion, subsumption prediction, and the most notorious, ontology alignment, which is the task of identifying correspondences between semantically related elements, such as classes, properties, and instances, across different ontologies. This thesis explores novel ways of mapping ontologies to a latent space through robust, scalable, and generalized representation learning. Following the recent advances in language models, we focus on how ontology embeddings can leverage them by building more accurate vector representations. We then utilize our novelties to address the foremost issues of representation learning in ontology-related tasks by developing state-of-the-art systems. During this research, our work has been divided into three articles. First, SEBMatcher, is an ontology alignment system that relies on two BERT networks and a context-enhanced input to produce alignment. Second, SORBET is an ontology embedding model inspired by SEBMatcher that leverages a distance-based regression loss and a pre-trained SentenceBERT to produce high-quality ontology embeddings. Third, SORBETMatcher is a simple schema matching and subsumption prediction system whose primary objective was to showcase the potential of SORBET embeddings in ontology-related tasks.