RiQ-KGC: Relation Instantiation Enhanced Quaternionic Attention for Complex-Relation Knowledge Graph Completion
Source abstract
Abstract
A knowledge graph is a structured semantic network designed to describe physical entities and relations in the world. A comprehensive and accurate knowledge graph is essential for tasks such as knowledge inference and recommendation systems, making link prediction a popular problem for knowledge graph completion. However, existing approaches struggle to model complex relations among entities, which severely hampers their ability to complete knowledge graphs effectively. To address this challenge, we propose a novel hierarchical multi-head attention network embedding framework, called RiQ-KGC, which integrates different-grained contextual information of knowledge graph triples and models quaternion rotation relations between entities. Furthermore, we propose a relation instantiation method for alleviating the difficulty of expressing complex relations between entities. To enhance the expressiveness of relation representation, the relation is integrated by Transformer to obtain multi-hop neighbor information, so that one relation can be embedded into different embeddings according to different entities. Experimental results on four datasets demonstrate that RiQ-KGC exhibits strong competitiveness compared to state-of-the-art models in link prediction, while the ablation experiments reveal that the proposed relation instantiation method achieves great performance.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Graph Neural Networks · Topic Modeling · Machine Learning in Healthcare
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Information Science and Technology, Dalian Maritime University, Dalian 116026, China
Funding
National Natural Science Foundation of China
References (51)
Show source-provided references
1. Hayashi, H., Hu, Z., Xiong, C., and Neubig, G. (2020, January 7–12). Latent Relation Language Models. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.
2. Verga, P., Sun, H., Soares, L.B., and Cohen, W.W. (2021, January 6–11). Adaptable and Interpretable Neural MemoryOver Symbolic Knowledge. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Online.10.18653/v1/2021.naacl-main.288 ↗
3. A survey on deep learning based knowledge tracing10.1016/j.knosys.2022.110036 ↗
4. Boosting question answering over knowledge graph with reward integration and policy evaluation under weak supervision10.1016/j.ipm.2022.103242 ↗
5. A graph embedding based model for fine-grained POI recommendation10.1016/j.neucom.2020.01.118 ↗
6. Incorporating Multi-Source Urban Data for Personalized and Context-Aware Multi-Modal Transportation Recommendation10.1109/TKDE.2020.2985954 ↗
7. Wang, X., Cui, P., Wang, J., Pei, J., Zhu, W., and Yang, S. (2017, January 4–9). Community Preserving Network Embedding. Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, San Francisco, CA, USA.10.1609/aaai.v31i1.10488 ↗
8. Nie, F., Zhu, W., and Li, X. (2017, January 4–9). Unsupervised Large Graph Embedding. Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, San Francisco, CA, USA.10.1609/aaai.v31i1.10814 ↗
9. Persistent Graph Stream Summarization For Real-time Graph Analytics
10. FLAG: Towards Graph Query Autocompletion for Large Graphs10.1007/s41019-022-00182-8 ↗
11. KGVQL: A knowledge graph visual query language with bidirectional transformations10.1016/j.knosys.2022.108870 ↗
12. A Survey of Arabic Named Entity Recognition and Classification10.1162/COLI_a_00178 ↗
13. Link Prediction on Complex Networks: An Experimental Survey10.1007/s41019-022-00188-2 ↗
14. WordNet: A Lexical Database for English
15. Wikidata: A free collaborative knowledgebase
16. Suchanek, F.M., Kasneci, G., and Weikum, G. (2007, January 8–12). Yago: A core of semantic knowledge. Proceedings of the 16th International Conference on World Wide Web, Banff, AB, Canada.10.1145/1242572.1242667 ↗
17. Chen, Z., Wang, X., Wang, C., and Li, J. (2022, January 17–21). Explainable Link Prediction in Knowledge Hypergraphs. Proceedings of the 31st ACM International Conference on Information and Knowledge Management, Atlanta, GA, USA.10.1145/3511808.3557316 ↗
18. HKGB: An Inclusive, Extensible, Intelligent, Semi-auto-constructed Knowledge Graph Framework for Healthcare with Clinicians’ Expertise Incorporated10.1016/j.ipm.2020.102324 ↗
19. Bordes, A., Usunier, N., García-Durán, A., Weston, J., and Yakhnenko, O. (2013, January 5–10). Translating Embeddings for Modeling Multi-relational Data. Proceedings of the Annual Conference on Neural Information Processing Systems, Lake Tahoe, NV, USA.
20. Sun, Z., Deng, Z., Nie, J., and Tang, J. (2019, January 6–9). RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space. Proceedings of the International Conference on Learning Representations, New Orleans, LA, USA.
21. Kim, B., Hong, T., Ko, Y., and Seo, J. (2020, January 8–13). Multi-Task Learning for Knowledge Graph Completion with Pre-trained Language Models. Proceedings of the 28th International Conference on Computational Linguistics, Barcelona, Spain.10.18653/v1/2020.coling-main.153 ↗
22. Chen, S., Liu, X., Gao, J., Jiao, J., Zhang, R., and Ji, Y. (2021, January 7–11). HittER: Hierarchical Transformers for Knowledge Graph Embeddings. Proceedings of the Empirical Methods in Natural Language Processing, Punta Cana, Dominican Republic.10.18653/v1/2021.emnlp-main.812 ↗
23. Zhang, S., Tay, Y., Yao, L., and Liu, Q. (2019, January 8–14). Quaternion Knowledge Graph Embeddings. Proceedings of the Annual Conference on Neural Information Processing Systems, Vancouver, BC, Canada.
24. Wang, Z., Zhang, J., Feng, J., and Chen, Z. (2014, January 27–31). Knowledge Graph Embedding by Translating on Hyperplanes. Proceedings of the AAAI Conference on Artificial Intelligence, Québec City, QC, Canada.10.1609/aaai.v28i1.8870 ↗
25. Lin, Y., Liu, Z., Sun, M., Liu, Y., and Zhu, X. (2015, January 25–30). Learning Entity and Relation Embeddings for Knowledge Graph Completion. Proceedings of the AAAI Conference on Artificial Intelligence, Austin, TX, USA.10.1609/aaai.v29i1.9491 ↗
26. Ji, G., He, S., Xu, L., Liu, K., and Zhao, J. (2015, January 26–31). Knowledge Graph Embedding via Dynamic Mapping Matrix. Proceedings of the Association for Computational Linguistics, Beijing, China.10.3115/v1/P15-1067 ↗
27. Yang, B., Yih, W., He, X., Gao, J., and Deng, L. (2015, January 7–9). Embedding Entities and Relations for Learning and Inference in Knowledge Bases. Proceedings of the International Conference on Learning Representations, San Diego, CA, USA.
28. Trouillon, T., Welbl, J., Riedel, S., Gaussier, É., and Bouchard, G. (2016, January 19–24). Complex Embeddings for Simple Link Prediction. Proceedings of the 33rd International Conference on Machine Learning, New York, NY, USA.
29. Nickel, M., Tresp, V., and Kriegel, H. (July, January 28). A Three-Way Model for Collective Learning on Multi-Relational Data. Proceedings of the International Conference on Machine Learning, Bellevue, WA, USA.
30. Nickel, M., Rosasco, L., and Poggio, T.A. (2016, January 12–17). Holographic Embeddings of Knowledge Graphs. Proceedings of the AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA.10.1609/aaai.v30i1.10314 ↗
31. Ebisu, T., and Ichise, R. (2018, January 2–7). TorusE: Knowledge Graph Embedding on a Lie Group. Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.10.1609/aaai.v32i1.11538 ↗
32. Cao, Z., Xu, Q., Yang, Z., Cao, X., and Huang, Q. (2024, January 20–27). Dual Quaternion Knowledge Graph Embeddings. Proceedings of the AAAI Conference on Artificial Intelligence, Vancouver, BC, Canada.
33. Balazevic, I., Allen, C., and Hospedales, T.M. (2019, January 8–14). Multi-relational Poincaré Graph Embeddings. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada.
34. Chami, I., Wolf, A., Juan, D., Sala, F., Ravi, S., and Ré, C. (2020, January 5–10). Low-Dimensional Hyperbolic Knowledge Graph Embeddings. Proceedings of the Association for Computational Linguistics, Online.10.18653/v1/2020.acl-main.617 ↗
35. Extrapolation over temporal knowledge graph via hyperbolic embedding10.1049/cit2.12186 ↗
36. Dettmers, T., Minervini, P., Stenetorp, P., and Riedel, S. (2018, January 2–7). Convolutional 2D Knowledge Graph Embeddings. Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.10.1609/aaai.v32i1.11573 ↗
37. Balazevic, I., Allen, C., and Hospedales, T.M. (2019, January 17–19). Hypernetwork Knowledge Graph Embeddings. Proceedings of the 28th International Conference on Artificial Neural Networks, Munich, Germany.10.1007/978-3-030-30493-5_52 ↗
38. GNN-based long and short term preference modeling for next-location prediction10.1016/j.ins.2023.01.131 ↗
39. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017, January 4–9). Attention is All you Need. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA.
40. Devlin, J., Chang, M., Lee, K., and Toutanova, K. (2019, January 2–7). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Proceedings of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, MN, USA.
41. Wang, B., Shen, T., Long, G., Zhou, T., Wang, Y., and Chang, Y. (2021, January 19–23). Structure-Augmented Text Representation Learning for Efficient Knowledge Graph Completion. Proceedings of the WWW ’21: The Web Conference, Ljubljana, Slovenia.10.1145/3442381.3450043 ↗
42. Lv, X., Lin, Y., Cao, Y., Hou, L., Li, J., Liu, Z., Li, P., and Zhou, J. (2022, January 22–27). Do Pre-trained Models Benefit Knowledge Graph Completion?. A Reliable Evaluation and a Reasonable Approach. In Proceedings of the Association for Computational Linguistics, Dublin, Ireland.10.18653/v1/2022.findings-acl.282 ↗
43. Wang, L., Zhao, W., Wei, Z., and Liu, J. (2022, January 22–27). SimKGC: Simple Contrastive Knowledge Graph Completion with Pre-trained Language Models. Proceedings of the Association for Computational Linguistics, Dublin, Ireland.10.18653/v1/2022.acl-long.295 ↗
44. Youn, J., and Tagkopoulos, I. (2023, January 13–14). KGLM: Integrating Knowledge Graph Structure in Language Models for Link Prediction. Proceedings of the 12th Joint Conference on Lexical and Computational Semantics, Toronto, ON, Canada.10.18653/v1/2023.starsem-1.20 ↗
45. Lacroix, T., Usunier, N., and Obozinski, G. (2018, January 10–15). Canonical Tensor Decomposition for Knowledge Base Completion. Proceedings of the 35th International Conference on Machine Learning, Stockholm, Sweden.
46. Toutanova, K., and Chen, D. (2015, January 31). Observed versus latent features for knowledge base and text inference. Proceedings of the Continuous Vector Space Models and their Compositionality, Beijing, China.10.18653/v1/W15-4007 ↗
47. Bollacker, K.D., Evans, C., Paritosh, P.K., Sturge, T., and Taylor, J. (2008, January 10–12). Freebase: A collaboratively created graph database for structuring human knowledge. Proceedings of the International Conference on Management of Data, Vancouver, BC, Canada.10.1145/1376616.1376746 ↗
48. Safavi, T., and Koutra, D. (2020, January 16–20). CoDEx: A Comprehensive Knowledge Graph Completion Benchmark. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, Online.10.18653/v1/2020.emnlp-main.669 ↗
49. Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., and Antiga, L. (2019, January 8–14). PyTorch: An Imperative Style, High-Performance Deep Learning Library. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada.
50. Broscheit, S., Ruffinelli, D., Kochsiek, A., Betz, P., and Gemulla, R. (2020, January 16–20). LibKGE—A knowledge graph embedding library for reproducible research. Proceedings of the Empirical Methods in Natural Language Processing, Online.10.18653/v1/2020.emnlp-demos.22 ↗
51. Kingma, D.P., and Ba, J. (2015, January 7–9). Adam: A Method for Stochastic Optimization. Proceedings of the International Conference on Learning Representations, San Diego, CA, USA.