Sparse Subgraph Prediction Based on Adaptive Attention
Source abstract
Abstract
Link prediction is a crucial problem in the analysis of graph-structured data, and graph neural networks (GNNs) have proven to be effective in addressing this problem. However, the computational and temporal costs associated with large-scale graphs remain a concern. This study introduces a novel method for link prediction called Sparse Subgraph Prediction Based on Adaptive Attention (SSP-AA). The method generates sparse subgraphs and utilizes Graph SAmple and aggreGatE (GraphSAGE) for prediction, aiming to reduce computation and time costs while providing a foundation for future exploration of large-scale graphs. Certain key issues in GraphSAGE are addressed by integrating an adaptive attention mechanism and a jumping knowledge module into the model. To address the issue of adaptive weight distribution in GraphSAGE, an aggregation function is employed, which is based on the attention mechanism. This modification enables the model to distribute weights adaptively among neighboring nodes, significantly improving its ability to capture node relationships. Furthermore, to tackle the common issue of over-smoothing in GNNs, a jumping knowledge module is integrated, enabling information sharing across different layers and providing the model with the flexibility to select the appropriate representation depth based on the specific situation. By enhancing the quality of node representations, SSP-AA further boosts the performance of GraphSAGE in various prediction tasks involving graph-structured data.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Graph Neural Networks · Complex Network Analysis Techniques · Bioinformatics and Genomic Networks
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Computer Science and Engineering, North Minzu University, Yinchuan 750021, China
Funding
Ningxia Natural Science Foundation · National Natural Science Foundation of China · Key Research Project of Northern University for Nationalities
References (50)
Show source-provided references
1. Open graph benchmark: Datasets for machine learning on graphs
2. Zhang, C., Song, D., Huang, C., Swami, A., and Chawla, N.V. (2019, January 4–8). Heterogeneous graph neural network. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Anchorage, AK, USA.10.1145/3292500.3330961 ↗
3. You, Y., Chen, T., Shen, Y., and Wang, Z. (2021, January 8–24). Graph contrastive learning automated. Proceedings of the International Conference on Machine Learning PMLR, Virtual Event.
4. Neo-gnns: Neighborhood overlap-aware graph neural networks for link prediction
5. Ying, R., He, R., Chen, K., Eksombatchai, P., Hamilton, W.L., and Leskovec, J. (2018, January 19–23). Graph convolutional neural networks for web-scale recommender systems. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, London, UK.10.1145/3219819.3219890 ↗
6. Wu, Z., Pan, S., Long, G., Jiang, J., and Zhang, C. (2019). Graph wavenet for deep spatial-temporal graph modeling. arXiv.10.24963/ijcai.2019/264 ↗
7. Kong, L., Chen, Y., and Zhang, M. (2022). Geodesic Graph Neural Network for Efficient Graph Representation Learning. arXiv.
8. Hamilton, W.L. (2020). Graph Representation Learning, Morgan & Claypool Publishers. Synthesis Lectures on Artifical Intelligence and Machine Learning.10.1007/978-3-031-01588-5 ↗
9. Wu, F., Souza, A., Zhang, T., Fifty, C., Yu, T., and Weinberger, K. (2019, January 9–15). Simplifying graph convolutional networks. Proceedings of the International Conference on Machine Learning PMLR, Long Beach, CA, USA.
10. Sun, F.Y., Hoffmann, J., Verma, V., and Tang, J. (2019). Infograph: Unsupervised and semi-supervised graph-level representation learning via mutual information maximization. arXiv.
11. Hamilton, W., Ying, Z., and Leskovec, J. (2017). Advances in Neural Information Processing Systems, The MIT Press.
12. Hamilton, W.L., Ying, R., and Leskovec, J. (2017). Representation learning on graphs: Methods and applications. arXiv.
13. Measuring and relieving the over-smoothing problem for graph neural networks from the topological view
14. Teji, B., and Roy, S. (2022, January 14–16). Missing Link Identification from Node Embeddings using Graph Auto Encoders and its Variants. Proceedings of the 2022 OITS International Conference on Information Technology (OCIT), Bhubaneswar, India.10.1109/OCIT56763.2022.00025 ↗
15. Seo, Y., Defferrard, M., VanderGheynst, P., and Bresson, X. (2018, January 13–16). Structured sequence modeling with graph convolutional recurrent networks. Proceedings of the Neural Information Processing 25th International Conference, ICONIP 2018, Siem Reap, Cambodia.10.1007/978-3-030-04167-0_33 ↗
16. Kipf, T.N., and Welling, M. (2016). Variational graph auto-encoders. arXiv.
17. Wavelet generalized regression neural network approach for robust field strength prediction10.1007/s11277-020-07550-5 ↗
18. Alfke, D., and Stoll, M. (2019). Semi-supervised classification on non-sparse graphs using low-rank graph convolutional networks. arXiv.
19. Boutorh, A., Marref, K., and Dehiri, N.E. (2022). Advances in Computing Systems and Applications, Proceedings of the 5th Conference on Computing Systems and Applications, San Diego, CA, USA, 7–10 April 2022, Springer International Publishing.
20. Lo, W.W., Layeghy, S., Sarhan, M., Gallagher, M., and Portmann, M. (2022, January 25–29). E-graphsage: A graph neural network based intrusion detection system for iot. Proceedings of the NOMS 2022–2022 IEEE/IFIP Network Operations and Management Symposium, Budapest, Hungary.10.1109/NOMS54207.2022.9789878 ↗
21. E-minBatch GraphSAGE: An Industrial Internet Attack Detection Model10.1155/2022/5363764 ↗
22. Providing a new method for link prediction in social networks based on the meta-heuristic algorithm10.33545/27075907.2020.v1.i1a.6 ↗
23. The Application of Particle Swarm Optimization in Neural Networks10.1088/1742-6596/2278/1/012027 ↗
24. Wang, P., Wu, C., Huang, T., and Chen, Y. (2022). A Supervised Link Prediction Method Using Optimized Vertex Collocation Profile. Entropy, 24.10.3390/e24101465 ↗
25. Xie, Z., Zhu, R., Zhang, M., and Liu, J. (2022, January 17–20). SparseMult: A Tensor Decomposition model based on Sparse Relation Matrix. Proceedings of the 2022 IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT), Niagara Falls, ON, Canada.10.1109/WI-IAT55865.2022.00124 ↗
26. A modified DeepWalk method for link prediction in attributed social network10.1007/s00607-021-00982-2 ↗
27. Arsov, N., and Mirceva, G. (2019). Network embedding: An overview. arXiv.
28. Intelligent link prediction management based on community discovery and user behavior preference in online social networks10.1155/2021/3860083 ↗
29. Meilicke, C., Chekol, M.W., Ruffinelli, D., and Stuckenschmidt, H. (2019, January 23–26). An introduction to AnyBURL. Proceedings of the KI 2019: Advances in Artificial Intelligence, 42nd German Conference on AI, Kassel, Germany.10.1007/978-3-030-30179-8_20 ↗
30. Ott, S., Meilicke, C., and Samwald, M. (2021). SAFRAN: An interpretable, rule-based link prediction method outperforming embedding models. arXiv.
31. Heterogeneous combat network link prediction based on representation learning10.1109/JSYST.2020.3028168 ↗
32. Integrating node centralities, similarity measures, and machine learning classifiers for link prediction10.1007/s11042-022-12854-8 ↗
33. Floating pollutant image target extraction algorithm based on immune extremum region10.1016/j.dsp.2022.103442 ↗
34. Deep neural network regression for automated retinal layer segmentation in optical coherence tomography images10.1109/TIP.2019.2931461 ↗
35. Veličković, P., Fedus, W., Hamilton, W.L., Liò, P., Bengio, Y., and Hjelm, R.D. (2018). Deep graph infomax. arXiv.
36. A comprehensive survey on graph neural networks10.1109/TNNLS.2020.2978386 ↗
37. Xu, X., Zhang, P., He, Y., Chao, C., and Yan, C. (2022). Subgraph neighboring relations infomax for inductive link prediction on knowledge graphs. arXiv.10.24963/ijcai.2022/325 ↗
38. Wang, D., Cui, P., and Zhu, W. (2016, January 13–17). Structural deep network embedding. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.10.1145/2939672.2939753 ↗
39. Biased random walk with restart for link prediction with graph embedding method10.1016/j.physa.2021.125783 ↗
40. Kim, D., and Oh, A. (2022). How to find your friendly neighborhood: Graph attention design with self-supervision. arXiv.
41. Xu, K., Li, C., Tian, Y., Sonobe, T., Kawarabayashi, K.I., and Jegelka, S. (2018, January 14–16). Representation learning on graphs with jumping knowledge networks. Proceedings of the International Conference on Machine Learning, Beijing, China.
42. Graph representation learning via simple jumping knowledge networks10.1007/s10489-021-02889-z ↗
43. Improving the accuracy, scalability, and performance of graph neural networks with roc
44. Emergence of scaling in random networks10.1126/science.286.5439.509 ↗
45. Friends and neighbors on the web10.1016/S0378-8733(03)00009-1 ↗
46. Kipf, T.N., and Welling, M. (2016). Semi-supervised classification with graph convolutional networks. arXiv.
47. Xu, K., Hu, W., Leskovec, J., and Jegelka, S. (2018). How powerful are graph neural networks?. arXiv.
48. Symeonidis, P., and Zioupos, A. (2016). Matrix and Tensor Factorization Techniques for Recommender Systems, Springer International Publishing.10.1007/978-3-319-41357-0 ↗
49. Grover, A., and Leskovec, J. (2016, January 13–17). node2vec: Scalable feature learning for networks. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.10.1145/2939672.2939754 ↗
50. Louis, P., Jacob, S.A., and Salehi-Abari, A. (2022, January 17–21). Sampling Enclosing Subgraphs for Link Prediction. Proceedings of the 31st ACM International Conference on Information & Knowledge Management, Atlanta, GA, USA.10.1145/3511808.3557688 ↗