journal article·2024-09-25Open access

Energy-Efficient Joint Partitioning and Offloading for Delay-Sensitive CNN Inference in Edge Computing

Zhiyong Zha, Yifei Yang, Yongjun Xia, Zhaoyi Wang, Bin Luo, Kaihong Li, Chenkai Ye, Bo Xu, Kai Peng

Applied Sciences· Volume 14· Issue 19

Abstract

With the development of deep learning foundation model technology, the types of computing tasks have become more complex, and the computing resources and memory required for these tasks have also become more substantial. Since it has long been revealed that task offloading in cloud servers has many drawbacks, such as high communication delay and low security, task offloading is mostly carried out in the edge servers of the Internet of Things (IoT) network. However, edge servers in IoT networks are characterized by tight resource constraints and often the dynamic nature of data sources. Therefore, the question of how to perform task offloading of deep learning foundation model services on edge servers has become a new research topic. However, the existing task offloading methods either can not meet the requirements of massive CNN architecture or require a lot of communication overhead, leading to significant delays and energy consumption. In this paper, we propose a parallel partitioning method based on matrix convolution to partition foundation model inference tasks, which partitions large CNN inference tasks into subtasks that can be executed in parallel to meet the constraints of edge devices with limited hardware resources. Then, we model and mathematically express the problem of task offloading. In a multi-edge-server, multi-user, and multi-task edge-end system, we propose a task-offloading method that balances the tradeoff between delay and energy consumption. It adopts a greedy algorithm to optimize task-offloading decisions and terminal device transmission power to maximize the benefits of task offloading. Finally, extensive experiments verify the significant and extensive effectiveness of our algorithm.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: IoT and Edge/Fog Computing · Advanced Neural Network Applications · Advanced Memory and Neural Computing

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

State Grid Information Telecommunication Co., Ltd., Wuhan 430048, China

Hubei Key Laboratory of Smart Internet Technology, School of Electronic Information and Communications, Huazhong University of Science and Technology, Wuhan 430074, China

Hubei Huazhong Electric Power Technology Development Co., Ltd., Wuhan 430079, China

School of Electronic Information Science and Technology, Wuhan University, Wuhan 430072, China

Hubei ChuTianYun Co., Ltd., Wuhan 430076, China

Funding

Key Research and Development Program of Hubei Province · Wuhan Artificial Intelligence Innovation

References (29)

Show source-provided references
1. Internet of Things(IoT): A vision, architectural elements, and future directions10.1016/j.future.2013.01.010 ↗
2. Chien, S.Y., Chan, W.K., Tseng, Y.H., Lee, C., Somayazulu, V.S., and Chen, Y. (2015, January 19–22). Distributed computing in IoT: System-on-a-chip for smart cameras as an example. Proceedings of the 20th Asia and South Pacific Design Automation Conference (ASP-DAC), Chiba, Japan.10.1109/ASPDAC.2015.7058993 ↗
3. Edge computing: Vision and challenges10.1109/JIOT.2016.2579198 ↗
4. Lane, N.D., Bhattacharya, S., Georgiev, P., Forlivesi, C., and Kawsar, T.F. (2015, January 26–28). An early resource characterization of deep learning on wearables, smartphones and Internet-of-Things devices. Proceedings of the International Workshop Internet Things Towards Appl. (IoT-A), Seoul, Republic of Korea.10.1145/2820975.2820980 ↗
5. Kang, A.M., Hauswald, J., Gao, C., Rovinski, A., Mudge, T., Mars, J., and Tang, L. (2017, January 8–12). Neurosurgeon: Collaborative intelligence between the cloud and mobile edge. Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), Xi’an, China.10.1145/3037697.3037698 ↗
6. Teerapittayanon, S., McDanel, B., and Kung, H.-T. (2017, January 5–8). Distributed deep neural networks over the cloud, the edge and end devices. Proceedings of the International Conference on Distributed Computing Systems (ICDCS), Atlanta, GA, USA.10.1109/ICDCS.2017.226 ↗
7. Zhang, B., Mor, N., Kolb, J., Chan, D.S., Lutz, K., Allman, E., Wawrzynek, J., Lee, E., and Kubiatowicz, J. (2015, January 6–7). The cloud is not enough: Saving IoT from the cloud. Proceedings of the USENIX Workshop Hot Topics Cloud Computing (HotCloud), Santa Clara, CA, USA.
8. Ribeiro, F.M., and Kamienski, C.A. (November, January 26). Timely Anomalous Behavior Detection in Fog-IoT Systems using Unsupervised Federated Learning. Proceedings of the IEEE 8th World Forum on Internet of Things (WF-IoT), Yokohama, Japan.
9. Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012, January 3–6). ImageNet classification with deep convolutional neural networks. Proceedings of the Advances in Neural Information Processing Systems (NIPS), Lake Tahoe, NV, USA.
10. He, K., Zhang, X., Ren, S., and Sun, J. (July, January 26). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.
11. Samie, F., Tsoutsouras, V., Xydis, S., Bauer, L., Soudris, D., and Henkel, J. (2016, January 2–7). Distributed QoS management for Internet of Things under resource constraints. Proceedings of the 2016 International Conference on Hardware/Software Codesign and System Synthesis (CODES+ISSS), Pittsburgh, PA, USA.10.1145/2968456.2974005 ↗
12. Service Deployment of C4ISR Based on Genetic Simulated Annealing Algorithm10.1109/ACCESS.2020.2981624 ↗
13. On Credibility-Based Service Function Chain Deployment10.1109/OJCS.2021.3064887 ↗
14. A Trade-Off Task-Offloading Scheme in Multi-User Multi-Task Mobile Edge Computing10.1109/ACCESS.2022.3228403 ↗
15. An Online Simulated Annealing-Based Task Offloading Strategy for a Mobile Edge Architecture10.1109/ACCESS.2024.3402611 ↗
16. Liu, C.S.W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7–12). Going deeper with convolutions. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.
17. DeepThings: Distributed adaptive deep learning inference on resource-constrained IoT edge clusters10.1109/TCAD.2018.2858384 ↗
18. Li, H., Hu, C., Jiang, J., Wang, Z., Wen, Y., and Zhu, W. (2018, January 11–13). JALAD: Joint Accuracy- and Latency-Aware Deep Structure Decoupling for Edge-Cloud Execution. Proceedings of the IEEE 24th International Conference on Parallel and Distributed Systems (ICPADS), Singapore.10.1109/PADSW.2018.8645013 ↗
19. Edge AI: On-Demand Accelerating Deep Neural Network Inference via Edge Computing10.1109/TWC.2019.2946140 ↗
20. Hu, C., and Li, B. (2022, January 2–5). Distributed Inference with Deep Learning Models across Heterogeneous Edge Devices. Proceedings of the IEEE Infocom, London, UK.10.1109/INFOCOM48880.2022.9796896 ↗
21. Mao, J., Chen, X., Nixon, K.W., Krieger, C., and Chen, Y. (2017, January 1–4). MoDNN: Local distributed mobile computing system for deep neural network. Proceedings of the IEEE Infocom, Atlanta, GA, USA.10.23919/DATE.2017.7927211 ↗
22. DeepSlicing: Collaborative and Adaptive CNN Inference With Low Latency10.1109/TPDS.2021.3058532 ↗
23. Liu, J., Mao, Y., and Zhang, J. (2016, January 10–15). Delay-optimal computation task scheduling for mobile-edge computing systems. Proceedings of the IEEE International Symposium on Information Theory, Barcelona, Spain.10.1109/ISIT.2016.7541539 ↗
24. Dynamic Computation Offloading for Mobile-Edge Computing With Energy Harvesting Devices10.1109/JSAC.2016.2611964 ↗
25. MAGA: A Mobility-Aware computation Decision for Distributed Mobile Cloud Computing10.1109/JIOT.2017.2776252 ↗
26. Chen, M.H., Liang, B., and Dong, M. (July, January 28). A semidefinite relaxation approach to mobile cloud offloading with computing access point. Proceedings of the IEEE International Workshop on Signal Processing Advances in Wireless Communications, Stockholm, Sweden.
27. Optimal Schedule of Mobile Edge Computing for Internet of Things using Partial Information10.1109/JSAC.2017.2760186 ↗
28. Edge server placement in mobile edge computing10.1016/j.jpdc.2018.06.008 ↗
29. Simonyan, K., and Zisserman, A. (2015, January 7–9). Very deep convolutional net works for large-scale image recognition. Proceedings of the 3rd International Conference on Learning Representations, San Diego, CA, USA.