Programmable Deterministic Zero-Copy DMA Mechanism for FPGA Accelerator
Source abstract
Abstract
With the expansion of network scales, the B/S architecture of monolithic applications is gradually being replaced by microservices. The unbundling of services has led to exponential growth in the size of APIs. When handling massive microservice requests, the commercial NIC shows limitations in three aspects: deterministic, programmability, and data copy. To ensure that each microservice node handles requests efficiently, flexibly, and precisely, this paper proposes a programmable deterministic multi-queue FPGA Accelerator. The Accelerator relies on the instantiated 1000 queues and the queue management unit to extend the rule-based RSS algorithm for the serverless-friendly programmability of packet distribution. A PTP hardware clock is added to collaborate with the queue management unit to control the deterministic delivery. To improve the sending and receiving efficiency of network node data, a driver adapted to the FPGA accelerator is designed to realize zero-copy. Experiments conducted on a 100 Gbps FPGA show that the Accelerator can support the multi-queue transmission with various packet sizes, define the forwarding behavior, and almost approach the line rate on an 8-core FPGA device. In addition, it can forward packets with low latency close to that of the current state-of-the-art ovs-DPDK. This Accelerator overcomes, to some extent, the limitations of commercial NICs when oriented to microservice architectures.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Software-Defined Networks and 5G · Network Packet Processing and Optimization · Caching and Content Delivery
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Computer College, National University of Defense Technology, Changsha 410073, China
Funding
National Key Research and Development Program of China
References (38)
Show source-provided references
1. García-Dorado, J.L., Mata, F., Ramos, J., del Río Santiago, P.M., Moreno, V., and Aracil, J. (2013). High-performance network traffic processing systems using commodity hardware. Data Traffic Monitoring and Analysis, Springer.10.1007/978-3-642-36784-7_1 ↗
2. Liao, G., Znu, X., and Bnuyan, L. (2011, January 12–16). A new server I/O architecture for high speed networks. Proceedings of the 2011 IEEE 17th International Symposium on High Performance Computer Architecture, San Antonio, TX, USA.10.1109/HPCA.2011.5749734 ↗
3. Kim, J., Soh, Y.J., Izraelevitz, J., Zhao, J., and Swanson, S. (2020). SubZero: Zero-copy IO for persistent main memory file systems. APSys, 1–8. Available online: https://juno-kim.github.io/papers/apsys2020-kim.pdf.10.1145/3409963.3410489 ↗
4. MacMichael, D. (2017). Introduction to Receive Side Scaling, Microsoft Hardware Dev Center.
5. Iyengar, J., and Thomson, M. (2021, May 27). QUIC: A UDP-Based Multiplexed and Secure Transport. RFC 9000. May 2021. Available online: https://www.tzi.de/~cabo/rfc9000.pdf.10.17487/RFC9000 ↗
6. Thomson, M., and Turner, S. (2021, May 27). Using TLS to Secure QUIC. RFC9001. May 2021. Available online: https://www.rfc-editor.org/info/rfc9001.10.17487/RFC9001 ↗
7. Iyengar, J., and Swett, I. (2021, May 27). Internet Engineering Task Force. QUIC Loss Detection and Congestion Control. RFC9002. Available online: https://www.hjp.at/doc/rfc/rfc9002.html.10.17487/RFC9002 ↗
8. Oliveira, M.V.M., Sampaio, A., and Cavalcanti, A. (2016). Local livelock analysis of component-based models. International Conference on Formal Engineering Methods, Springer.
9. Performance review of zero copy techniques
10. Kim, J.H., and Na, J.C. (2017, January 19–22). A study on one-way communication using PF_RING ZC. Proceedings of the 2017 19th International Conference on Advanced Communication Technology (ICACT), PyeongChang, Korea.10.23919/ICACT.2017.7890102 ↗
11. Rizzo, L. (, 2012). netmap: A novel framework for fast packet I/O. Proceedings of the 21st USENIX Security Symposium (USENIX Security 12), Boston, MA, USA. Available online: https://www.usenix.org/system/files/conference/atc12/atc12-final186.pdf.
12. Tu, W., Wei, Y.H., Antichi, G., and Pfaff, B. (2021, January 23–27). Revisiting the open vswitch dataplane ten years later. Proceedings of the 2021 ACM SIGCOMM Conference, Online.10.1145/3452296.3472914 ↗
13. (2020). Data Plane Development Kit (DPDK): A Software Optimization Guide to the User Space-Based Network Applications, CRC Press.
14. Gallenmüller, S., Emmerich, P., Wohlfart, F., Raumer, D., and Carle, G. (2015, January 7–8). Comparison of frameworks for high-performance packet IO. Proceedings of the 2015 ACM/IEEE Symposium on Architectures for Networking and Communications Systems (ANCS), Oakland, CA, USA.10.1109/ANCS.2015.7110118 ↗
15. (2014, April 14). Ntop Website Introducing PF_RING ZC (Zero Copy). Available online: http://www.ntop.org/pf_ring/introducing-pf_ring-zc-zero-copy/.
16. Deri, L., Cardigliano, A., and Fusco, F. (2013, January 14–19). 10 Gbit line rate packet-to-disk using n2disk. Proceedings of the 2013 IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), Turin, Italy.10.1109/INFCOMW.2013.6562903 ↗
17. Deri, L. (2011, April 09). How to Monitor Latency Using nProbe. Available online: https://www.ntop.org/nprobe/how-to-monitor-latency-using-nprobenagios-world-conference-europe/.
18. Rizzo, L., Carbone, M., and Catalli, G. (2012, January 25–30). Transparent acceleration of software packet forwarding using netmap. Proceedings of the 2012 Proceedings IEEE INFOCOM, Orlando, FL, USA.10.1109/INFCOM.2012.6195638 ↗
19. Kim, J., Huh, S., Jang, K., Park, K., and Moon, S. (2012, January 23–24). The power of batching in the click modular router. Proceedings of the Asia-Pacific Workshop on Systems, Seoul, Korea.10.1145/2349896.2349910 ↗
20. Casoni, M., Grazia, C.A., and Patriciello, N. (2013, January 11–13). On the performance of linux container with netmap/vale for networks virtualization. Proceedings of the 2013 19th IEEE International Conference on Networks (ICON), Singapore.10.1109/ICON.2013.6781957 ↗
21. Rizzo, L., Lettieri, G., and Maffione, V. (2016, January 13–15). Very high speed link emulation with TLEM. Proceedings of the 2016 IEEE International Symposium on Local and Metropolitan Area Networks (LANMAN), Rome, Italy.10.1109/LANMAN.2016.7548841 ↗
22. Brouer, J.D., and Høiland-Jørgensen, T. (2018, January 4–7). XDP: Challenges and future work. Proceedings of the Linux Plumbers Conference, Heraklion, Greece.
23. Brouer, J.D., and Gospodarek, A. (2018, November 03). A practical introduction to XDP. In Proceedings of the Linux Plumbers Conference. Available online: https://people.netfilter.org/hawk/presentations/LinuxPlumbers2018/presentation-lpc2018-xdp-tutorial.pdf.
24. Emmerich, P., Ellmann, S., Bonk, F., Egger, A., Sánchez-Torija, E.G., Günzel, T., Di Luzio, S., Obada, A., Stadlmeier, M., and Voit, S. (2019, January 24–25). User space network drivers. Proceedings of the 2019 ACM/IEEE Symposium on Architectures for Networking and Communications Systems (ANCS), Cambridge, UK.
25. Pfaff, B., Pettit, J., Koponen, T., Jackson, E., Zhou, A., Rajahalme, J., Gross, J., Wang, A., Stringer, J., and Shelar, P. (2015, January 4–6). The Design and Implementation of Open vSwitch. Proceedings of the 12th USENIX Symposium on Networked Systems Design and Implementation (NSDI 15), Oakland, VA, USA.
26. Suné, M., Köpsel, A., Alvarez, V., and Jungel, T. (2013). xDPd: eXtensible DataPath Daemon, EWSDN.
27. Pal, S.P., Ian, K.K.S., and Ray, K.C. (2014, January 24–27). FPGA implementation of stream cipher using Toeplitz Hash function. Proceedings of the 2014 International Conference on Advances in Computing, Communications and Informatics (ICACCI), Delhi, India.10.1109/ICACCI.2014.6968290 ↗
28. Forencich, A., Snoeren, A.C., Porter, G., and Papen, G. (2020, January 3–6). Corundum: An open-source 100-gbps nic. Proceedings of the 2020 IEEE 28th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM), Fayetteville, AR, USA.10.1109/FCCM48280.2020.00015 ↗
29. Chavan, A., Nagurvalli, S., Jain, M., and Chaudhari, S. (2018). Implementation of fpga-based network synchronization using IEEE 1588 precision time protocol (ptp). Recent Findings in Intelligent Computing Techniques, Springer.10.1007/978-981-10-8636-6_15 ↗
30. Yan, Y., and Wang, H. (2016, January 10–11). Open vSwitch Vxlan performance acceleration in cloud computing data center. Proceedings of the 5th International Conference on Computer Science and Network Technology (ICCSNT), Changchun, China.10.1109/ICCSNT.2016.8070222 ↗
31. Shanmugalingam, S., Ksentini, A., and Bertin, P. (2016, January 16–18). DPDK Open vSwitch performance validation with mirroring feature. Proceedings of the 2016 23rd International Conference on Telecommunications (ICT), Thessaloniki, Greece.10.1109/ICT.2016.7500387 ↗
32. Lin, J., Patel, K., Stephens, B.E., Sivaraman, A., and Akella, A. (2020, January 4–6). PANIC: A High-Performance Programmable NIC for Multi-tenant Networks. Proceedings of the 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI 20), Banff, Alberta, Canada.
33. (2019, August 19). RDMA and RoCE for Ethernet Network Efficiency Performance. Available online: http://www.mellanox.com/page/products_dyn?product_family=79&mtag=roce.
34. (2019, August 01). TCP Offload Engine (TOE). Available online: https://www.chelsio.com/nic/tcp-offload-engine/.
35. Arashloo, M.T., Ghobadi, M., Rexford, J., and Walker, D. (December, January 30). Hotcocoa: Hardware congestion control abstractions. Proceedings of the 16th ACM Workshop on Hot Topics in Networks, Palo Alto, CA, USA.
36. Firestone, D., Putnam, A., Mundkur, S., Chiou, D., Dabagh, A., Andrewartha, M., Angepat, H., Bhanu, V., Caulfield, A., and Chung, E. (2018, January 9–11). Azure Accelerated Networking: SmartNICs in the Public Cloud. Proceedings of the 15th USENIX Symposium on Networked Systems Design and Implementation (NSDI 18), Renton, WA, USA.
37. Lavasani, M., Dennison, L., and Chiou, D. (2012, January 22–24). Compiling high throughput network processors. Proceedings of the ACM/SIGDA International Symposium on Field Programmable Gate Arrays, Monterey, CA, USA.10.1145/2145694.2145709 ↗
38. Li, B., Tan, K., Luo, L., Peng, Y., Luo, R., Xu, N., Xiong, Y., Cheng, P., and Chen, E. (2016, January 22–26). Clicknp: Highly flexible and high performance network processing with reconfigurable hardware. Proceedings of the 2016 ACM SIGCOMM Conference, Florianopolis, Brazil.10.1145/2934872.2934897 ↗