Open Source Summit Europe 2024

Within AI workloads, a few key traffic flows drive significant data movement between GPUs across nodes. Optimizing these flows for efficient bandwidth, low latency, and minimal jitter is critical to prevent GPU underutilization. Additionally, in the context of AI Cloud infrastructure, accommodating numerous users and concurrent AI workloads introduces competition for shared network resources, potentially impacting application performance. Hence, ensuring isolation between workloads within and across tenants is paramount. This session will explain ways to achieve network isolation (overlay virtual network topology) and efficient bandwidth (end-to-end QoS) between AI workloads using Open Source SDN solution, namely, Open vSwitch (OVS), Open Virtual Network (OVN), and OVN-Kubernetes CNI. With OVS-offloadable hardware the gains are much more significant.