Cloud datacenter networks are tasked with providing connectivity between an ever-increasing number of end hosts whose link rates improve by orders of magnitude every few years.  Unfortunately the speed of individual datacenter switches and routers has not kept pace with the increases in bandwidth at the end points, meaning that it isn't possible to directly upgrade the network as in the past.  This project sought to explore alternative network design paradigms that can improve the performance of future datacenter network fabrics.  Concretely, we considered the potential of topologies based on expander graphs that eliminate the capacity bottlenecks inherent in today's hierarchical Clos-based topologies.

Our efforts resulted in multiple major publications each containing significant research results.  Two of the major outcomes are described in two papers, one that appeared at CoNEXT 2022 and another at SIGCOMM 2022.

In our CoNEXT 2022 paper we observe that modern datacenter operators are deploying  parallel fabric designs in the core of the network that deliver 𝑁-times the bandwidth by simply forwarding traffic over any of 𝑁 parallel network fabrics. In our work, we considered extending this parallel network idea all the way to the end host, and leveraging the fact that there are now multiple networks to deploy heterogeneous parallel network fabrics that can deliver even higher bandwidth, lower latency, and improved resiliency than traditional designs constructed from the same constituent components.

Our initial studies found that direct application of existing path selection and forwarding techniques resulted in poor performance. Instead, we showed that appropriate path selection and forwarding protocols can not only improve the performance of existing, homogeneous parallel fabrics, but fully realize the potential of heterogenous fabrics.

Our SIGCOMM 2022 builds on the theme, but considers the challenges of an alternate approach where, rather than deploying multiple, heterogenous networks in parallel, an operator constructs a single network that is reconfigurable--i.e., the set of connects can vary over time.  Prior research has shown that providing multiple time-varying paths can improve network capacity and lower physical latency. However, our work shows that existing TCP variants are ill-suited to utilize available capacity because their congestion control cannot react quickly enough to drastic variations in bandwidth and latency. We developed Time-division TCP (TDTCP), a new TCP variant designed for reconfigurable data center networks. TDTCP recognizes that communication in these fabrics happens over a set of paths, each having its own physical characteristics and cross traffic. TDTCP multiplexes each connection across multiple independent congestion states--one for each distinct path--while managing connection-wide tasks in a shared fashion.  It leverages network support to receive timely notification of path changes and promptly matches its local view to the current path. We implementedTDTCP in the Linux kernel. Results on an emulated network show that TDTCP improves throughput over both traditional TCP variants, such as DCTCP and CUBIC, and multipath TCP by 24-41% without requiring significant in-network buffering to hide path variations.

Last Modified: 10/11/2023
Modified by: Alex C Snoeren