pHost: Distributed Near-Optimal Datacenter Transport Over Commodity Network Fabric

The importance of minimizing flow completion times (FCT) in datacenters has led to a growing literature on new network transport designs. Of particular note is pFabric, a protocol that achieves near-optimal FCTs. However, pFabric’s performance comes at the cost of generality, since pFabric requires specialized hardware that embeds a specific scheduling policy within the network fabric, making it hard to meet diverse policy goals. Aiming for generality, the recent Fastpass proposal returns to a design based on commodity network hardware and instead relies on a centralized scheduler. Fastpass achieves generality, but (as we show) loses many of pFabric’s performance benefits.

We present pHost, a new transport design aimed at achieving both: the near-optimal performance of pFabric and the commodity network design of Fastpass. Similar to Fastpass, pHost keeps the network simple by decoupling the network fabric from scheduling decisions. However, pHost introduces a new distributed protocol  that allows end-hosts to directly make scheduling decisions, thus avoiding the overheads of Fastpass’s centralized scheduler architecture. We show that pHost achieves performance on par with pFabric (within 4% for typical conditions) and significantly outperforms Fastpass (by a factor of 3.8x) while relying only on commodity network hardware.

Authors: Peter X. Gao, Akshay Narayan, Gautam Kumar, Rachit Agarwal, Sylvia Ratnasamy, Scott Shenker
Publication Date: December 2015
Conference: International Conference on emerging Networking EXperiments and Technologies (CoNext)