Every Timestamp Counts: Accurate Tracking of Network Latencies Using Reconcilable Difference Aggregator
Issue No. 01 - Feb. (2018 vol. 26)
Yongquan Fu , Science and Technology Laboratory of Parallel and Distributed Processing, College of Computer Science, National University of Defense Technology, Changsha, China
Pere Barlet-Ros , Computer Architecture Department, Universitat Politècnica de Catalunya, Barcelona, Spain
Dongsheng Li , Science and Technology Laboratory of Parallel and Distributed Processing, College of Computer Science, National University of Defense Technology, Changsha, China
User-facing services deployed in data centers must respond quickly to user actions. The measurement of network latencies is of paramount importance. Recently, a new family of compact data structures has been proposed to estimate one-way latencies. In order to achieve scalability, these new methods rely on timestamp aggregation. Unfortunately, this approach suffers from serious accuracy problems in the presence of packet loss and reordering, given that a single lost or out-of-order packet may invalidate a huge number of aggregated samples. In this paper, we unify the problem to detect lost and reordered packets within the set reconciliation framework. Although the set reconciliation approach and the data structures for aggregating packet timestamps are previously known, the combination of these two principles is novel. We present a space-efficient synopsis called reconcilable difference aggregator (RDA). RDA maximizes the percentage of useful packets for latency measurement by mapping packets to multiple banks and repairing aggregated samples that have been damaged by lost and reordered packets. RDA simultaneously obtains the average and the standard deviation of the latency. We provide a formal guarantee of the performance and derive optimized parameters. We further design and implement a user-space passive latency measurement system that addresses practical issues of integrating RDA into the network stack. Our extensive evaluation shows that compared with existing methods, our approach improves the relative error of the average latency estimation in 10–15 orders of magnitude, and the relative error of the standard deviation in 0.5–6 orders of magnitude.
Receivers, Loss measurement, Standards, Data structures, Protocols, Synchronization
Y. Fu, P. Barlet-Ros and D. Li, "Every Timestamp Counts: Accurate Tracking of Network Latencies Using Reconcilable Difference Aggregator," in IEEE/ACM Transactions on Networking, vol. 26, no. 1, pp. 90-103, 2018.