Throughput and Satellite Delay

Researcher:   Hank Nussbacher, Independent Networking Consultant, working part-time for the Israeli InterUniversity Computation Center (IUCC)

Technical background – A recent RFC details what is needed to achieve fast throughput on a satellite link (RFC2488: Enhancing TCP Over Satellite Channels using Standard Mechanisms). This document details what needs to be done at each system, such as Path MTU (RFC1191), large TCP windows (RFC1323), large socket buffers, TCP Selective Ack (SACK – RFC2018) and many other tricks. Without the changes listed, a single TCP stream via satellite is limited to 936 kb/sec. NLANR's Network Engineering Services at the Pittsburgh Supercomputer Center (PSC) has a site that lists all the details necessary for a system administrator to create an enhanced TCP stack (http://www.psc.edu/networking/perf_tune.html).

The limitation of this method is that each and every system has to be upgraded and often there are thousands, if not tens of thousands, of systems that need to be "tweaked." In addition, we have found that researchers do not want to modify or enhance their TCP stacks, and even when they agree to do so, they cannot succeed in convincing their partners in Internet-2 to modify their systems as well.

In April 1999, we tested two systems that can act as a black box and spoof the session with regard to TCP window size. The results of that testing can be found at: http://www.internet-2.org.il/satellite-testing.html.

 

We then purchased a set of SkyX XH45 boxes from Mentat (http://www.mentat.com); one to be installed in Chicago, and one in Tel-Aviv. The SkyX boxes work only when data traffic is via symmetric paths. The following is a list of some of the tricks SkyX uses to speed up throughput: They use nacks to indicate timeouts. They have an unlimited window size. There is no slow start over the satellite link. They use a streamlined handshake on the initial connection. They use a TCP rate control over the satellite link. Interestingly, NASA has also done some tests, not only over high delay (600 ms) links, but on 70 ms links as well, the results of which can be found at: http://www.mentat.com/skyx/skyx-nasa.html. The two graphs on the left are from these tests (used with permission).

Benchmarks – Many users are familiar with Windows and Solaris systems that have the default 8 kilobyte TCP window. We decided to perform three tests with different TCP window sizes. The results comparing the default throughput and the throughput after SkyX was enabled, are presented in the following table.

As can be seen in the table, once the SkyX was enabled, the TCP window size had no affect on throughput. The setup established that only a single Internet-2 host was routed via the SkyX boxes. SkyX boxes work in pairs – and need to be symmetrical (the topic of the next in a series of papers). The boxes need to know the available bandwidth, so it was set to 30 Mb/sec. (The actual line is a T3 – 45 Mb/sec satellite link via Intelsat 801, but 30 Mb/sec was used for this test since some of the T3 bandwidth is used for our commodity Internet traffic.) The round trip time to/from the tested host was approximately 600 ms (data courtesy of the University of Oregon and Joe St. Sauver: silo.oregon-gigapop.net). TCP performance was measured with and without the SkyX, using various window sizes. All benchmarks were performed with iperf (http://dast.nlanr.net/Projects/Iperf/).

When a very large data transfer was done (334 Mbytes) and the SkyX was enabled, we attained 26.3 Mb/sec – without touching the client and server systems on either end of the TCP session.

Conclusions – The SkyX boxes, when used over a high delay satellite link, greatly improve TCP throughput. The concept can also be used to improve throughput on WAN links with 70 ms delay.

Acknowledgments – The majority of the work setting up and benchmarking the SkyX boxes was done by Oded Comay of Tel Aviv University.

back to top

To subscribe, or if you have comments and/or questions, please contact us.
© 2000    The NLANR Measurement and Network Analysis Group,  located at the San Diego Supercomputer Center (SDSC), University of California, San Diego (UCSD).   This work is supported by the National Science Foundation (NSF) (cooperative agreement no. ANI-9807479). Any opinions, findings and conclusions or recommendations expressed in this publication are those of the author(s), and do not necessarily reflect the views of the NSF.