IEPM

Bulk Thruput Measurements-IN2P3

SLAC Home Page
Bulk Thruput Measurements | Bulk throughput simulation | Windows vs. streams | Effect of load on RTT and loss | Bulk file transfer measurements

SLAC to IN2P3 (Lyon France)

The IN2P3 host (ccasn02) was a 4 * 296 MHz processor Sun 450 with 1GB memory and a 100 Mbps Ethernet interface connected to a Cisco 3524XL switch. It was running Solaris 5.6. The first plot is throughput measured from SLAC to IN2P3 on September 2, 2000. The second plot is throughput from IN2P3 to SLAC measured on October 28, 2000.
SLAC to IN2P3 bulk thruput IN2P3 to SLAC thruput
The next 3 plots show thruput from SLAC to IN2P3 measured on July 6, 2001 around 6am and around 9am, and on July 11, 2001 around 10pm. In June 2001 a 155Mbps link from IN2P3 to CERN was activated, and the traffic between SLAC & IN2P3 was routed by this path. The traceroute for July 6th shows the route then went through CERN. The pipechar shows that the bottleneck is over 100Mbps. Note that pharlap is a Sun 4 cpu host running Solaris 5.8 at 336MHz with a Gbps Ethernet interface. In the three graphs it can be seen that the improvement in throughput is linear with number of parallel streams for window sizes below 64 kBytes. The fourth plot is from pharlap.slac.stanford.edu to ccasn07.in2p3.fr measured on August 8, 2001. Ccasn02 was a special test machine. It was a Sun Enterprise 450 with 4*400MHz processors, 1GByte memory and a 1Gbit/s network interface. It is running Solaris 5.7. It is seen that the throughput is improved by about 25% compared to ccasn02 (with a 100Mbps Ethernet interface). We also measured the pipechar characterization to this host.

The optimum window size for a single stream is predicted using the bandwidth * RTT product. The bandwidth estimated from the throughput measurements is about 80Mbps, and from pipechar is about 100Mbps. The RTT from ping measurements is about 175 msec. Thus the optimum window size is about 2.2 MBytes. This is above the maximum (tcp_max_buf = 1048576 by default) that Solaris by default allows the send and receive socket buffer window sizes to be set to. We are working to increase tcp_max_buf to 4194304 Bytes and tcp_cwnd_max to 2097152 at both ends (see TCP Tuning Guide for Distributed Application on Wide Area Networks). It can also be observed that an optimal throughput of between 70 and 80 Mbps appears to be achieved for a product of window size * number of streams of 2048 Bytes. As the product of window size * number of streams exceeds 2048, the throughput becomes very variable. Finally we have measured the Effect of cpu utilization of the iperf client and as might be expected kernel cpu utilization goes up with throughput and for a fixed window size * number of streams or fixed throughput, cpu utilization increases with number of streams (e.g. for a throughput of 30 Mbps, the kernel cpu utilization goes from about 5% for one or two streams to about 8% for 30 streams).
SLAC to IN2P3, July 6/2001 SLAC to IN2P3, July 6/2001 SLAC to IN2P3, July 11/2001 SLAC to IN2P3, August 6/2001

IN2P3 to SLAC

We measured the iperf TCP throughput from IN2P3 (ccasn02) to SLAC (pharlap) on July 29th, 2001. The results are shown to the right. It can be seen that the maxima (between 50 and 60 Mbits/s) are less than those for SLAC to IN2P3 (70 to 80Mbits/s). When we obtained access to the test host at IN2P3 (ccasn07 an Enterprise 450 with 4*400MHz cpus with a Git/s Ethernet link) we measured the iperf throughput from SLAC to IN2P3 on August 6th. The results are also shown to the right. It can be seen that the maxima of the throughputs did not change very much. By comparing with the graphs above for the SLAC to IN2P3 throughputs, it can be seen that the throughput from SLAC to IN2P3 is about 80% greater than from In2P3 to SLAC.

Created August 25, 2000, last update August 29, 2001.
Comments to iepm-l@slac.stanford.edu