SLAC to IN2P3 (Lyon France)
The IN2P3 host (ccasn02) was a 4 * 296 MHz processor Sun 450 with 1GB memory and
a 100 Mbps Ethernet interface connected to a Cisco 3524XL switch. It was
running Solaris 5.6.
The first plot is throughput measured from SLAC to IN2P3 on September 2, 2000.
The second plot is throughput from IN2P3 to SLAC measured on October 28, 2000.
The next 3 plots show thruput from SLAC to IN2P3 measured on
July 6, 2001
around 6am and around 9am, and on July 11, 2001 around 10pm.
In June 2001 a 155Mbps link from IN2P3 to CERN was activated, and
the traffic between SLAC & IN2P3 was routed by this path.
The traceroute for July 6th shows the
route then went through
CERN. The pipechar shows that the
bottleneck is over 100Mbps. Note that pharlap is a Sun 4 cpu host running
Solaris 5.8 at 336MHz with a Gbps Ethernet interface.
In the three graphs it can be seen that the improvement
in throughput is linear with number of parallel streams for
window sizes below 64 kBytes.
The fourth plot is from pharlap.slac.stanford.edu to
ccasn07.in2p3.fr measured on August 8, 2001. Ccasn02 was a
special test machine. It was a
Sun Enterprise 450 with 4*400MHz processors, 1GByte memory
and a 1Gbit/s network interface. It is
running Solaris 5.7. It is seen that the throughput is improved
by about 25% compared to ccasn02 (with a 100Mbps Ethernet interface).
We also measured the pipechar
characterization to this host.
The optimum window size for a single stream
is predicted using the bandwidth * RTT product. The bandwidth
estimated from the throughput measurements is about 80Mbps, and from pipechar
is about 100Mbps. The RTT from ping measurements is about 175 msec. Thus the
optimum window size is about 2.2 MBytes. This is above the maximum
(tcp_max_buf = 1048576 by default) that Solaris by default allows the
send and receive socket buffer window sizes to be set to.
We are working to increase tcp_max_buf to 4194304 Bytes and
tcp_cwnd_max to 2097152 at both ends (see
TCP Tuning Guide for Distributed Application on Wide Area Networks).
It can also be observed that an optimal throughput of between 70 and 80
appears to be achieved for a product of
window size * number of streams of 2048 Bytes.
As the product of
window size * number of streams
exceeds 2048, the throughput becomes very variable. Finally we
have measured the Effect
of cpu utilization of the iperf client and as might be expected
kernel cpu utilization goes up with throughput and
for a fixed window size * number of streams or fixed
throughput, cpu utilization
increases with number of streams (e.g. for a throughput of
30 Mbps, the kernel cpu
utilization goes from about 5% for one or two streams to about 8% for
IN2P3 to SLAC
We measured the iperf TCP throughput from IN2P3 (ccasn02) to SLAC
(pharlap) on July 29th, 2001. The results are shown to the right.
It can be seen that the maxima (between 50 and 60 Mbits/s) are less than
those for SLAC to IN2P3 (70 to 80Mbits/s).
When we obtained access to the test host at IN2P3 (ccasn07 an
Enterprise 450 with 4*400MHz cpus with a Git/s Ethernet link) we
measured the iperf throughput from SLAC to IN2P3 on August 6th.
The results are also shown to the right. It can be seen that
the maxima of the throughputs did not change very much.
By comparing with the graphs above for the SLAC to IN2P3 throughputs, it
can be seen that the throughput from SLAC to IN2P3 is about 80%
greater than from In2P3 to SLAC.
Created August 25, 2000, last update August 29, 2001.
Comments to email@example.com