Bulk Throughput Measurements - University of Wisconsin, Madison

Bulk Throughput Measurements | Bulk Throughput Simulation | Windows vs. streams | Effect of load on RTT and loss | Bulk file transfer measurements

Rice to SLAC

On October 28, 2001, measurements were made between from pharlap.slac.stanford.edu to donald.cs.wisc.edu. Donald was a PC a 1000MHz PC with a GE interface running Solaris 2.4. The site was connected to Internet 2 via an OC12 (622Mbps) link. It was located at the University of Wisconsin in Madison. Pharlap was a Sun E4500 with 6*336MHz cpus and a GE interface running Solaris 5.8. SLAC had a 1Gbps link to the Stanford campus and from there a 622Mbps link to CalREN and Internet 2. The routes were asymmetric. From SLAC to U Wisconsin it uses Internet 2 vis Stanford, CalREN 2 & Abilene. From U Wisconsin it uses ESnet. The ping response for 1962 default length (64 Bytes) from SLAC to Wisconsin was min/avg/max (std) = 53.8/55.85/16.8 (4.8) ms. The pipechar from SLAC to U Wisconsin was also recorded. The window buffer sizes on pharlap are shown below:
ndd /dev/tcp tcp_max_buf = 4194304
;ndd /dev/tcp tcp_cwnd_max = 2097152
;ndd /dev/tcp tcp_xmit_hiwat = 16384
;ndd /dev/tcp tcp_recv_hiwat = 24576

The window buffer sizes on donald are shown below:
more /proc/sys/net/core/wmem_max = 8388608
;more /proc/sys/net/core/rmem_max = 8388608
;more /proc/sys/net/core/rmem_default = 65536
;more /proc/sys/net/core/wmem_default = 65536
;more /proc/sys/net/ipv4/tcp_rmem = 4096 87380 4194304
;more /proc/sys/net/ipv4/tcp_wmem = 4096 65536 4194304

The performances shows that the maxima (the top 10% measurements) achieve over 75Mbits/s throughput. It is evident that to get the best performance one needs many parallel streams. We used snoop on pharlap to verify that the window scaling factor was being set correctly in both directions. We were unable to get iperf to run with greater than 40 streams. The asymmetries inthe throughput (SLAC to Wisconsin is greater than Wisconsin to SLAC) may be related to the route asymmetry.
SLAC-Wisc throughput Wisc-SLAC throughput

Created October 28, 2001, last update October 29, 2001.
Comments to iepm-l@slac.stanford.edu