Bulk Throughput Measurements - LANL, New Mexico

Bulk Throughput Measurements | Bulk Throughput Simulation | Windows vs. streams | Effect of load on RTT and loss | Bulk file transfer measurements

SLAC to LANL

On November 5th, 2001, measurements were made between from pharlap.slac.stanford.edu to plop.lanl.gov. On Novemeber 5th plop was a PC a 933MHz PC with a GE interface running Linux 2.4. The site was connected to Internet 2 via an OC12 (622Mbps) link. LANL is located near Santa Fe New Mexico. Pharlap was a Sun E4500 with 6*336MHz cpus and a GE interface running Solaris 5.8. On November 5th SLAC had an OC3 link to ESnet. The routes are symmetric and over ESnet in both directions. The window buffer sizes on pharlap are shown below:
ndd /dev/tcp tcp_max_buf = 4194304
;ndd /dev/tcp tcp_cwnd_max = 2097152
;ndd /dev/tcp tcp_xmit_hiwat = 16384
;ndd /dev/tcp tcp_recv_hiwat = 24576

The window buffer sizes on plop on November 5th were the default of 64Kbytes are shown below. It is seen that the limited window sizes of 64kBytes result in no change in performance as the requested window size goes above 32KBytes (bear in mind iperf on Linux gives twice the window size requested). However despite this limitation the maxima reach over 120 Mbps which is not bad considering the SLAC link was an ATM OC3 (155Mbps) link.

On November 6th plop was upgraded to Linux 2.4 and the window buffer sizes set to
:more /proc/sys/net/core/wmem_max = 2097152
;more /proc/sys/net/core/rmem_max = 2097152
;more /proc/sys/net/core/rmem_default = 65535
;more /proc/sys/net/core/wmem_default = 65535
;more /proc/sys/net/ipv4/tcp_rmem = 4096 87380 2097152
;more /proc/sys/net/ipv4/tcp_wmem = 4096 16384 2097152

In addition the link from pharlap to ESnet was upgraded to OC12 Packet Over SoNet (POS) (622Mbps). The bandwidth was measured with pipechar. The iperf improvement is striking and seen below. The maxima went for 120Mbps to over 400Mbps.

Created November 5, 2001, last update November 6, 2001.
Comments to iepm-l@slac.stanford.edu