We have used iperf, bbcp and pipechar to make bandwidth estimates from SLAC to about 32 hosts in 8 countries. The following table summarizes how well pipechar bandwidth estimates track iperf or bbcp memory to memory throughputs. The numbers in the cells in the table below are the rough bandwidth estimates (by eye) from the various methods for 21 days starting Feb 9 '02 and ending Mar 2 '02. The right hand column summarizes the agreement of pipechar with the other two methods. In the iperf and bbcp columns, the window size follows w=, and the number of streams follow s=.

The data being analyzed came from time series plots. One may also look at the scatter plots of pipechar vs iperf and the frequency histograms of the measurements.

Pipechar gives reasonable agreement with iperf in about 50% of the cases.

It is seen that in general, with the exception of two hosts at ORNL, that if iperf predicts a throughput of 140Mbits or more, then the pipechar prediction is low. Six hosts (~25%) fall into this category. Possibly there are some clock timing errors due to the small separation of packets at these high bit rates. Some of the cases where pipechar is high with respect to iperf maybe due to incorrect choice of the windows and stream settings for iperf. We are working with Jin Guojun the developer of pipechar to get a new version of pipecahr to address this. In the meantime we have stopped running pipechar to these hosts. The hosts node1.roma1.infn.it and node1.mib.infn.it both display an interesting step down in predicted throughput around February 20 '02. After Feb 20, they exhibit much better agreement with iperf.

From the author of pipechar Jin Guojon:
Be aware when comparing iperf/bbcp/any TCP measurement tools vs. bandwidth measurement tools. They are to scale, but will not match because iperf/bbcp/other TCP benchmark tools are measuring the feasible throughput, not bandwidth. The TCP throughput is lower than the available bandwidth in 100% cases. The original pipechar was not designed to measure bandwidth. This is a common error during the pipechar dissemination.

Summary of Pipechar agreement with iperf and bbcpmem

Nodenames (33)	BBCPMEM Mbits/s	IPERF Mbits/s	PIPMIN Mbits/s	Summary
node1.cacr.caltech.edu	100-150 w=256k s=8	200-300 w=256k s=8	100	Pipechar low by factor 2-3
node1.ccs.ornl.gov	50 w=1024k s=8	150-200 w=1024k s=8	200	Reasonable agreement with iperf
node1.cern-chi.ch	80-120 w=256k s=8	80-120 w=256k s=8	100	Reasonable agreement
node1.cern.ch	20 w=256k s=8	30 w=256k s=8	90	Pipechar factor 3 high
node1.clrc.ac.uk	40 w=256k s=15	75 w=256k s=15	90	Pipechar agreement with iperf
node1.cs.wisc.edu	N/A w=128k s=40	N/A w=128k s=40	225
node1.ctaalliance.org	03/02-08:47 99.51 w=1024k s=8	250 w=1024k s=8	100	Pipechar factor 2 low
node1.dl.ac.uk	35 w=256k s=8	35 w=256k s=8	20	Pipechar factor  1.8 low
node1.ece.rice.edu	35 w=256k s=8	35 w=256k s=8	35	Pipechar good agreement
node1.eecis.udel.edu	30 w=1024k s=1	30 w=1024k s=1	60	Pipechar factor 2 high
node1.gsfc.nasa.gov	w=512k s=1	30 w=512k s=1	115	Pipechar factor 3 high
node1.in2p3.fr	45 w=256k s=12	50 w=256k s=12	100	Pipechar factor 2 high
node1.jlab.org	35 w=256k s=8	035 w=256k s=8	65	Pipechar factor 2 high
node1.kek.jp	16 w=128k s=8	17 w=128k s=8	18	Pipechar good agreement
node1.lanl.gov	230 w=512k s=2	290 w=512k s=2	100	Pipechar factor 3-3 low
node1.mcs.anl.gov	380 w=64k s=40	380 w=64k s=40	50	Pipechar factor 7-8 low
node1.mib.infn.it	13 w=128k s=20	15 w=128k s=20	16	Pipechar reasonable agreement
node1.nersc.gov	90 w=256k s=8	94 w=256k s=8	95	Pipechar reasonable agreement
node1.nikhef.nl	55 w=512k s=2	58 w=512k s=2	60	Pipechar reasonable agreement
node1.nslabs.ufl.edu	30 w=256k s=8	80 w=256k s=8	80	Pipechar agrees with iperf
node1.rcf.bnl.gov	w=64k s=10	70 w=64k s=10	130	Pipechar factor 1.8 high
node1.riken.go.jp	26 w=128k s=30	45 w=128k s=30	60	Pipechar factor 1,25 high
node1.roma1.infn.it	11 w=64k s=20	24 w=64k s=20	28	Pipechar reasonable agreement with iperf
node1.sdsc.edu	133 w=64k s=8	150 w=64k s=8	110	Pipechar factor 1.3 low
node1.sox.i2.edu	135 w=1024k s=1	140 w=1024k s=1	115	Pipechar factor 1.2 low
node1.stanford.edu	90 w=256k s=8	93 w=256k s=8	95	Pipechar good agreement
node1.triumf.ca	15 w=128k s=8	15 w=128k s=8	17	Pipechar good agreement
node1.utdallas.edu	14 w=256k s=8	15 w=256k s=8	42	Pipechar factor 2.5 high
node2.ccs.ornl.gov	150-200 w=1024k s=8	245 w=1024k s=8	190	Pipechar reasonable agreement
node2.cern.ch	30-60 w=256k s=8	40-80 w=256k s=8	100	Pipechar factor 2 high
node2.in2p3.fr	70-100 w=64k s=40	25 w=64k s=40		N/A
node2.nersc.gov	090 w=256k s=8	94 w=256k s=8	80	Pipechar reasonable agreement
node2.nslabs.ufl.edu	315 w=256k s=8	330 w=256k s=8	100	Pipechar factor 3 low