Bulk File Transfer Measurements
Les Cottrell and Andy Hanushevsky, SLAC
Bulk throughput measurements | Bulk throughput simulation | Windows vs. streams | Effect of load on RTT and loss | Bulk file transfer measurements | QBSS measurements
|
Bulk File Transfer MeasurementsLes Cottrell and Andy Hanushevsky, SLAC
Bulk throughput measurements | Bulk throughput simulation | Windows vs. streams | Effect of load on RTT and loss | Bulk file transfer measurements | QBSS measurements |
We also made measurements with different file
sizes to verify that
we were close to the maximum throughput, i.e.
to within the measurement deviation errors,
the throughput had reached the asymptotic limit for an
infinitely long file. Examples of such measurements are shown to the right
for CERN (each point was measured ~ 30 times) and on the
SLAC LAN (each p0int was measured about 25 times). The differences in
transfer rate maxima are ~70Mbits/s compared to about 450Mbits/s. The measurements
indicate asympotia is reached
for file sizes of 100MBytes (transfer time ~ 7-10 secs) and
50Mbytes - 100Mbytes (transfer time 1-2 seconds). The transfer time is here estimated as
the file size / throughput. Henceforth we mainly use 100MByte files for the
measurements. In order to see how the throughput increased at start of a copy, we
also used bbcp with 64KByte window and 4 streams
to record the incremental throughput from plato.cacr.caltech.edu to
pharlap.slac.stanford.edu for each second for the first
couple hundred of secconds of a copy. The results are shown below for 4 copies.
The 3rd and 4th copies use the axis to the right which is offset from the
left hand y axis to help distinguish overlapping points.
It is seen that the throughput rapidly rises to its maximum throughput.
Though the previous measurements on the effect of file sizes (or individual measurement
durations) indicated that for 100MByte files or durations of < 10 seconds
asymptotia was reached, the measurements were made for multiple streams. The slow rise
of the throughput with time of copy (duration) seen above for a single stream, required us
to measure carefully the impact of duration on throughput with window size
for a single stream. In particular we were concerned about biases against measurements with
a single stream leading to mistaken conclusions about the effectiveness of using
multiple streams.
To measure copies taking a long time (say 320 seconds) would require
a very long file (over 10GBytes) for transfer rates of 300Mbits/s, so we decided to make the
measurements using iperf rather than bbcp. We selected durations of 2, 5, 10, 20, 40, 80,
160, 250 and 320 seconds, and window sizes of 256, 512, 1024, 2048 and 4096 KBytes. For each
of the above possible pairs we made a single stream
measurement of the iperf TCP throughput 10 times from
plato.cacr.caltech.edu to pharlap.slac.stanford.edu. The results are shown to the right
for 5 measurements for each pair of possible values (window, duration). The points are
the medians of each set of 17 measurements, and the error bars are determined from the
Inter Quartile Ranges (IQRs).
It is seen that though the media continue to rise for durations of over 10 seconds
(by about 10% going from 10 to 20 seconds)
to within the accuracy of the measurements this is a small effect. So for most of our
measurements we settled on 10 second durations.
Using the Unix top
command we looked at memory utilization and it appeared that both the iperf client and
server resided comfortably in main memory. The client on plato appeared to use about 1MByte
and typically there were 367MBytes free, The server on pharlap
used about 75MBytes when running, and there were typically 450MBytes free.
