Email from Julian Bunn 11/12/03

We will create this week several large data files each of
about 200GB. These files will be placed on the various servers we 
have at LA, Chicago, CERN and eventually Phoenix. Each file will be a 
tar of about one hundred 2GB ROOT data files that 
can be analysed by our ROOT-based application. (ROOT cannot handle 
data files larger than 2GB.)

We need these very large aggregate files so that the transfer 
times are sufficiently long that the TCP windows open up fully.

For the Bandwidth Challenge the idea will be to set up several 
simultaneous file transfers of these tar files that attempt to 
saturate the 20GBits drops to the CACR Booth, and the 
SLAC booth as well. Selected files, on arrival, will be un-tarred, an 
individual ROOT file selected, and the event contents shown in the 
ROOT display dynamically. Some files will be transferred to the 
disk servers in the Booth, others will be transferred to /dev/null 
on the servers without fast disk subsystems.

We think this scheme gives us the best shot at saturating the 
available bandwidth, whilst at the same time using a real physics 
application. It is of course somewhat artificial, but there is 
real physics in there.

We have plenty of time next week to explore many different
aspects:

1) iperf transfers from CERN attempting to break our existing 
records with the extra routed distance between LA and Phoenix 

2) disk to disk file transfers: we want to try to better our 
existing single server to single server file transfer, over the 
longest path possible

3) Use of the FAST protocol in comparison with the standard 
stacks for all the activities

4) others?

My feeling is we have plenty of time during the show to organise 
ourselves and achieve a lot. The only "crunch time" is the Bandwidth 
Challenge slot on Wednesday, when we absolutely have to have as much 
working as possible. (However, I strongly doubt that we will win 
the challenge, since there are other folks at SC2003 who are using 
UDP and have more drops than we do.)

What do others think?

Julian