Installing GridFTP in a Persistent Monitoring Infrastructure

Fawad Nazir, Les Cottrell, updated: 6/15/05

We started with building and installing GridFTP only from the globus package. The process was quite simple and GridFTP was installed quite easily. While testing GridFTP we came up with the security (certificates) problem. Bill suggested that sites would not allow the use of GridFTP-Lite features (e.g. anonymous FTP server access) on the WAN.  Bill also pointed out that one needs the certificates to open the connections so one cannot avoid certificates by using memory to memory copies.

We got the PPDG certificate and planned to follow the commands to configure security for GridFTP. We thought we saw that security commands like openssl etc were not present in the GridFTP-only installation (this is incorrect, GridFTP does install openssl). Bill Allcock pointed out that Globus was not necessary if we used grid-cert-request which is pre-installed with GridFTP. However, we decided to use the PPDG X.509 certificates and so we believed we had to pre-install Globus.

After trying different suggestions/commands from Bill, we were not successful. Then from the globus4.0 documentation we read the simple security setup tutorial and we decided to uninstall the GridFTP only version and reinstall the whole globus package. This time we did the security setup and then tried testing GridFTP and it worked. The security setup took like a whole day not because it was difficult but it had lots of commands to run (some of which ran in user space, others in root) and we missed some and were stuck, then with the help of Bill we got certificates configured successfully. A cleaned up listing of the console shows some of the successful commands we used.

Currently we have security certificates configured and GridFTP installed. The problem now is that to run iperf server remotely, to run memory to memory tests or to run disk to disk test for all of the above, we have to have certificates at both ends and need Globus and GridFTP set up at the other end (SLAC initially).  Currently we run the IEPM-BW tests at regular intervals without manual intervention. However, the Grid proxy for the certificate has a lifetime (specified on the grid-proxy-int command with the -hours H option). Bill suggest setting the validity time to say a month, and having a cron job to check the duration left and emailing the IEPM monitoring maintainer to manually renew the proxy. This should work for a few sites but would be painful for multiple sites. An alternative might be to start the remote server on demand by using ssh (using something like globus-job-run cmssrv10.fnal.gov /opt/iperf/bin/iperf -s -w 1MB -p 20000 would still require a certificate) on the remote end. 

We had hoped to get away from using ssh to start remote servers since it requires an account at the remote end. Several sites are not willing to give out accounts unless one goes through security training at the remote site which does not scale to tens of sites.  Instead they are willing to have a server running all the time and possibly only allow access from specific clients. This model does not work for GridFTP since for persistent servers one needs a certificate and an account at the remote end.

The command for running disk to disk transfer was found to be grid-url-copy, to run memory to memory one replaces the filenames with /dev/zero and /dev/null..A problem with running memory to memory tests is that the client will need to be killed since /dev/zero has an infinite length and there is no timeout option for GridFTP (Bill will add one in future in GridFTP4.2, so this is only a temporary problem).

The current state is that we have GridFTP/Globus and PPDG certificates installed etc. at BNL and SLAC. We have started the server at the SLAC end, but the client has problems with the certificates (see the console listing).