OSCARS Results

IEPM Home Page IEPM Site Map Network Monitoring PingER IEPM Papers and Presentations

Introduction

Measurements

OSCARS_76 Measurements

Following this we ran an iperf/TCP client sending data for 60 seconds in a single stream from SLAC to CERN port 5010 (best-effort) on pcgiga.cern.ch. The RTT was about 170ms and we used a window of 10MBytes. We then repeated this test except to port 5011 (the OCSARS QoS port). The results indicate that with best effort (no limit on the bandwidth) we were able to achieve about 7.6Mbits/s in both cases. The OSCARS log showed no packets being policed.

We then repeated the test with 20 parallel streams and a 256KByte TCP send window. The results indicate that with best-effort we could achieve an aggregate of about 150Mbits/s, but with the OSCARS 10Mbits/s QoS reservation we were limited to 9.45Mbits/s, and individual streams typically were running at about 500Kbits/s. At the end of these multi-stream tests the OSCARS counters showed considerable number of policed packets.

We then logged onto OSCARS again and reduced the dedicated bandwith to 1 Mbits/s and repeated the tests with one stream. The results show there was no change in the best effort (port 5010) performance, however, as expected the performance on the OSCARS 1 Mbits/s QoS reservation was less than 1 Mbits/s (actual value 874Kbits/s).

Since iperf has difficulties providing 1 second interval updates we then repeated the 1Mbits/s reservation tests for 10 second durations with 1 second interval reporting with thrulay. Since thrulay only supports a single stream the measurements were made with a single stream only. The results indicate that with best effort service we achieved about 8.2Mbits/s. Thrulay also reports the Round Trip Time (RTT). Looking at the RTT it appears there was only elongation of the RTTs (indicating possible congestion) during the first second of the transfer. During this first second the throughput was also less than for the following 9 seconds. Presumably the first second effects are due to TCP's slow start. The test with the OSCARS 1Mbits/s reservation showed a similar behavior for the first second. In the second second the throughput measured was 5.95Mbits/s which is greater than the advertised limit of 1 Mbits/s. The remaining 8 seconds the troughputs ranged form 0.6 to 0.064Mbits/s. The aggregate throughput was 0.961Mbits/s which is, as expected, just under the advertised 1Mbits/s limit

OSCARS-91 Measurements

To see the longer term stability of throughput we repeated the QoS measurement for 600 seconds and analyzed the throughputs reported. The plot shows that the throughput is reasonably stable (median throughput =0.944Mbits/s, Inter Quartile Range (IQR)=0.26Mbits/s, min=0.332 Mbits/s, max=1.654Mbits/s). For the policing on Junipers, there is a buffer size setting that can be set to absorb instantaneous burst. Current we've set the default to 1MByte regardless of the policing rate. This potentially can skew slow duration transfers. If we set the buffer size extremely low, then the sending host must strictly police the flow to the requested bandwidth. We're still playing around to figure out what the buffer size should be in relation to the policing bandwidth. A suggestion that we got from Juniper was to set it at 1/10 of the policing rate. We'll need to investigate more, but I think the end goal is that it will be some fraction of the policing rate.

We also looked at thrulay's reported average RTT. As can be seen from the scatter plot there is a negative correlation.

At the completion of the above measurements we recorded the OSCARS policing counters.

We then set the the burst-size-limit down from 1MB to 100KB for the oscar_91 policer, as well as cleared out the counters. Following this we repeated the thrulay measurement for 8 minutes. The plot shows that the start up is much better behaved. The long term plot shows similar behavior to that with the burst-size-limit set to 1MB. the statistics for the two settings are compared in the table below:

burst-size-limit	start	end	median	IQR	min	max
1MByte	102s	600s	0.94Mbits/s	0.26Mbits/s	0.33Mbits/s	1.65Mbits/s
0.1MByte	0s	480s	0.93Mbits/s	0.26Mbits/s	0.31Mbits/s	1.389Mbits/s

We verified that when the OSCARS reservation had terminated we were able to get 8.193Mbits/s with thrulay to port 5011. It should also be noted that with more than 20 streams of iperf, the aggregate throughput increases and maxes out at about 450-500Mbits/s.

OSCARS - 96 Measurements, Jitter

9cottrell@iepm-resp:~>ping -Q 4 iepmbw.bnl.gov | tee /tmp/940pmrs

9cottrell@iepm-resp:~>ping -iepmbw.bnl.gov | tee /tmp/940pmbe

date ; iperf -c iepmbw.bnl.gov -p 5000 -t 600 -i 5 -w 256k -P 40 | tee /tmp/940p