Big vs. Small Packets

Introduction

PingER is one of the few active end-to-end Internet performance measurement projects to use multiple packets sizes in its long-term measurements. By default the packet sizes used by PingER are 100Bytes and 1000Bytes. Most other end-to-end active monitoring projects (AMP, RIPE, Surveyor, Skitter) only use small (<= 100Byte) packets. What is the added value obtained from these larger packets? The most obvious difference will be the round trip time (RTT) since it takes longer to process and transmit more bytes.

By measuring the differences in the RTT for the 2 packet sizes we can get an estimate of the Bytes/sec for the pair of sites. In general the RTT is proportional to l (where l is the packet length) up to the maximum datagram size (typically 1472 bytes including the 8 ICMP echo bytes). For a discussion on the effect of distance on RTT see the Tutorial on Internet Monitoring & PingER.

Longer packets may also be more likely to provoke congestion in the routers and may be more subject to Rate Limiting effects, both of which can result in higher packet losses.

Example of different response to small and large ping packets

A rather dramatic example of the different treatment of 100 and 1000Byte packets is seen below for the routers along the path from SLAC to the University of Colorado that occured on May 7th, 2000. Each router was pinged with 50 * 100Byte pings and then with 50 * 1000Byte pings and the minimum, average and maximum ping RTT noted for each batch of 50 pings together with the loss. For the 100Byte packets the loss is very small, however for the 100Byte packets bad (> 12%) loss starts at the DENV-SCRM (Denver - Sacramento) link. The increase in RTT for the longer (1000Byte) packets is also evident and increases as the RTT increases.

         pings/node=50                            100 byte packets   1000 byte packets
          NODE                                  %loss  min  max  avg %loss min  max  avg
134.79.x.x      SLAC core router                  0%    0    3    0   0%    0   12    0 Sun May  7 18:20:47 PDT 2000
134.79.x.x      SLAC legacy router                0%    0    1    0   0%    1   13    1 Sun May  7 18:22:26 PDT 2000
134.79.x.x      SLAC border router                0%    1    2    1   0%    1    3    1 Sun May  7 18:24:05 PDT 2000
192.68.191.34   RTR-STAN47.SLAC.STANFORD.EDU      0%    1    2    1   0%    2    5    2 Sun May  7 18:25:44 PDT 2000
171.64.1.115    CORE-GATEWAY.STANFORD.EDU         0%    1    3    1   0%    2    4    2 Sun May  7 18:27:23 PDT 2000
171.64.1.209    I2-GATEWAY.STANFORD.EDU           0%    1    3    1   0%    2    3    2 Sun May  7 18:29:02 PDT 2000
171.64.1.213    STAN.POS.CALREN2.NET              0%    1   39    1   0%    2    3    2 Sun May  7 18:30:42 PDT 2000
198.32.249.73   SUNV--STAN.POS.CALREN2.NET        0%    3    7    3   0%    4    9    4 Sun May  7 18:32:21 PDT 2000
198.32.249.62   ABILENE--QSV.POS.CALREN2.NET      0%    6    7    6   0%    7    9    7 Sun May  7 18:34:00 PDT 2000
198.32.8.2      DENV-SCRM.ABILENE.UCAID.EDU       0%   27   29   27  26%   29   33   29 Sun May  7 18:35:39 PDT 2000
204.131.62.41   204.131.62.41                     0%   29  267   53  22%   31  352   66 Sun May  7 18:37:19 PDT 2000
128.138.80.10   ITS-CUATM.COLORADO.EDU            8%   27   78   29  22%   30   31   30 Sun May  7 18:39:09 PDT 2000
128.138.129.25  PATCH.COLORADO.EDU                0%   27   28   27  22%   30   32   30 Sun May  7 18:40:50 PDT 2000

The differences in RTT and Loss can also be seen in the plots below where we show the average RTT (black and left hand scale in ms.) and the loss (red and right hand scale in %) for Friday May 5 thru Monday May 8th, 2000 (PST) for sets of 10 pings with 100 and 1000 byte packets sent at 1 second intervals with a 20 second timeout from SLAC to the the University of Colorado. There is one point per set/metric and the points are measured at 30 minute intervals. The first plot is for 100Byte pings the second is for 1000Byte pings. range.

It can be seen that starting at about 17:00 on May 5th thru 10:00 May 7th the losses were much higher for the 1000Byte pings. The RTT also dropped from about 60 ms to 40 ms at the same time. Looking at the Traceping Summary for this pair at this time, we can see the sudden change between 17:00 and 18:00 hours on May 5th was due to a route change. This was the result of the Abilene folks giving ESnet a new list of prefixes they route on May 5th. The new route was more symmetric and used the Abilene backbone in both directions. At this time the Abilene NOC reported that they appeared to have a serious SONET problem on the circuit from Sacramento to Denver (and not from Denver to Sacramento). Abilene worked with Qwest May 8, 2000 night to isolate a bad OC-48 Trib card in the Qwest Nortel equipment in Denver. Nortel shipped a replacement card and in the meantime, Abilene diverted traffic around the circuit. So the loss went down, but the latency was a bit higher. The change in RTT seen in the 100Byte ping plot around 21:00 on May 8th was a consequence of the above change. Abilene repaired the problem between 6:30 and 7:00am Mountain Summer time on May 9th, 2000, and the change in RTT when the traffic diversion was removed is also seen in the plots.

The net effect of the high packet loss for big packets in this instance was a measured reduction in FTP transfer rates from between 300-500kBytes/sec to under 10kBytes/sec.

Created: May 9, 2000
URL: http://www-iepm.slac.stanford.edu/pinger/tools.html
Comments to iepm-l@slac.stanford.edu