Previous section   Next section

Recipe 2.3 Tuning Router Buffers

2.3.1 Problem

You want to change your default buffer allocations to improve router efficiency.

2.3.2 Solution

The router maintains two different sets of buffers; public buffers and interface buffers. The router uses these as temporary storage while processing packet data. You can tune the public buffer pools as follows:

Router1#configure terminal
Enter configuration commands, one per line.  End with CNTL/Z.
Router1(config)#buffers big initial 100
Router1(config)#buffers big max-free 200
Router1(config)#buffers big min-free 50
Router1(config)#buffers big permanent 50
Router1(config)#end
Router1#

And you can adjust the interface buffer pools using a similar set of commands:

Router1#configure terminal
Enter configuration commands, one per line.  End with CNTL/Z.
Router1(config)#buffers Ethernet0 initial 200
Router1(config)#buffers Ethernet0 max-free 300
Router1(config)#buffers Ethernet0 min-free 50
Router1(config)#buffers Ethernet0 permanent 50
Router1(config)#end
Router1#

2.3.3 Discussion

Before we start this discussion, we have three notes of caution on tuning buffers. First, adjusting your router's buffers is usually not necessary. Second, a poor set of buffer parameters can cause serious performance problems on your router and for traffic passing through the router. Third, if you do find that you need to adjust these parameters, the necessary adjustments will be unique to your network, and perhaps even to each router. Unfortunately, we can only offer general guidance.

The router maintains two different sets of buffers: public pools that the router can use for anything, and interface specific pools that it can only use for processing packets on that interface.

The public buffers fall into several different pools, according to their size. These are shown in Table 2-1.

Table 2-1. Public buffer pools

Buffer size

Buffer pool name

104 bytes

Small buffers

600 bytes

Middle buffers

1536 bytes

Big buffers

4520 bytes

VeryBig buffers

5024 bytes

Large buffers

18024 bytes (default)

Huge buffers

Note that the huge buffers are 18,024 bytes by default. But, unlike the other public buffer pools, you can actually change the size of the buffers in this pool:

Router1(config)#buffers huge size 36048

You can configure any size between 18,024 and 100,000 bytes for your huge buffers. Since the router can only use memory in buffer-sized chunks, having extremely large buffers can be useful if you find that you need to manipulate extremely large packets. However, the default value of 18,024 should be large enough to handle the largest MTU values for all standard interface types. It's unlikely that you'll need to adjust this parameter. The remaining buffer sizes are all fixed and cannot be adjusted.

There are four different parameters that you can adjust on each of the public buffer pools:

Router1(config)#buffers big initial 100
Router1(config)#buffers big max-free 200
Router1(config)#buffers big min-free 50
Router1(config)#buffers big permanent 50

The first command sets the number of buffers of this type that the router will allocate at boot time. If this router is in an extremely high-traffic environment, it may take a while to allocate enough buffers to handle the load. You may find that the router has a few buffer failures right after booting. You can resolve this problem by increasing the number of initial buffers.

The second command uses the keyword max-free to set the maximum number of free buffers of this type (in this case, big) that the system should keep. In the router's normal functioning, it will see periodic bursts of activity that may force it to allocate more buffers. When the burst is over, setting a relatively low value for max-free ensures that the router frees this extra memory to make it available for other purposes. But if you set it too low in an extremely busy environment, the router may not be able to allocate new buffers quickly enough to meet the demand.

In the third command, we have applied the min-free keyword to take care of the opposite side of the same problem. In order to help ensure that the router is able to handle the rising demand for packets, it will start allocating more system memory as soon as it finds that it has fewer than min-free more unused buffers of a particular type. If you specify a min-free value that is large enough, the router will be able to cope with any demands. But making the value too large will force the router to do additional work allocating additional buffers that it will never need.

In the final command we set the minimum number of buffers of this type using the keyword permanent. The router allocates this many buffers at boot time, and it will not return their memory to the general pool of memory. A good value for this parameter is high enough to reduce the amount of work that the router has to do allocating and trimming buffers, but not so high as to waste precious memory resources.

As you can see from the second example, the parameters for tuning the interface buffer pools are exactly the same as the ones we have just described for the public pools:

Router1(config)#buffers Ethernet0 initial 200
Router1(config)#buffers Ethernet0 max-free 300
Router1(config)#buffers Ethernet0 min-free 50
Router1(config)#buffers Ethernet0 permanent 50

The best way to tell whether your buffers need adjusting is to look at the output of the show buffers command:

Router1>show buffers
Buffer elements:
     498 in free list (500 max allowed)
     760166 hits, 0 misses, 0 created
   
Public buffer pools:
Small buffers, 104 bytes (total 50, permanent 50):
     50 in free list (20 min, 150 max allowed)
     265016 hits, 0 misses, 0 trims, 0 created
     0 failures (0 no memory)
Middle buffers, 600 bytes (total 25, permanent 25, peak 49 @ 1d09h):
     23 in free list (10 min, 150 max allowed)
     40749 hits, 10 misses, 30 trims, 30 created
     0 failures (0 no memory)
Big buffers, 1536 bytes (total 50, permanent 50):
     50 in free list (5 min, 150 max allowed)
     33780 hits, 0 misses, 0 trims, 0 created
     0 failures (0 no memory)
VeryBig buffers, 4520 bytes (total 10, permanent 10):
     10 in free list (0 min, 100 max allowed)
     0 hits, 0 misses, 0 trims, 0 created
     0 failures (0 no memory)
Large buffers, 5024 bytes (total 0, permanent 0):
     0 in free list (0 min, 10 max allowed)
     0 hits, 0 misses, 0 trims, 0 created
     0 failures (0 no memory)
Huge buffers, 18024 bytes (total 0, permanent 0):
     0 in free list (0 min, 4 max allowed)
     0 hits, 0 misses, 0 trims, 0 created
     0 failures (0 no memory)
   
Interface buffer pools:
Ethernet0 buffers, 1524 bytes (total 32, permanent 32):
     8 in free list (0 min, 32 max allowed)
     24 hits, 0 fallbacks
     8 max cache size, 8 in cache
     30963 hits in cache, 0 misses in cache
Serial0 buffers, 1524 bytes (total 32, permanent 32):
     4 in free list (0 min, 32 max allowed)
     54 hits, 3 fallbacks
     8 max cache size, 7 in cache
     172593 hits in cache, 32 misses in cache
Serial1 buffers, 1524 bytes (total 32, permanent 32):
     7 in free list (0 min, 32 max allowed)
     25 hits, 0 fallbacks
     8 max cache size, 8 in cache
     0 hits in cache, 0 misses in cache
   
Router1>

Let's zoom in on one of the public buffer pools to explain what the fields mean:

Small buffers, 104 bytes (total 50, permanent 50):
     50 in free list (20 min, 150 max allowed)
     265016 hits, 0 misses, 0 trims, 0 created
     0 failures (0 no memory)

This section looks at Small buffers, which are 104-byte chunks of memory. The router currently has allocated a total of 50 of these buffers; all 50 of them are permanent, meaning that the router will not attempt to return any of them to the pool of generally available memory.

In the second line, you can see that all 50 of these buffers are currently in the free list, meaning that they are all unused. The numbers 20 and 150 in this line are the min-free and max-free parameters that we discussed earlier.

In the third line, the number of hits indicates how many times the router has successfully allocated buffers from this pool. The number of misses indicates how many times the router successfully allocated a buffer from this pool, but in doing so, had to allocate additional buffers. The trims field counts the number of dynamically allocated buffers that the router has subsequently returned. The created field shows how many buffers the router has actually created in response to miss events.

The last line shows serious problems, which are the only reasons that you should alter your buffer parameters. The failures field counts the number of times that the router has attempted to allocate a buffer and failed, causing it to drop the packet. The last field is labeled "no memory". This counts the number of times a failure occurred because the router had no memory from which to allocate additional buffers. This is an extremely serious problem, which is usually best treated by adding memory to the router.

It is also important to remember that if the router tries and fails to allocate a buffer from one pool, it will request a buffer from the next largest pool. So, for example, if the router is unable to get a Big buffer to handle a 1500-byte packet, it will use one from the VeryBig pool. This is why you can sometimes see buffer hits in the VeryBig pool, even if every interface on the router has an MTU of 1500 bytes. For this reason, it is a good idea to allocate a few permanent buffers from the pool larger than your highest MTU.

Now let's look at the interface buffers:

Ethernet0 buffers, 1524 bytes (total 32, permanent 32):
     8 in free list (0 min, 32 max allowed)
     24 hits, 0 fallbacks
     8 max cache size, 8 in cache
     30963 hits in cache, 0 misses in cache

This shows a similar set of values to what we just discussed for the public buffer pools, but there are a few differences. The first difference is the fallbacks field. This counts the number of times that the router has needed additional buffers on this interface, and has allocated them from the corresponding public buffer pool of the appropriate size. In this case, the Ethernet buffers are 1524 bytes, so the router would allocate additional buffers from the Big buffer public pool.

The router keeps a cache of buffers on each interface that are effectively in use whether there is data or not. This field varies somewhat depending on the hardware type. But, once again, you should watch out for misses. As long as the number of misses and fallbacks are low, there is no need to adjust the interface buffers.

We would like to offer one final caution on adjusting buffers. Always make sure to look at your router's free memory with the show memory command before and after making any adjustment:

Router1#show memory 
                Head    Total(b)     Used(b)     Free(b)   Lowest(b)  Largest(b)
Processor     17DA4C    13112756     2308632    10804124    10577100    10663072
      I/O     E00000     2097152      336980     1760172     1740988     1759812

Keep close track of how much the free memory changes when you adjust your router's buffers. Both the processor and the I/O memory can be affected by these changes. If you inadvertently overallocate your buffers while trying to improve system performance, you may find that the router does not have enough memory to operate properly when the load increases.


  Previous section   Next section
Top