Quality of Service Policy Enforcement Review
Packeteer PacketShaper v. Allot NetEnforcer
September 5, 2002 Peter
Hill, Kevin Miller, Kunal Trivedi
Both the PacketShaper and
NetEnforcer products give network administrators a higher degree of control
over quality-of-service policy enforcement than traditional network switches
and routers. They offer a myriad of methods for traffic classification (IP
address, TCP/UDP ports, QoS tags, and others). After classification, policy-based
bandwidth control mechanisms are applied to the traffic. Alternately, the boxes
can mark ToS/CoS[1] bits and
allow other network elements to perform the actual traffic policing.
While both products are strong
contenders, our analysis is that the NetEnforcer line is a stronger platform
for implementing bandwidth control policies. The concept of equitable bandwidth
allocation is fundamental to the design of the unit, and its strategy of
unifying the inbound and outbound traffic for purposes of queuing seems to give
it a fundamental advantage over the PacketShaper.
Before diving into a review of the products, it’s useful to understand the rationale behind the quality of service policy enforcement products. The products in the field are slowly gaining popularity as the need becomes apparent. Specifically, traditional Ethernet-and-IP network elements (routers and switches) perform well in lightly-loaded situations, but problems arise as traffic increases. For example, shared-access Ethernet rapidly degrades in throughput after about 30% utilization. In this situation, the number of collisions and retransmissions grows quickly, reducing the network efficiency.
While the issues of half-duplex shared-Ethernet are not a problem on full duplex core network links, oversubscription of links, especially outbound links, is a problem. When machines try pushing more data than administratively allowed, routers are forced to begin queuing packets or eating into burst-buffer space. As the link utilization increases dramatically beyond the limit, constant over-subscription and queuing become the norm. Queuing has limits, however, as does user patience. While a large router with nearly infinite queues might guarantee packet delivery, most users aren’t willing to wait several days. Indeed, infinite queuing is not a solution. Most routers implement a ‘tail-drop’ system, where packets are simply dropped as the queue limit is reached. In this case, high-rate connections can overpower low-rate connections (such as simple web browsing) by simply sending or receiving more data.
The solution, as the sales people keep reminding us, is to install a QoS policy shaper, such as the PacketShaper or NetEnforcer. The policy shapers provide controls to administrators that allow far greater control of traffic than routers. Different types of traffic can be categorized and grouped, and specific policies applied to the groups. Categorization of traffic happens by inspecting the packets and matching on certain elements of the packet. This can include the source or destination IP address, source or destination TCP/UDP port numbers, or pre-defined quality of service tags. Today, categorization engines are also looking ‘deeper’ into packets, inspecting application layer content such as HTTP URLs and peer-to-peer filenames.
After categorization, administrators can then apply specific policies to the different groups of traffic. Standard policies that can be implemented include giving certain traffic a higher priority (and thus access to the bandwidth) than other traffic. Alternatively, a policy might grant a specific amount of bandwidth to traffic in a certain group, or a certain amount of bandwidth to each unique connection (for example, giving each voice-over-IP connection a specific bandwidth allocation).
The expectation, in categorizing and policing the traffic, is to reduce the need for queuing on the router. The packet shapers use different methods to manipulate TCP’s built-in flow control mechanisms, slowing down lower priority traffic. This causes some connections to begin flowing at slower speeds than they would otherwise and reduce congestion at the router.
The subtext, however, is that the packet shapers will be most effective when specific policy is encoded. They can certainly use the rate control mechanisms to reduce congestion at the router, but in general this will continue to cause high-rate connections to dominate low-rate connections. At minimum, users will continue to receive sub-par bandwidth for what they consider “important” uses (typically, interactive applications such as web browsing). Consequently, creating policies that prioritize important connections, while slowing those that do not require immediacy, can reduce the consumed bandwidth while making the network appear more responsive.
With the need for quality of service mechanisms apparent, router and switch vendors offer some controls in these products for enforcing QoS policy. The Catalyst 6500 platform, which we currently have deployed as the campus backbone, has some basic QoS controls. The 6500s can classify traffic similar to the packet shaping devices: based upon IP addresses, TCP/UDP port numbers, and ToS/CoS tags. Once classified, specific rate limits can be applied either as ‘microflow’ rates or ‘aggregate’ rates, or both. Microflow rates limit the rate a single flow can consume, while aggregate rates limit the rate that all traffic of the particular type can consume.
Unfortunately, the 6500 platform cannot perform complicated traffic shaping, such as manipulating TCP’s flow control mechanisms, to slow down out-of-profile connections. Instead, it uses a simple tail-drop system. Traffic exceeding the rate limit is simply dropped. When packets are dropped, TCP responds by scaling down the transfer rate and retransmitting the packet. Thus, while some amount of dropping is expected, excessive dropping causes a dramatic increase in the retransmission rate, further reducing the network efficiency.
Currently,
our QoS policy on our commodity Internet connection specifies explicit rates
for dormitory and non-dormitory traffic (25Mbps and 55Mbps, respectively).
Today, we are hitting these limits almost constantly (from about
In a deployment of the NetEnforcer unit to packet shape egress traffic, a sample configuration might be to have an LDAP server with the policy. This would enable easy delegation of administrative control, as well as provide rapid updates as needed. Traffic would be limited to 80Mbps in both directions.
The primary method of traffic balancing would be to use traffic priorities, enabling traffic from a low-priority group to consume more bandwidth if higher-priority classes are unused. Priority 10 is the highest priority and priority 1 the lowest. A sample provisioning of priority values might be:
|
Priority Level |
Type of Traffic |
|
10 |
Network
critical traffic (routing protocols) Interactive
traffic (telnet, SSH) below a configured rate |
|
8 |
Inbound
web traffic Priority
outbound traffic (certain web servers) |
|
7 |
Other
inbound traffic (internal connections) |
|
6 |
Other
outbound traffic (external connections) |
|
5 |
Specific
non-interactive outbound traffic (large FTP, SMTP) |
|
3 |
Specifically
identified low-priority ports or identified applications (peer-to-peer)
importing or exporting data |
The Packeteer PacketShaper is a
popular product among Universities, including peer institutions (e.g.
Stanford). They boast 30+ patents on the packet shaping technology in the unit.
The range of Packeteer products includes those that shape from 2Mbps to 200Mbps
(half-duplex[2]). Related
products to the PacketShaper include: PolicyCenter, a central policy
administration server, and ReportCenter, a central server for recording
monitoring data.
The Allot Communications
NetEnforcer comes with recommendations from Merit Networks, but is a relatively
unknown product. The range of Allot products includes those that shape from
128Kbps to 155Mbps (full-duplex[3]).
Allot also sells NetPolicy, a central policy administration server. Available
as add-ons for the NetEnforcer include: NetAccountant, which tracks usage
information (so-called “Internet call records”), NetBalancer, which redirects
client requests to available content servers, and CacheEnforcer, which redirects local content requests
through a central cache.
The matrix
below highlights many of the basic features of the units. Important differences
are displayed boldfaced. The PacketShaper has the concept of “partitions” of
bandwidth with “classes” beneath the partition. Classes identify specific types
of traffic. The NetEnforcer similarly uses “pipes” to indicate bandwidth
partitions, and “Virtual Channel” (VC) to identify specific types of traffic.
|
|
PacketShaper |
NetEnforcer |
|||
|
6500 |
8500 |
601C |
701C |
701F |
|
|
Shaping
Rate – Total |
100M |
200M |
200M |
310M |
310M |
|
List
Price (Thousands) |
$24 |
$32 |
$32 |
$38 |
$38 |
|
Maximum
Partitions/Pipes |
512 |
512 |
4,000 |
4,000 |
4,000 |
|
Maximum
Classes/VCs |
1,000 |
1,000 |
28,000 |
28,000 |
28,000 |
|
Maximum
Dynamic Partitions |
5,000 |
10,000 |
28,000 |
28,000 |
28,000 |
|
Maximum
IP Hosts |
25,000 |
100,000 |
n/a |
n/a |
n/a |
|
Maximum
Flows |
n/a |
n/a |
256,000 |
256,000 |
256,000 |
|
|
|
|
|
|
|
|
Configuration |
|
|
|
|
|
|
Web
Management Interface |
♦ |
♦ |
♦ |
♦ |
♦ |
|
CLI
Management Interface |
♦ |
♦ |
♦ |
♦ |
♦ |
|
External
LCD (+ Keypad) |
LCD |
LCD |
LCD + |
LCD + |
LCD + |
|
|
|
|
|
|
|
|
Policy Source |
|
|
|
|
|
|
Local
Policy Configuration |
♦ |
♦ |
♦ |
♦ |
♦ |
|
Single
Policy Config Screen |
|
|
♦ |
♦ |
♦ |
|
COPS |
|
|
♦ |
♦ |
♦ |
|
LDAP |
◊[4] |
◊ |
♦ |
♦ |
♦ |
|
|
|
|
|
|
|
|
Classification |
|
|
|
|
|
|
IP/MAC
Address & Port |
♦ |
♦ |
♦ |
♦ |
♦ |
|
ISL/802.1q
Classification |
♦ |
♦ |
♦ |
♦ |
♦ |
|
DiffServ/ToS |
♦ |
♦ |
♦ |
♦ |
♦ |
|
HTTP URL |
♦ |
♦ |
♦ |
♦ |
♦ |
|
Extensive Layer 7 |
♦ |
♦ |
|
|
|
|
|
|
|
|
|
|
|
Packet Shaping |
|
|
|
|
|
|
Admission
Control (ACL) |
♦ |
♦ |
♦ |
♦ |
♦ |
|
TCP
& UDP Rate Control |
♦ |
♦ |
♦ |
♦ |
♦ |
|
DiffServ/ToS/Cos
Marking |
♦ |
♦ |
♦ |
♦ |
♦ |
|
TCP Flow
Control |
♦ |
♦ |
|
|
|
|
Packet
Queuing Control |
♦ |
♦ |
♦ |
♦ |
♦ |
|
Flow-Based Traffic Control |
|
|
♦ |
♦ |
♦ |
|
Fairness of Equal Flows |
|
|
♦ |
♦ |
♦ |
|
Per-Class
Bandwidth Limits |
| ||||