Load Balancing Between Server Blades Within ATCA Platforms

Author: James Radley, Principal Architect
Issue Date: October 2007

Introduction

Many application platforms within the network are architected as a conglomerate of identical server blades, each supporting their own set of assigned clients. Very often these services act as "bumps in the wire," meaning that the servers sit between the client and the client's real intended server. Consequently, the client is unaware of this relationship with the intermediate server. As a result, when the client's packets arrive at the service platform they are addressed for the ultimate intended target and not for the intermediate server, which is providing some invisible value-add function. Similarly, returning packets destined for a client arrive at the "bump in the wire" with an IP header designed to reach the client and not the assigned intermediary server.

An important function in such architectures is the Load Balancer. For every incoming packet, the Load Balancer needs to be able to quickly determine which intermediate server is assigned to handle this particular traffic flow. When a new flow is initiated, the Load Balancer needs to employ a suitable algorithm to identify a server available to take on the responsibility of maintaining this new relationship.

This paper looks at how various solutions can be implemented to provide such a Load Balancing function. Solutions which use statistical Load Balancing can be implemented directly within the latest generation of 10 Gigabit Ethernet switches, while more sophisticated solutions can be built on top of modern Packet Processors. Such implementations can track millions of client sessions and, when assigning work load, take into account the existing level of commitment within the pool of potential servers. This paper gives a detailed overview as to how such solutions can be architected and implemented on an ATCA platform.

Typical Applications

A wide variety of packet applications are being devised to address the challenges being presented by the exponential growth in packet traffic on the world's networks. Operators want to be able to maximize network efficiency, be able to protect their investments by appropriately charging for the services being delivered over their networks, and protect their customers from receiving hostile packets sent to them with mal intent.

The various applications in this space can be broadly categorized as being a mix of traffic shaping/scheduling, content-aware, security, monitoring, filtering, etc. They do, however, share the common characteristic of being dynamically stateful only within the context of individual sessions and flows. Consequently, nearly all such packet processing applications can be implemented on an array of individual nodes, each serving an allocated subset of user sessions and flows.

Until recently, Load Balancers have been provided as expensive add-on boxes which would take up additional and valuable space in the system rack. With the move to 10GbE in the backplane driving ATCA platforms to being the compelling choice for network element platforms, the opportunity arises to fold the Load Balancing function into ATCA switches and/or packet processing blades. The net benefit is to reduce the cost of implementing the Load Balancing function while saving the space that discrete Load Balancing boxes would otherwise have taken up in a particular rack.

Going With The Flow

The stream of packets which is associated with a particular application running on a specific client device is known as a flow. It is important that packets within a particular flow are always forwarded to the same intermediate server; this is because most applications need to maintain some flow-related state information. Consequently, a forwarding algorithm needs to not only be "fair" but the Load Balancing decision must be repeatable for all packets within any one flow. There may often be a requirement that the returning downstream packets of a particular flow will also go through the same server as their associated upstream packets; therefore, the algorithm must also be capable of supporting "symmetrical" operation.

A "flow" can be fully identified from the following packet attributes:

• Layer 2 (Ethernet) source and destination MAC addresses
• Layer 3 IP source and destination addresses (plus sometimes the protocol ID)
• Layer 4 source and destination port numbers

Depending on the design of a service platform, some or all of these attributes may be important for identifying a flow. For instance, when clients may be hiding behind a network address translation (NAT) firewall, then the port numbers are critically important for differentiating individual clients (since many clients will share a common IP address). The protocol ID is useful when different service handling is required for different classes of Transport Protocol. For instance, some types of applications, such as Session Border Controller, may provide back-to-back proxy TCP termination but do not intervene on UDP packets. Use of Layer 2 MAC addresses for identifying flows is useful when an upstream service is using a promiscuous IP address to hide the fact that the service is being provided by a server farm (with its own Load Balancer).

Switch-Based Load Balancing

Typically an ATCA-based platform will incorporate dual redundant switch blades to provide a backplane interconnect between the other nodes in the chassis. Therefore, there is an architectural attraction in being able to incorporate a Load Balancing function into such a switch as it avoids the need of having an additional blade in the system to provide this service. Also, since packets have to pass through the switch anyway, there is an advantage in terms of latency by not having the packets pass through an additional device.

Modern 10GbE switch technology often incorporates sophisticated packet classifying and routing functions - and these features can be leveraged to provide a statistical Load Balancing service at relatively little incremental cost. A statistical Load Balancer does not specifically take into consideration the current spare capacity of the various servers to which it is assigning new flows, but instead tries blindly to be even-handed in how it allocates workload.

One such method is to leverage the logic in the switch that may have been provided to support Equal Cost Multipath Protocol (ECMP). ECMP is a method of statistically and evenly distributing packets over a set of parallel links. It does not really matter to the ECMP mechanism if it is being used to distribute packets over a parallel set of links or in fact is being coerced into distributing packets to a bank of servers. It is important that the ECMP mechanism can be relied upon to reach a consistent routing decision per flow. ECMP, as per RFC 2992, is particularly well suited for Load Balancing as it has been designed to overcome potential difficulties caused by out-of-order packet delivery by hashing header fields to identify and route flows consistently.

For a case in point, Fulcrum Microsystems' FM4224 10GbE switch chip has the capability not only to perform a Pearson's Hash on select L2, L3 and L4 fields to allocate an outbound port, but it can also add additional Ethernet attributes to the outgoing packet based upon the result derived from the hashing function. For instance, the Load Balanced packet can be assigned a VLAN tag to indicate further downstream routing requirements, or the F64 tag field can be used to add information about the original ingress and intended egress port.

The ability to encapsulate additional flow-related information into a packet allows the Load Balancer to relay important information to an application running on the interposed server which might not otherwise be discernable from the original packet itself.

Continuous Computing has worked with technology partner Fulcrum Microsystems to develop a comprehensive Load Balancing appliance on top of the FlexCore ATCA-FM40 10GbE ("FM40") switch. This value-add software can be ordered pre-installed on the FM40 and is available as an integrated solution.

Capacity-Aware Load Balancing

Sometimes statistical Load Balancing isn't a sufficiently precise method of identifying which server should be targeted for a new flow. A more complex, user-supplied algorithm may be required to identify which server is most suitable for accepting the new flow. There may also be a requirement for the Load Balancer to maintain a dynamic list of available servers; this is particularly relevant when the Load Balancer is fronting a large server farm in which servers may be regularly added or removed.

One possible algorithm is to associate new flows with servers using a weighted round-robin scheduler. The scheduler is informed of how to weigh each server through a periodic check on individual server loading. The observed instantaneous load on each server is scored against each server on the "available" list. This scoring will be used to drive a proportional allocation of new flows to servers based upon their relative spare capacity. When the failure of a server is detected, the server is taken off the available list. Replaced servers can be added to the list when they register their availability after having been put back into service.

Such an algorithm may be better at keeping a fair and even load distribution than that achieved by the simple statistical approach. This is particularly true when the set of available servers is anticipated to be dynamic in size. The statistical Load Balancer described earlier can offer resilience in an N+1 environment by redirecting traffic to a spare if a server fails; however, it cannot handle the introduction of additional blades without disrupting the hashing table and, in so doing, losing track of existing flow-to-server associations.

The Load Balancer can provide a stateful failover to a redundant partner so that existing flows will maintain their existing associations with servers, but now with packets being routed through the standby if the active Load Balancer fails.

Continuous Computing's FlexPacket ATCA-PP50 ("PP50") packet processing blade, which provides two RMI XLR 732 packet processors on a 10GbE- (PICMG 3.1 option 9) enabled ATCA blade, is ideally suited to such an application.

This type of Load Balancer solution could employ a Linux environment for OAM-type functions for running the connection control logic, handling any exception packets, and implementing state replication and timely failover to the standby unit. RMI's lightweight execution environment, known as RMIOS, can be utilized to ensure that there would be very high packet forwarding performance for client internet traffic. An OEM could develop its own Load Balancing application or could engage Continuous Computing's Trillium Professional Services team to implement a bespoke system on their behalf.

Performance Expectations

Fulcrum switch-based systems, such as FM40, can achieve Load Balancing at full 10GbE line rates.

The performance realized by a PP50-based Load Balancing solution is dependent to some extent upon the size of the lookup tables - although this can be mitigated through use of a TCAM¹ or sophisticated multi-level hash-based lookup tables. The performance will also be dependent upon how much of a PP50's XLR resource is dedicated to the Load Balancing task. The PP50 carries two XLRs, each of which have execution support available for up to 32 threads. A number of these threads can be dedicated to Load Balancing, the remainder being available for other packet processing or system management tasks.

Continuous Computing estimates that a single XLR should be able to handle a very large table of 1.5 million subscribers at rates of 3 million packets per second. It can be reasonably expected that a Load Balancer based upon the PP50 could support in excess of 1 million client sessions at sustained 10GbE line rates, with spare capacity for other packet processing functions.

Summary

Load Balancing is an important function with multiple alternatives for implementation. Performance, reliability, repeatability, and price - all are key considerations. As a leader in providing fully-integrated systems optimized for telecom equipment manufacturers, the experts at Continuous Computing are available to discuss your project's Load Balancing requirements and the solution alternatives.

For more information about Load Balancing options, tradeoffs, and implementation considerations, please contact a Continuous Computing Sales Director.

¹TCAM: Ternary Content Addressable Memory, a memory device that is referenced by its contents rather than being referenced by an address. In simple terms, a conventional memory device is given an address and the data stored at that address (index) is returned. A CAM is the opposite, in that you feed it some data and it gives you the index associated with that data. A TCAM is the same as a CAM except that it allows "don't care" states in the "searched-for" data. TCAMs are typically used for implementing IP forwarding tables. The PP50 supports a TCAM mezzanine card which can be shared by both of the XLRs on the blade.

About Continuous Computing

Continuous Computing provides integrated systems and services that enable telecom equipment manufacturers to rapidly deploy Next Generation Networks (NGN). Over 150 customers worldwide benefit from the company's unique blend of customized professional services, Trillium protocol software, AdvancedTCA and CompactPCI systems, and BladeCenter hardware. Continuous Computing helps customers reduce platform lifecycle costs, optimize data delivery, and accelerate deployments of NGN, 3G/4G Wireless, and IP Multimedia Subsystem (IMS) infrastructure. The company is ISO-9001 and CMMI certified and based in San Diego with development centers in China and India. For more information, visit www.ccpu.com.

Continuous Computing, the Continuous Computing logo, Create | Deploy | Converge, FlexTCA, Flex21, FlexChassis, FlexCompute, FlexCore, FlexDSP, FlexPacket, FlexStore, FlexSwitch, FlexTCA, Network Service-Ready Platform, Quick!Start, TAPA, Trillium, Trillium+plus, and the Trillium logo are trademarks or registered trademarks of Continuous Computing Corporation. Other names and brands may be claimed as the property of others.