Software components at front node of each cluster

Front node is responsible for distributing requests coming to cluster, it takes into account load on each server and previous request rate of client (if available), before dispatching request to any server. We are using only incoming IP packet destination IP address rewriting to dispatch client requests among servers in cluster. So selected server depends on client IP address (hence its previous request rate) and server load. We use IP firewalling mechanism that in turn uses Berkeley packet filter facility inside the kernel at low level to filter packets coming for HTTP port and using divert socket (that stops packet from going up in TCP/IP stack) packets are received by application program, which selects server and writes selected server's IP address in destination address field and re-injects that packet back in the network.

Front node also receives load updates (heart beat messages) and asynchronous alarm messages about overload and underload situations of server. It also receives client request rate in the last 64 seconds from each server and aggregates this information. Each front node is in sync with DNS for alarm ticks. So all front nodes in the system, receive load updates at almost the same time, aggregate and send the aggregated load information to DNS. Thus DNS receives latest and consistent information about all clusters. Front nodes also send request rate information of clients having very high request rate (above the average request rate of client domains) to DNS but this information can put more load on DNS so this information is sent in distributed manner by different front nodes every 64 seconds.

Separate optional application on front node also receives requests from DNS to send ICMP echo requests to select clients. It sends ICMP echo request messages to those clients and reports RTTs between clients and that cluster. Since only small percentage of clients are sent ping messages, and clients whose recent RTT information is available are contacted only after a refresh time interval in our proposed architecture, load on the front node due to it is not expected to be high. This responsibility can be handed over to the least loaded server in cluster easily.