Divert Socket Mechanism

For request distribution at front nodes, we needed some mechanism to capture incoming IP packets for HTTP connections before reaching to TCP layer and rewrite destination address of IP packets and re-inject them back in the network without TCP layer knowing about them. Similarly, we needed mechanism to change destination IP address of incoming IP packets for HTTP connection with local private address, rewrite their destination address and inject back in protocol stack to make TCP layer believe that these packets came for aliased secondary IP address.

We wanted to select destination server address and rewrite all IP packets at application layer at front node. Divert socket provides us exactly same set of features. We used standard firewalling mechanism (that uses high performance packet filtering within Kernel after a setsocketopt call) to divert all IP packet with destination TCP port 80 to divert socket port, where our application read them and modified them. After modification divert sockets provides options to re-inject packets in local TCP/IP stack or on network. At front node, packets are re-injected on network, while at servers packets are allowed to pass up to higher protocol layer in same machine.

Thus divert socket provided an easy mechanism for request distribution at front node within cluster. However, divert socket requires patching of kernel for divert socket support.

More details about divert sockets can be found at http://www.anr.mcnc.org/$\sim$divert/index.shtml