Note
CF over IP is not available in Linux.
Although the two terms "CF over IP" and "CIP" (also known as "IP over CF") sound similar, they are completely different functions.
In general, the cluster interconnects are separate from the public network and not used by the TCP/IP stack. To allow applications to use TCP/UPD protocols on top of CF, CF uses the CIP driver.
CIP routes the TCP/IP traffic through the cluster interconnects to the other nodes in the cluster. CIP uses the interfaces configured for CF and does failover and load balancing if multiple interfaces are available.
CIP defines a reliable IP interface for applications on top of the Cluster Foundation (CF).
Figure 1.1 CIP diagram
CIP should not be used in a CF over IP configuration. Instead of creating an additional CIP address for applications like RMS, take the IP address assigned to CF directly.
CF over IP is used for configurations where the nodes are separated by long distances and where standard Ethernet wiring cannot be used. This can be due to the maximum cable length or different segments connected through routers. In this case, CF sends its protocol messages through the IPv4 stack instead of using the low-level network interfaces. This is not operated on IPv6. The IP stack needs to be configured up front and must be available before you start the cluster.
The CF node discovery is done by a JOIN message. This message is sent to the configured destination address. This can be the address of the remote node or the broadcast address of this subnet. Broadcast messages do not travel across routers and subnets. If the remote node is on a different subnet the broadcast address cannot be used or the cluster will not join.
It is recommended to separate the CF networks from the public network. Up to four IP devices can be configured for CF but the best way would be to configure only one IP device and let the IP layer deal with multi path or bonding.
To configure the IP interface over the private interconnect, use an IP address designed for the private network, such as in the following example:
192.168.0.xx is an integer between 1 and 254.
During the cluster joining process, CF sends broadcast messages to other nodes; therefore, all the nodes must be on the same local network. If one of the nodes is on a different network or subnet, the broadcast will not be received by that node. Therefore, the node will fail to join the cluster.
Note
Use CF with the Ethernet link-level connection whenever possible because CF over IP implies additional network/protocol information and usually will not perform as well.
Figure 1.2 CF over IP diagram
