4.3.3 Replacing the cluster interconnect (Ethernet)

Save hardware configuration information in the file before replacing the cluster interconnect.
```
# /usr/sbin/prtpicl -v > pre_prtpicl.log
```

Disconnect the device from the cluster interconnect. Then, make sure "Receptacle" is "disconnected" and "Occupant" is "unconfigured" respectively.

For cfgadm command, see the documentation for devices published by Oracle Solaris.

Example: The device of the PCI card to be replaced is /dev/fjgi7, and the value indicating the location of PCI slot is pcipsy8:R0B00-PCI#slot05.
(For how to calculate the value indicating the location of PCI slot, see the documentation for devices published by Oracle Solaris.)

# cftool -d
Number Device     Type Speed    Mtu      State Configured Address
1      /dev/fjgi1 4    100      1432     UP    YES        00.00.0e.25.1a.38
2      /dev/fjgi7 4    100      1432     UP    YES        00.00.0e.25.1a.38

[If /dev/fjgi7 is displayed by "cftool -d"]

# cfrecon -d /dev/fjgi7

Delete Cluster Interconnect:

device "/dev/fjgi7" successfully removed from node configuration
# cfgadm -c disconnect pcipsy8:R0B00-PCI#slot05
# cfgadm pcipsy8:R0B00-PCI#slot05
Ap_Id                            Type       Receptacle    Occupant    Condition
pcipsy8:R0B00-PCI#slot05     unknown    disconnected  unconfigured  unknown

[If /dev/fjgi7 is not displayed by "cftool -d"]

Edit /etc/default/cluster to delete the settings of the device of the PCI card to be replaced.

[Before change]

nodename node0
clustername PRIMECLUSTER
device /dev/fjgi1
device /dev/fjgi7

[After change]

nodename node0
clustername PRIMECLUSTER
device /dev/fjgi1

If necessary, execute the following commands.

# cfgadm -c disconnect pcipsy8:R0B00-PCI#slot05
# cfgadm pcipsy8:R0B00-PCI#slot05
Ap_Id                           Type      Receptacle      Occupant     Condition
pcipsy8:R0B00-PCI#slot05     unknown    disconnected  unconfigured       unknown

CE removes the PCI card.
CE installs a new PCI card on the PCI slot.
Determine the value that indicates the PCI slot location at step 4.
For how to request the value, see the documentation for devices published by Oracle Solaris.
Add the replaced PCI card. Specify the value at step 5 after "cfgadm -c configure". Then, make sure "Receptacle" is "connected" and "Occupant" is "configured" respectively.
For cfgadm command, see the documentation for devices published by Oracle Solaris.
Example: The value determined in step 5 is pcipsy18:R0B02-PCI#slot02.
```
# cfgadm -c configure pcipsy18:R0B02-PCI#slot02
# cfgadm pcipsy18:R0B02-PCI#slot02
Ap_Id                       Type       Receptacle    Occupant    Condition
pcipsy18:R0B02-PCI#slot02     bridge/hp    connected    configured    ok
```
Save hardware configuration information in a different file than that previously used.
```
# /usr/sbin/prtpicl -v > post_prtpicl.log
```

Compare the files and specify the interface name and number of the PCI card.

# diff -c pre_prtpicl.log post_prtpicl.log | more
(snip)
!               :devfs-path	/pci@9c,4000/SUNW,hme@3,1
!               :driver-name	hme
!               :binding-name	SUNW,hme
!               :bus-addr 	3,1
!               :instance 		9
!               :_class     	obp-device
!               :name		SUNW,hme
 (The rest is omitted)

Add the PCI card in the cluster interconnect using the interface name and number.

Example: When adding /dev/hme9.

# cfrecon -a /dev/hme9

Add Cluster Interconnect:

device "/dev/hme9" successfully added to node configuration

Make sure the following message is output to the console of each node.
Example: When adding /dev/hme9.
```
cf:eventlog     CF: Route recovery on /dev/hme9 to node node0. (#0000 1 2 2 2)
```

Make sure the PCI card has successfully been added in the cluster interconnect.

Example: When /dev/hme9 is successfully added.

# cftool -d
Number Device    Type   Speed  Mtu   State  Configured  Address
1      /dev/fjgi1  4    100    1432  UP     YES         00.00.0e.25.1a.38
2      /dev/hme9   4    100    1432  UP     YES         00.00.0e.25.1a.38

Note

This section describes the operations when the cluster interconnect and GLS public LAN are connected individually to one PCI card that has multiple ports.

In the case of such connection status, you must use the same device name before and after replacing the PCI card by using PCI Hot Plug. Make sure to put the same PCI card in the same PCI slot before and after changing the PCI card.

Disconnect the NIC (PCI card) to be replaced from the GLS redundant configuration.
Since the interface name below will vary depending on the environment, replace the interface name according to the environment.
- Fast switching mode
  - For Solaris 10
```
# /opt/FJSVhanet/usr/sbin/hanetnic delete -n sha0 -i fjgi7
# /usr/sbin/ifconfig fjgi7 unplumb
# /usr/sbin/ifconfig fjgi7 inet6 unplumb
```
  - For Solaris 11 or later
```
# /opt/FJSVhanet/usr/sbin/hanetnic delete -n sha0 -i net7
# /usr/sbin/ipadm delete-ip net7
```
  Note
  To return to the multiplex configuration, the IP address and the netmask of the interface specified by the ipadm delete-ip command must be set.
  Note the output result of the ifconfig command.
```
# /usr/sbin/ifconfig net7
```
- NIC switching mode
  Depending on the status of the NIC to be replaced, the following steps will be necessary.
  - If the NIC is the operation NIC, it is necessary to manually switch the NIC before changing NIC.
  - In Solaris 11 or later, if the NIC is the primary interface, it is necessary to delete the IP address settings before changing the NIC.
  1. Checking the primary interface name and the operation NIC
    Check the primary interface name. The primary interface name is displayed at the beginning of "Interface List" field. In this example, it is "net1."
```
# /opt/FJSVhanet/usr/sbin/hanetconfig print
[IPv4,Patrol]

 Name        Hostname        Mode MAC Adder/Phys ip Interface List
+-----------+---------------+----+-----------------+-------------------------+
 sha0        192.168.70.1     d   192.168.70.2      net1,net7
 sha1        -                p   00:00:00:00:00:00 sha0

[IPv6]

 Name        Hostname/prefix                   Mode Interface List
+-----------+---------------------------------+----+-------------------------+
```
    Check the operation NIC name. The operation NIC is the interface displayed with "(ON)" in Device field. In this example, it is "net1."
```
# /opt/FJSVhanet/usr/sbin/dsphanet
[IPv4,Patrol]
 Name       Status   Mode CL  Device
+----------+--------+----+----+----------------------------------------------+
 sha0       Active    d   ON   net1(ON),net7(OFF)
 sha1       Active    p   OFF  sha0(ON) 
[IPv6]
 Name       Status   Mode CL  Device
+----------+--------+----+----+----------------------------------------------+
```
  2. Stopping the HUB monitoring function and the patrol monitoring function
```
# /opt/FJSVhanet/usr/sbin/hanetpoll off
# /opt/FJSVhanet/usr/sbin/stpptl -n sha1
```
  3. Switching the physical interface
    Execute this step only if the NIC to be replaced is the operations NIC.
```
# /opt/FJSVhanet/usr/sbin/hanetnic change -n sha0
```
  4. Deleting IP address settings
    Execute this step only in Solaris 11 or later and NIC to be replaced is the primary interface.
```
# /usr/sbin/ifconfig net7
# /usr/sbin/ipadm delete-ip net1
```
    Note
    To return to the multiplex configuration, the IP address and the netmask of the interface specified by the ipadm delete-ip command must be set.
    Note the output result of the ifconfig command.
    # /usr/sbin/ifconfig net7
- GS/SURE linkage mode
  Delete the virtual interface bundled with the NIC to be replaced from the redundant configuration.
```
# /opt/FJSVhanet/usr/sbin/hanetnic delete -n sha0 -i sha2
```