Top
PRIMECLUSTER DR/PCI Hot Plug User's Guide
Chapter 4 PRIMECLUSTER > 4.3 PCI Hot Plug operation in SPARC M10/SPARC Enterprise > 4.3.3 Replacing the cluster interconnect (Ethernet)
Previous

4.3.3 Replacing the cluster interconnect (Ethernet)

The procedure for replacing the PCI card can vary depending on whether the PCI card is only connected to the cluster interconnect or connected to the cluster interconnect and GLS Public LAN.

4.3.3.1 The PCI card is only connected to the cluster interconnect

  1. Save hardware configuration information in the file before replacing the cluster interconnect.

    # /usr/sbin/prtpicl -v > pre_prtpicl.log

  2. Check the location of the PCI slot on which the PCI card is installed. Check the PCI slot by blinking the ALARM LED of the PCI slot. Stop the blinking LED after the slot is checked.

    # cftool -d
Number Device    Type Speed    Mtu     State  Configured  Address
1      /dev/fjgi1  4    100     1432     UP    YES       00.00.0e.25.1a.38
2      /dev/fjgi7  4    100     1432     UP    YES       00.00.0e.25.1a.38
# /usr/sbin/FJSVmadm/inst2comp fjgi7
pcipsy8:R0B00-PCI#slot05
# cfgadm -x led=fault,mode=blink pcipsy8:R0B00-PCI#slot05
# cfgadm -x led=fault,mode=off pcipsy8:R0B00-PCI#slot05

  3. Disconnect the device from the cluster interconnect. Then, check if "Receptacle" is "disconnected" and "Occupant" is "unconfigured" respectively.

    # cfrecon -d /dev/fjgi7

Delete Cluster Interconnect:

device "/dev/fjgi7" successfully removed from node configuration
# cfgadm -c disconnect pcipsy8:R0B00-PCI#slot05
# cfgadm pcipsy8:R0B00-PCI#slot05
Ap_Id                            Type       Receptacle    Occupant    Condition
pcipsy8:R0B00-PCI#slot05     unknown    disconnected  unconfigured  unknown

  4. CE removes the PCI card that is installed on the PCI slot.

  5. CE installs a new PCI card on the PCI slot.

  6. Determine the value that indicates the PCI slot location.

    See the "ap_id" format described in "1.3.1 Displaying the PCI card" of the "Oracle Solaris Administration: Devices and File Systems.".

  7. Enable the PCI card on the Solaris OE. Specify the value at step 6 after "cfgadm -c configure". Then, check if "Receptacle" is "connected" and "Occupant" is "configured" respectively.

    # cfgadm -c configure pcipsy18:R0B02-PCI#slot02
# cfgadm pcipsy18:R0B02-PCI#slot02
    Ap_Id                       Type       Receptacle    Occupant    Condition
pcipsy18:R0B02-PCI#slot02     bridge/hp    connected    configured    ok

  8. Save hardware configuration information in a different file than that previously used.

    # /usr/sbin/prtpicl -v > post_prtpicl.log

  9. Compare the files and specify the interface name and number of the PCI card.

    # diff -c pre_prtpicl.log post_prtpicl.log | more
(snip)
!               :devfs-path	/pci@9c,4000/SUNW,hme@3,1
!               :driver-name	hme
!               :binding-name	SUNW,hme
!               :bus-addr 	3,1
!               :instance 		9
!               :_class     	obp-device
!               :name		SUNW,hme
 (The rest is omitted)

  10. Add the PCI card in the cluster interconnect using the interface name and number.

    # cfrecon -a /dev/hme9

Add Cluster Interconnect:

device "/dev/hme9" successfully added to node configuration

  11. Check that the following message is output to the console of each node.

    cf:eventlog     CF: Route recovery on /dev/hme9 to node node0. (#0000 1 2 2 2)

  12. Confirm that the PCI card has successfully been added in the cluster interconnect.

    # cftool -d
Number Device    Type   Speed  Mtu   State  Configured  Address
1      /dev/fjgi1  4    100    1432  UP     YES         00.00.0e.25.1a.38
2      /dev/hme9   4    100    1432  UP     YES         00.00.0e.25.1a.38

4.3.3.2 When the cluster interconnect and GLS public LAN are connected to the PCI card

Note

In this section, the cluster interconnect and GLS public LAN are connected to each port of the PCI card.

You cannot use a different device name before or after the PCI card replacement with PCI Hot Plug. Be sure to install the same PCI card into the same PCI slot as before its replacement.

  1. Delete the GLS physical interface using the following command. Change the interface name in the arguments of the following command according to a GLS configuration.

    • Fast switching mode

      # /opt/FJSVhanet/usr/sbin/hanetnic delete -n sha0 -i fjgi6
# /usr/sbin/ifconfig fjgi6 unplumb

    • NIC switching mode

      # /opt/FJSVhanet/usr/sbin/hanetpoll off
# /opt/FJSVhanet/usr/sbin/stpptl -n sha1
# /opt/FJSVhanet/usr/sbin/hanetnic change -n sha0

  2. Check the location of the PCI slot on which the PCI card is installed. Check the PCI slot by blinking the ALARM LED of the PCI slot (*1). Stop the blinking LED (*2) after the slot is checked.

    # cftool -d
Number Device    Type  Speed  Mtu   State Configured  Address
1      /dev/fjgi1  4     100   1432  UP     YES          00.00.0e.25.1a.38
2      /dev/fjgi7  4     100   1432  UP     YES          00.00.0e.25.1a.38
# /usr/sbin/FJSVmadm/inst2comp fjgi7
pcipsy8:R0B00-PCI#slot05
# cfgadm -x led=fault,mode=blink pcipsy8:R0B00-PCI#slot05           (*1)
# cfgadm -x led=fault,mode=off pcipsy8:R0B00-PCI#slot05             (*2)

  3. Disconnect the device from the cluster interconnect by using the device name and PCI slot number checked in step 2 . Then, check if "Receptacle" is "disconnected" and "Occupant" is "unconfigured" respectively.

    # cfrecon -d /dev/fjgi7

Delete Cluster Interconnect:

device "/dev/fjgi7" successfully removed from node configuration
# cfgadm -c disconnect pcipsy8:R0B00-PCI#slot05
# cfgadm pcipsy8:R0B00-PCI#slot05
Ap_Id                              Type       Receptacle    Occupant    Condition
pcipsy8:R0B00-PCI#slot05     unknown    disconnected  unconfigured  unknown

  4. CE replaces the PCI card that is installed on the PCI slot.

  5. Enable the replaced PCI card on the Solaris OE. Check if "Receptacle" is "connected" and "Occupant" is "configured" respectively.

    # cfgadm -c configure pcipsy8:R0B00-PCI#slot05
# cfgadm pcipsy8:R0B00-PCI#slot05
Ap_Id                       Type       Receptacle   Occupant   Condition
pcipsy8:R0B00-PCI#slot05      bridge/hp    connected   configured   ok

  6. Add the PCI card to the GLS settings using the following command. Change the interface name and IP address in the arguments of the following command according to a GLS configuration.

    • Fast switching mode

      # /usr/sbin/ifconfig fjgi6 plumb
# /usr/sbin/ifconfig fjgi6 192.168.10.10 netmask + broadcast + -trailers up
# /opt/FJSVhanet/usr/sbin/hanetnic add -n sha0 -i fjgi6

    • NIC switching mode

      # /opt/FJSVhanet/usr/sbin/hanetnic change -n sha0
# /opt/FJSVhanet/usr/sbin/strptl -n sha1
# /opt/FJSVhanet/usr/sbin/hanetpoll on

  7. Add the replaced PCI card to the cluster interconnect.

    # cfrecon -a /dev/fjgi7

Add Cluster Interconnect:

device "/dev/fjgi7" successfully added to node configuration

  8. Check that the following message is output to the console of each node

    cf:eventlog     CF: Route recovery on /dev/fjgi7 to node node0. (#0000 1 2 2 2)

  9. Confirm that the PCI card has successfully been added in the cluster interconnect.

    # cftool -d
Number Device   Type  Speed  Mtu   State  Configured  Address
1      /dev/fjgi1  4    100   1432   UP     YES          00.00.0e.25.1a.38
2      /dev/fjgi7  4    100   1432   UP     YES          00.00.0e.25.1a.38
Previous
Copyright FUJITSU LIMITED 2015