PCI Hot Plug User's Guide I/O device edition - for Solaris (TM) Operating Environment -
Contents PreviousNext

Chapter 4 Network Devices

4.2 Extension of PCI cards

This section describes extension procedure of the PCI card by using PCI Hot Plug on non-redundant system and redundant system.

4.2.1 Extension of PCI cards on non-redundant system 

This subsection describes extension procedure of PCI cards without the redundant configuration software.

  1. Stop the hardware monitoring of the Machine Administration

    The hardware monitoring of the Machine Administration is stopped by using the following command.

    # /usr/sbin/FJSVmadm/prephp <Return>
  2. Add PCI cards
    1. Before extending the PCI card, please save the result (current configuration information) of the prtpicl (1M) command.
      # prtpicl -v > /tmp/prtpicl.pre <Return>
    2. Check the status of the slot.

      Subsequent examples describe extension of PCI cards to the slot "C0M01-PCI#slot02." The "C0M01-PCI#slot02" indicates the cabinet number : 0, the system board number : 1, physical slot number : 02.

      The relationship of the position of a PCI card and format of "Ap_Id" is shown below. (N: an integral number)

      Location of PCI slot Element Format of "Ap_Id"
      Cabinet of
      PRIMEPOWER 900/1500/2500
      Cabinet Number : X
      System Board Number : Y
      Physical Slot Number : ZZ
      • Solaris 8 OE and Solaris 9 OE
        pcipsyN:CXM0Y-PCI#slotZZ
      • Solaris 10 OE
        CXM0Y-PCI#ZZ
      PCI / Disk Box I/O Cabinet Number : X
      PCI / Disk Box Number : Y
      Physical Slot Number : ZZ
      • Solaris 8 OE and Solaris 9 OE
        pcipsyN:RXB0Y-PCI#slotZZ
      • Solaris 10 OE
        RXB0Y-PCI#ZZ
      other than those above Physical Slot Number : ZZ
      • Solaris 8 OE and Solaris 9 OE
        pcipsyN:PCI#slotZZ
      • Solaris 10 OE
        PCI#ZZ

      Please confirm that the PCI slot status which extended PCI cards is "empty unconfigured" by using cfgadm (1M) command.

      # cfgadm | grep C0M01-PCI#slot02 <Return>
      pcicsy15:C0M01-PCI#slot02 unknown empty unconfigured unknown
    3. To confirm the slot position at extension operation, blink the ALARM LED of Ap_Id displayed in procedure 2.b.
      # cfgadm -x led=fault,mode=blink pcipsy15:C0M01-PCI#slot02 <Return>
    4. Please confirm that READY LED of the PCI bus slot that extends a PCI card has gone out, and extend a PCI card. This operation is performed by our customer support.
    5. Please confirm that the PCI slot status which extended PCI cards is "disconnected unconfigured" by using cfgadm (1M) command.
      # cfgadm pcipsy15:C0M01-PCI#slot02 <Return>
      Ap_Id Type Receptacle Occupant Condition
      pcicsy15:C0M01-PCI#slot02 unknown disconnected unconfigured unknown
  3. Connect PCI cards

    The cfgadm command is executed with the "configure" option to "Ap_Id," or the push button of the PCI bus slot is pushed. Then extended PCI card is connected.

    In addition, a push button is valid only in multiple user mode.

    Please confirm that the slot status is "connected configured" by using the cfgadm (1M) command after the above-mentioned procedure.

    # cfgadm -c configure pcipsy15:C0M01-PCI#slot02 <Return>
    # cfgadm pcipsy15:C0M01-PCI#slot02 <Return>
    Ap_Id Type Receptacle Occupant Condition
    pcipsy15:C0M01-PCI#slot02 pci-pci/hp connected configured ok
  4. Start the hardware monitoring of the Machine Administration

    Please execute the following commands in order to update configuration information, and restart the hardware monitoring of the Machine Administration.

    # usr/sbin/FJSVmadm/postphp <Return>
  5. Connect equipment to PCI cards

    The extended PCI card is connected with network equipment by the cable.

  6. Setup driver

    The driver configuration is added by following operations.

    1. The interface name is investigated in order to configure drivers and high layer products.

      Please save the result of the prtcicl (1M) command, and obtain the difference information between current result and the result taken at "2.a." Then, the driver instance number for the extended PCI card is obtained.

      In the following example, since the instance number is 1, it can be determined that the interface name of the extended PCI card is "hme1."

      # prtpicl -v > /tmp/prtpic.post <Return>
      # diff /tmp/prtpicl.pre /tmp/prtpicl.post | more <Return>
      :
      > :status okay
      > :devfs-path /pci@8d,2000/network@1
      > :driver-name hme
      > :binding-name SUNW,hme
      > :bus-addr 1
      > :instance 1
      > :_class obp-device
      > :name network
      :
    2. It confirms that the interface name that obtained above-mentioned procedure is an interface name that corresponds to the PCI bus slot of the added PCI card.

      Please carry out this procedure, when it cannot identify relationship between an interface name and PCI bus slot by expanding two or more PCI cards especially.

      # /usr/sbin/FJSVmadm/inst2comp hme1 <Return>
      pcipsy15:C0M01-PCI#slot02
    3. The configuration of each driver is added.

      Please refer to each driver manual for details.

  7. Setup applications

    The configuration of extended PCI card used by high layer products (services, applications) is added.

    Please refer to each high layer product manual for details.

  8. Start applications

    By the following operations, please start the high layer product (service, application) used by the extended PCI card.

    1. When using TCP/IP interface

      The example in the case of using TCP/IP interface is shown below.

      Furthermore, when the high layer program which is using the PCI card needs to restart, please refer to the manual of each high layer program.

      # /usr/sbin/ifconfig hme1 plumb <Return>
      # /usr/sbin/ifconfig hme1 inet 192.168.10.10 netmask + broadcast + -trailers up <Return>
    2. When using FNA/SNA communication protocol over Ethernet

      When not using the redundant configuration software, it is necessary to start all interfaces of Netcompo FNA-LAN, and all high layer applications that are using Netcompo FNA-LAN.

      For details, please refer to the manual of Netcompo FNA-LAN and high layer applications.

4.2.2 Extension of PCI cards with new redundant system 

This subsection describes extension procedure of a new redundant system with the following redundant configuration software.

The replacement and extension of PCI cards are supported only on Redundant Line Control Function of SynfinityLink.

Notes:
Because of the dynamic exteinsion, if the environment of SynfinityLink or PRIMECLUSTER GLS was newly added, system reboot is not required.

When using other redundant software, please refer to manuals of each product.

  1. Stop the hardware monitoring of the Machine Administration

    The hardware monitoring of the Machine Administration is stopped by using the following command.

    # /usr/sbin/FJSVmadm/prephp <Return>
  2. Add PCI cards
    1. Before extending the PCI card, please save the result (current configuration information) of the prtpicl (1M) command.
      # prtpicl -v > /tmp/prtpicl.pre <Return>
    2. Check the status of the slot.

      Subsequent examples describe extension of PCI cards to the slot "R0B01-PCI#slot02." The "R0B01-PCI#slot02" indicates the I/O cabinet number : 0, the PCI / Disk Box number : 1, physical slot number : 02.

      In the extension of a new redundant system, although two or more PCI cards are extended, whenever you add a PCI card, please perform from procedure "2.b" to "3. Connect PCI cards."

      The relationship of the position of a PCI card and format of "Ap_Id" is shown below. (N: an integral number)

      Location of PCI slot Element Format of "Ap_Id"
      Cabinet of
      PRIMEPOWER 900/1500/2500
      Cabinet Number : X
      System Board Number : Y
      Physical Slot Number : ZZ
      • Solaris 8 OE and Solaris 9 OE
        pcipsyN:CXM0Y-PCI#slotZZ
      • Solaris 10 OE
        CXM0Y-PCI#ZZ
      PCI / Disk Box I/O Cabinet Number : X
      PCI / Disk Box Number : Y
      Physical Slot Number : ZZ
      • Solaris 8 OE and Solaris 9 OE
        pcipsyN:RXB0Y-PCI#slotZZ
      • Solaris 10 OE
        RXB0Y-PCI#ZZ
      other than those above Physical Slot Number : ZZ
      • Solaris 8 OE and Solaris 9 OE
        pcipsyN:PCI#slotZZ
      • Solaris 10 OE
        PCI#ZZ

      Please confirm that the PCI slot status which extended PCI cards is "empty unconfigured" by using cfgadm (1M) command.

      # cfgadm | grep R0B01-PCI#slot02 <Return>
      pcicsy18:R0B01-PCI#slot02 unknown empty unconfigured unknown
    3. To confirm the slot position at extension operation, blink the ALARM LED of Ap_Id displayed in procedure 2.b.
      # cfgadm -x led=fault,mode=blink pcipsy18:R0B01-PCI#slot02 <Return>
    4. Please confirm that READY LED of the PCI bus slot that extends a PCI card has gone out, and extend a PCI card. This operation is performed by our customer support.
    5. Please confirm that the PCI slot status which extended PCI cards is "disconnected unconfigured" by using cfgadm (1M) command.
      # cfgadm pcipsy15:R0B01-PCI#slot02 <Return>
      Ap_Id Type Receptacle Occupant Condition
      pcicsy18:R0B01-PCI#slot02 unknown disconnected unconfigured unknown
  3. Connect PCI cards

    The cfgadm command is executed with the "configure" option to "Ap_Id," or the push button of the PCI bus slot is pushed. Then a extended PCI card is connected.

    In addition, a push button is valid only in multiple user mode.

    Please confirm that the slot status is "connected configured" by using the cfgadm (1M) command after the above-mentioned procedure.

    # cfgadm -c configure pcipsy18:R0B01-PCI#slot02 <Return>
    # cfgadm pcipsy18:R0B01-PCI#slot02 <Return>
    Ap_Id Type Receptacle Occupant Condition
    pcipsy18:R0B01-PCI#slot02 pci-pci/hp connected configured ok
  4. Start the hardware monitoring of the Machine Administration

    Please execute the following command in order to update configuration information, and restart the hardware monitoring of the Machine Administration.

    # /usr/sbin/FJSVmadm/postphp <Return>
  5. Connect equipment to PCI cards

    The extended PCI card is connected with network equipment by the cable.

  6. Setup driver

    The driver configuration is added by following operations.

    1. The interface name is investigated in order to configure drivers and high layer products.

      Please save the result of the prtcicl (1M) command, and obtain the difference information between current result and the result taken at "2.a." Then, the driver instance number for the extended PCI card is obtained.

      In the following example, since the instance number is 1, it can be determined that the interface name of the extended PCI card is "hme1."

      # prtpicl -v > /tmp/prtpic.post <Return>
      # diff /tmp/prtpicl.pre /tmp/prtpicl.post | more <Return>
      :
      > :status okay
      > :devfs-path /pci@8d,2000/network@1
      > :driver-name hme
      > :binding-name SUNW,hme
      > :bus-addr 1
      > :instance 1
      > :_class obp-device
      > :name network
      :
    2. It confirms that the interface name that obtained above-mentioned procedure is an interface name that corresponds to the PCI bus slot of the added PCI card.

      Please carry out this procedure, when it cannot identify relationship between an interface name and PCI bus slot by expanding two or more PCI cards especially.

      # /usr/sbin/FJSVmadm/inst2comp hme1 <Return>
      pcicsy18:R0B01-PCI#slot02
    3. The configuration of each driver is added.

      Please refer to each driver manual for details.

  7. Setup redundant system

    When establishing the redundant configuration system, it is necessary to configure the environmental configuration of the Redundant Line Control Function or the Multipath Function, and to activate the redundant configuration system.

    For details of environmental configuration and activation, please refer to each function manual of SynfinityLink or PRIMECLUSTER GLS.

  8. Setup applications

    The configuration of high layer product (services, applications) is added.

    Please refer to the manual of each high layer product for details.

  9. Start applications

    By following operations, please start the high layer product.

    1. When using TCP/IP interface

      The high layer product (services, applications) using the TCP/IP interface is started.

      For details of starting the high layer products, please refer to the manual of each product.

    2. When using FNA/SNA communication protocol over Ethernet

      It is necessary to start all interfaces of Netcompo FNA-LAN, and all high layer applications that are using Netcompo FNA-LAN. When using the Multipath Function, the FNA (Fujitsu Network Architecture) /SNA (System Network Architecture) communication protocol over Ethernet is valid.

      For details, please refer to the manual of Netcompo FNA-LAN and high layer applications.

4.2.3 Extension of PCI cards to redundant system 

This subsection describes the procedure extending PCI cards to the existing redundant configuration system with the following redundant configuration software.

The replacement and extension of PCI cards are supported only on Redundant Line Control Function of SynfinityLink.

Notes:
Because of the dynamic extension, if the environment of SynfinityLink or PRIMECLUSTER GLS has been modified, system reboot is not required.

When using other redundant software, please refer to manuals of each product.

Moreover, when extending PCI cards to the existing redundant configuration system, it is unnecessary to operate about the high layer products. Consequently, PCI cards can be extended without stopping the high layer products.

  1. Stop the hardware monitoring of the Machine Administration

    The hardware monitoring of the Machine Administration is stopped by using the following command.

    # /usr/sbin/FJSVmadm/prephp <Return>
  2. Add PCI cards
    1. Before extending the PCI card, please save the result (current configuration information) of the prtpicl (1M) command.
      # prtpicl -v > /tmp/prtpicl.pre <Return>
    2. Check the status of the slot.

      Subsequent examples describe extension of PCI cards to the slot "R0B01-PCI#slot02." The "R0B01-PCI#slot02" indicates the I/O cabinet number : 0, the PCI / Disk Box number : 1, physical slot number : 02.

      In the extension of a new redundant system, although two or more PCI cards are extended, whenever you add a PCI card, please perform from procedure "2.b" to "3. Connect PCI cards."

      The relationship of the position of a PCI card and format of "Ap_Id" is shown below. (N: an integral number)

      Location of PCI slot Element Format of "Ap_Id"
      Cabinet of
      PRIMEPOWER 900/1500/2500
      Cabinet Number : X
      System Board Number : Y
      Physical Slot Number : ZZ
      • Solaris 8 OE and Solaris 9 OE
        pcipsyN:CXM0Y-PCI#slotZZ
      • Solaris 10 OE
        CXM0Y-PCI#ZZ
      PCI / Disk Box I/O Cabinet Number : X
      PCI / Disk Box Number : Y
      Physical Slot Number : ZZ
      • Solaris 8 OE and Solaris 9 OE
        pcipsyN:RXB0Y-PCI#slotZZ
      • Solaris 10 OE
        RXB0Y-PCI#ZZ
      other than those above Physical Slot Number : ZZ
      • Solaris 8 OE and Solaris 9 OE
        pcipsyN:PCI#slotZZ
      • Solaris 10 OE
        PCI#ZZ

      Please confirm that the PCI slot status which extended PCI cards is "empty unconfigured" by using cfgadm (1M) command.

      # cfgadm | grep R0B01-PCI#slot02 <Return>
      pcicsy18:R0B01-PCI#slot02 unknown empty unconfigured unknown
    3. To confirm the slot position at extension operation, blink the ALARM LED of Ap_Id displayed in procedure 2.b.
      # cfgadm -x led=fault,mode=blink pcipsy18:R0B01-PCI#slot02 <Return>
    4. Please confirm that READY LED of the PCI bus slot that extends a PCI card has gone out, and extend a PCI card. This operation is performed by our customer support.
    5. Please confirm that the PCI slot status which extended PCI cards is "disconnected unconfigured" by using cfgadm (1M) command.
      # cfgadm pcipsy18:R0B01-PCI#slot02 <Return>
      Ap_Id Type Receptacle Occupant Condition
      pcicsy18:R0B01-PCI#slot02 unknown disconnected unconfigured unknown
  3. Connect PCI cards

    The cfgadm command is executed with the "configure" option to "Ap_Id," or the push button of the PCI bus slot is pushed. Then a extended PCI card is connected.

    In addition, a push button is valid only in multiple user mode.

    Please confirm that the slot status is "connected configured" by using the cfgadm (1M) command after the above-mentioned procedure.

    # cfgadm -c configure pcipsy18:R0B01-PCI#slot02 <Return>
    # cfgadm pcipsy18:R0B01-PCI#slot02 <Return>
    Ap_Id Type Receptacle Occupant Condition
    pcipsy18:R0B01-PCI#slot02 pci-pci/hp connected configured ok
  4. Start the hardware monitoring of the Machine Administration

    Please execute the following command in order to update configuration information, and restart the hardware monitoring of the Machine Administration.

    # /usr/sbin/FJSVmadm/postphp <Return>
  5. Connect equipment to PCI cards

    The extended PCI card is connected with network equipment by the cable.

  6. Setup driver

    The driver configuration is added by following operations.

    1. The interface name is investigated in order to configure drivers and high layer products.

      Please save the result of the prtcicl (1M) command, and obtain the difference information between current result and the result taken at "2.a." Then, the driver instance number for the extended PCI card is obtained.

      In the following example, since the instance number is 1, it can be determined that the interface name of the extended PCI card is "hme1."

      # prtpicl -v > /tmp/prtpic.post <Return>
      # diff /tmp/prtpicl.pre /tmp/prtpicl.post | more <Return>
      :
      > :status okay
      > :devfs-path /pci@8d,2000/network@1
      > :driver-name hme
      > :binding-name SUNW,hme
      > :bus-addr 1
      > :instance 1
      > :_class obp-device
      > :name network
      :
    2. It confirms that the interface name that obtained above-mentioned procedure is an interface name that corresponds to the PCI bus slot of the added PCI card.

      Please carry out this procedure, when it cannot identify relationship between an interface name and PCI bus slot by expanding two or more PCI cards especially.

      # /usr/sbin/FJSVmadm/inst2comp hme1 <Return>
      pcipsy18:R0B01-PCI#slot02
    3. The configuration of each driver is added.

      Please refer to each driver manual for details.

  7. Connect to redundant system

    Please execute following commands in order to connect the extended PCI card to the existing redundant configuration system.

    Notes:

    On the NIC switching mode in Redundant Line Control Function, it is impossible to extend a new interface to the existing redundant configuration system. Moreover, when extending an interface on the GS/SURE linkage mode, it is necessary to extend an interface after deactivating the virtual interface (e.g. sha0) used by the GS/SURE linkage mode.

    Fast switching mode in Redundant Line Control Function
    # /usr/sbin/ifconfig hme2 plumb <Return>
    # /usr/sbin/ifconfig hme2 IP_address netmask + broadcast + -trailers up <Return>

    GS/SURE linkage mode in Redundant Line Control Function
    # /opt/FJSVhanet/usr/sbin/stphanet -n sha0 <Return>
    # /opt/FJSVhanet/usr/sbin/hanetconfig create -n sha2 -m n -i 192.168.20.10 -t hme2 <Return>
    # /opt/FJSVhanet/usr/sbin/hanetnic modify -n sha0 -t sha2,sha3,sha4 <Return>
    # /opt/FJSVhanet/usr/sbin/strhanet -n sha0 <Return>

    Multipath Function
    # /usr/opt/FJSViomp/bin/iompadm -c mpnet add mpnet0 hme2 <Return>
    # /usr/opt/FJSViomp/bin/iompadm -c mpnet start mpnet0 hme2 <Return>

    Notes:

    It is necessary to configure the IP address specified by ifconfig(1M) command in /etc/inet/hosts and /etc/hostname.hme1. Unless these are configured, the virtual interface for fast switching mode can not be activated after rebooting the system. In addition, in order to use IPv6 address, please refer to the PRIMECLUSTER GLS (4.1A20 or later) manual.

  8. Switch the redundant path

    Please switch an online communication path for extended communication path if needed. Please refer to manuals of each redundant configuration software about how to switch a communication path.


Contents PreviousNext

All Rights Reserved, Copyright(C) FUJITSU LIMITED 2005