PCI Hot Plug User's Guide I/O device edition - for Solaris (TM) Operating Environment -
Contents PreviousNext

Chapter 3 File Devices

3.2 Extension of PCI cards

This section describes the procedures to add PCI cards using the PCI Hot Plug feature on non-redundant and redundant systems with Solaris 8 OE and Solaris 9 OE or later. The difference between the procedures for the two OEs is that I/O devices to be added must be pre-defined before system operation with Solaris 8 OE, but pre-definition is not needed with Solaris 9 OE or later.

3.2.1 Solaris 8 OE 

The procedures to add PCI cards on a Solaris 8 OE system are explained here. To use the PCI Hot Plug function for extension on Solaris OE, I/O devices to be added must be pre-defined and the system must be rebooted before system operation. If the I/O devices to be added are already defined, pre-definition is not needed, and procedure 1. below can be skipped.

If you use Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*), ensure the automap function is effective. Boot the system after modifying lpfc.conf.

# vi /kernel/drv/lpfc.conf <Return>
:
# If automap is set, SCSI IDs for all FCP nodes without
# persistent bindings will be automatically generated.
# If new FCP devices are added to the network when the system is down,
# there is no guarantee that these SCSI IDs will remain the same
# when the system is booted again.
# The bind method of the port is used as the binding method of
# automap devices to preserve SCSI IDs between link down and link up.
# If automap is 0, only devices with persistent bindings will be
# recognized by the system.
automap=1; <-- set to 1
:

3.2.1.1 Extension of PCI cards on non-redundant system 

Procedures to add PCI cards without making them redundant on a Solaris 8 OE system are explained here.

  1. Setup I/O devices which will be added to system

    Pre-define devices to be added with the following procedures according to the device type. Extension using PCI Hot Plug cannot be done without definition, so devices must be defined before the system goes into operation.

    [ Disk/Tape devices(Solaris OE: sd/st) ]

    This is an example where a disk device at target=2, lun=0 is pre-defined. If the device is a tape device, re-read "sd" in the following as "st" and perform the same operations.

    1. Confirm that target=2, lun=0 is defined in /kernel/drv/sd.conf.

      If this definition does not exist, add the definition. If it is commented out, remove the comments.

      Disk device
      # vi /kernel/drv/sd.conf <Return>
      :
      name="sd" class="scsi"
      target=0 lun=0;
      :
      name="sd" class="scsi" In case the definition does not exist,
      target=2 lun=0; add or remove comments.
      :
      Tape device
      # vi /kernel/drv/st.conf <Return>
      :
      name="st" class="scsi"
      target=0 lun=0;
      :
      name="st" class="scsi" In case the definition does not exist,
      target=5 lun=0; add or remove comments.
      :
    2. In case /kernel/drv/sd.conf was updated in 1.a., reconfigure the kernel , restart the system, and then start system operation.

      If it was not updated, this procedure is not required.

      # touch /reconfigure <Return>
      # shutdown -y -i6 -g0 <Return>

    [ Disk Array device(hard disk driver:hddv) ]

    This is an example where a disk array device at target=3, lun=0-2 is pre-defined.

    If this is the first time to pre-define a disk array device, start from procedure 1.a. If this is not the first time to pre-define a disk array device, start from procedure 1.e.

    1. Check if /kernel/drv/hddv.conf exists

      If it does not exist, the hard disk driver is being installed, so start from procedure 1.c. If it does exist, the system is being operated with non-active extension procedures, so start from procedure 1.b.

      Not exist(new install) [ to procedure 1.c.]
      # ls /kernel/drv/hddv.conf <Return>
      /kernel/drv/hddv.conf: No such file or directory
      Exists(operation with non-active extension procedures) [ to procedure 1.b.]
      # ls /kernel/drv/hddv.conf <Return>
      /kernel/drv/hddv.conf
    2. When operating with non-active extension procedures, execute the following commands and confirm that /kernel/drv/hddv.conf is deleted.
      # hddvadm -i <Return>
      # ls /kernel/drv/hddv.conf <Return>
      /kernel/drv/hddv.conf: No such file or directory
    3. Use the following command to create an hddv.conf file for the pre-defined extension function.

      The message examples for "operation with non-active extension procedures" means that GR720 currently used with taget=1(lun=0) is registered twice on the sd/hddv driver.

      New installation [ to procedure 1.e.]
      # hddvadm -M <Return>

      Operation with non-active extension procedures [ to procedure 1.d.]
      # hddvadm -M <Return>
      already ID= 1 exists. /devices/pci@1d,2000/fibre-channel@1/sd@1,0:a,raw : GR720
    4. When performing pre-defined extension, the same target number cannot be registered in sd.conf and hddv.conf, so delete or comment out the conflicting definition parts in sd.conf file when definition in the two files conflict with each other.

      The following example shows the case that target=1 is repeated.

      # vi /kernel/drv/sd.conf <Return>
      :
      name="sd" class="scsi"
      target=0 lun=0;
      :
      #name="sd" class="scsi"
      target=1 lun=0;
      #name="sd" class="scsi" Delete or comment out
      target=1 lun=1; conflicting definition parts in sd/hddv.
      #name="sd" class="scsi"
      target=1 lun=2;
      :

      A target number that is already used by the sd driver of the system disk or CD-ROM cannot be deleted. In this case, change the target number in hddv.conf and of the disk array device so that it does not conflict with sd.conf.

    5. Edit the /kernel/drv/hddv.conf file created automatically by the hddvadm -M command, and remove comments of the LUN definition to pre-define and the LUN definition currently being used with non-active extension procedures.

      If case active extension has been performed before and the LUN definition to extend the next procedures 1.f. - 1.g. are not needed.

      The following example shows target=3, lun=0~2 for pre-defined extension and disk array device GR720 (target=1) in use with non-active extension procedures being registered.

      # ls /kernel/drv/hddv.conf <Return>
      /kernel/drv/hddv.conf
      # vi /kernel/drv/hddv.conf <Return>
      :
      #name="hddv" class="scsi" target=0 lun=0;
      :
      #name="hddv" class="scsi" target=0 lun=7;
      :
      name="hddv" class="scsi" target=1 lun=0;
      name="hddv" class="scsi" target=1 lun=1; Remove comments of the LUN in use
      name="hddv" class="scsi" target=1 lun=2;
      :name="hddv" class="scsi" target=3 lun=0;
      name="hddv" class="scsi" target=3 lun=1; Remove comments of extended LUN
      name="hddv" class="scsi" target=3 lun=2;
      #name="hddv" class="scsi" target=3 lun=3;
      :
    6. Check for LUN definition conflict between sd.conf and hddv.conf by executing the hddvchk_conf command.

      The message example for "Check NG" means that target=3 is defined in both files,

      and the procedures must be performed again from procedure 1.d.

      The followings are examples of execution of the FJSVhddv package installation to /opt. The path of the hddvchk_conf command depends on the installed directory.

      Check OK [ to procedure 1.g.]
      # /opt/FJSVhddv/bin/hddvchk_conf <Return>
      /kernel/drv/sd.conf and /kernel/drv/hddv.conf seems to be correct

      Check NG [ to procedure 1.d.]
      # /opt/FJSVhddv/bin/hddvchk_conf <Return>
      NG ID = [ 3 ]
    7. Reconfigure the kernel, restart the system, and then start system operation.
      # touch /reconfigure <Return>
      # /usr/sbin/shutdown -y -i6 -g0 <Return>
  2. Stop the machine administration hardware monitoring daemon

    Execute the following commands to stop the machine administration hardware monitoring daemon.

    # /usr/sbin/FJSVmadm/prephp <Return>
  3. If you use the Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*), execute the following.

    The daemons will be stopped.

    # /etc/rc0.d/K10ElxRMSrv stop <Return>
    # /etc/rc0.d/K10ElxDiscSrv stop <Return>
  4. Add PCI cards

    Add PCI cards according to the following procedure.

    [ for PCI Fibre Channel(PW008FC3U/PW008FC2U/ GP7B8FC1U)]:

    If you use SAN management function of Systemwalker StorageMGR/Softek SANView for ETERNUS (except for Vixel) /SP5000 SRM Facility

    No procedure is necessary. Go to step 5.

    If you don't use SAN management function of the above products

    To add PCI cards with the following configurations, Fibre Channel switch SN200 series and disk array device ETERNUS3000/ETERNUS6000/GR700/800 series need to be reconfigured individually.

    - If zone configuration by WWPN(World Wide Port Name) is done on SN200 series.

    - If the Host Affinity function of ETERNUS3000/ETERNUS6000/GR700/800 series is used.

    For details, see "SN200 Series Affinity User's Guide" or "ETERNUS3000/ETERNUS6000/GR700/800 series GRmgr User's Guide." If you use the Fibre Channel switch or disk array device other than described above, see the document of each product.

    To perform the above reconfiguration, the WWPN(a 16-digit number) of the replacement card is needed. The WWPN of the PCI card can be known from the eight characters shown on a label on the front plate of the card. These characters represent the bottom eight digit of the WWPN in hexadecimal form. The top eight digit are fixed to 10000000 in hexadecimal form.

    For example, if the following label is shown on the front plate of the card, the WWPN of the replacement card is 100000000e244061.

    0e24
    4061

    [ for Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*)]:

    To replace PCI cards with the following configurations, Fibre Channel switch and disk array device need to be reconfigured individually.

    - If zone configuration by WWPN (World Wide Port Name) is done on Fibre Channel switch.

    - If the Host Zoning function of disk array device is used.

    For details, see the document of each product.

    To perform the above reconfiguration, the WWPN(a 16-digit number) of the replacement card is needed. The WWPN of the PCI card can be known from the twelve characters shown on a label on the back of the card. These characters represent the bottom twelve digit of the WWPN in hexadecimal form. The top four digit are fixed to 1000 in hexadecimal form.

    For example, if the following label is shown on the back of the card, the WWPN of the replacement card is 10000000c9366037.

    IEEE:0000c9366037

    Note:

    When changing Affinity configuration on SN200 series or other Fibre Channel switch, I/O to other devices is effected by the change, and may result in temporal errors.

    I/O to disk array devices recovers normally because of retry processes, but on Fibre Channel tape devices, backup processes may end in errors. Stop backup before changing Affinity configuration.

    1. Before extension of PCI cards, save the current execution result of the prtpicl(1M) command (current configuration at present.)
      # prtpicl -v > /tmp/prtpicl.pre <Return>
    2. Check the status of the slot.

      The relationship of the position of a PCI card and format of "Ap_Id" is shown below.

      Location of PCI slot Definition of slot Format of "Ap_Id"
      Inside chassis of
      PRIMEPOWER
      900/1500/2500
      Chassis No.: X
      System board No.: Y
      Physical slot No.: ZZ
      • Solaris 8 OE and Solaris 9 OE
        pcipsyN:CXM0Y-PCI#slotZZ
      • Solaris 10 OE
        CXM0Y-PCI#ZZ
      PCI/Disk BOX I/O Chassis No.: X
      PCI/Disk BOX No.: Y
      Physical slot No.: ZZ
      • Solaris 8 OE and Solaris 9 OE
        pcipsyN:RXB0Y-PCI#slotZZ
      • Solaris 10 OE
        RXB0Y-PCI#ZZ
      Other Physical slot No.: ZZ
      • Solaris 8 OE and Solaris 9 OE
        pcipsyN:PCI#slotZZ
      • Solaris 10 OE
        PCI#ZZ

      Below, an example of adding a PCI card to chassis number : 0, system board number : 1, physical slot number : 02,slot in chassis(C0M01-PCI#slot02) is explained.

      Check the status of the target slot is "empty unconfigured."

      # cfgadm | grep C0M01-PCI#slot02 <Return>
      pcipsy15:C0M01-PCI#slot02 unknown empty unconfigured unknown
    3. To confirm the slot position at extension operation, blink the ALARM LED of Ap_Id displayed in procedure 4.b.
      # cfgadm -x led=fault,mode=blink pcipsy15:C0M01-PCI#slot02 <Return>
    4. After checking that the READY LED on the PCI slot to add the PCI card is off, add PCI cards and connect cable to devices. This operation is performed by our customer support.
    5. Use the cfgadm(1M) command to check that the slot status of the added PCI card changed to "disconnected unconfigured".
      # cfgadm pcipsy15:C0M01-PCI#slot02 <Return>
      Ap_Id Type Receptacle Occupant Condition
      pcipsy15:C0M01-PCI#slot02 unknown disconnected unconfigured unknown
  5. Connect PCI cards

    Connect the replaced PCI card using the cfgadm(1M) command with the configure option, or by pushing the button corresponding to the extension slot position. Note that the push button is only effective in multiuser mode. After the new PCI card is connected, use the cfgadm(1M) command and confirm that the slot status has changed to "connected configured."

    If a large-scale configuration of I/O devices is connected to the PCI card in the target slot, command execution for status confirmation may take time.

    # cfgadm -c configure pcipsy15:C0M01-PCI#slot02 <Return>
    # cfgadm pcipsy15:C0M01-PCI#slot02 <Return>
    Ap_Id Type Receptacle Occupant Condition
    pcipsy15:C0M01-PCI#slot02 mult/hp connected configured ok
  6. If you use the Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*), execute the following.

    The daemons will be started.

    # /etc/rc2.d/S99ElxRMSrv start <Return>
    # /etc/rc2.d/S99ElxDiscSrv start <Return>
  7. Update hardware configuration information of machine administration/ Start the hardware monitoring daemon.

    Execute the following commands to update hardware configuration information of machine administration and to restart the hardware monitoring daemon.

    # /usr/sbin/FJSVmadm/postphp <Return>
  8. Preparation to recognize devices for Fibre Channel

    When the Host Affinity function is being used with Fibre Channel SN200 series, Host Affinity must be configured for the device added.

    [ for PCI Fibre Channel(PW008FC3U/PW008FC2U/ GP7B8FC1U)]:

    If you use SAN management function of Systemwalker StorageMGR/Softek SANView for ETERNUS (except for Vixel) /SP5000 SRM Facility

    WWPN zoning of SN200 series, Host affinity of Array disk device ETERNUS3000/ETERNUS6000/GR700/800 series and binding setting for fjpfca.conf/sd.conf of host can be added from SAN management window automatically.
    See "Operation guide" or "User's Manual" of each product for details.
    Before using this function, the following procedure 9.a.1. and 9.a.2. [except fcp-bind-target setting] are necessary.
    Moreover, begin from procedure 9.a.3. after executing this function.

    If you don't use SAN management function of the above products

    No procedure is necessary. Go to step 9.

    [ for Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*)]:

    No procedure is necessary. Go to step 9.

    Note:

    When changing Affinity configuration on SN200 series, I/O to other devices is effected by the change, and may result in temporal errors.

    I/O to disk array devices recovers normally because of retry processes, but on Fibre Channel tape devices, backup processes may end in errors. Stop backup before changing Affinity configuration.

  9. Set up driver

    Add the configuration of driver according to the following operation.

    As for file related drivers, additional configuration of the PCI card driver and the device driver is needed.

    1. Add configurations of PCI card driver

      [ SCSI driver(Solaris OE: glm , FUJITSU PCI SCSI Adapter Driver) ]

      Configuration is not required.

      [ Fibre Channel driver (PW008FC2U/ W008FC2X/GP7B8FC1U/GP7B8FC1X) ]

      1. After saving execution result of prtpicl (1M) command, take the difference between the result and the information obtained in procedure 4.a. to calculate the instance number of the fjpfca driver of the PCI card added.

        In the following example, instance number is 1, indicating that the interface name of the PCI card added is fjpfca1.

        Making the LED of the PCI card blink using the fc_adm command can specify the relation between the instance number and the slot position. The relation between the WWPN and instance number of the PCI card added can be known from the "fc_info -a" command.

        # prtpicl -v > /tmp/prtpicl.pfca <Return>
        # diff /tmp/prtpicl.pre /tmp/prtpicl.pfca | more <Return>
        :
        > :status okay
        > :devfs-path /pci@8d,2000/fibre-channel@1
        > :driver-name fjpfca
        > :binding-name pci10cf,105c
        > :bus-addr 1
        > :instance 1
        > :_class obp-device
        > :name fibre-channel
        :
        # /usr/sbin/FJSVpfca/fc_adm -l fjpfca1 <Return>
      2. Edit /kernel/drv/fjpfca.conf and add the following property definitions to the instance number from 9.a.1.

        port (topology classification , link speed)

        Define this property when PCI cards using Fabric connection are added.

        This definition is not needed with FC-AL connection.

        fcp-bind-target(connection of target ID and WWN)

        Define this property when disk array equipment using Fabric connections are added.

        This definition is not needed with FC-AL connection.

        max-throttle(number of maximum SCSI command issue)

        This property is defined for restriction on simultaneous issue of SCSI commands.

        The necessity and value of this definition depends on the specification of the disk array device, so refer to the user's manual of each equipment.

        The following example shows the definition of a fjpfca1 PCI card (instance number : 1) and disk array equipment( target = 3) being added.

        # vi /kernel/drv/fjpfca.conf <Return>
        :
        port=
        fjpfca0:nport",
        "fjpfca1:nport:sp1"; <- added

        fcp-bind-target=
        "fjpfca0t0:0x50000e1000002501",
        "fjpfca1t3:0x50000e1000002ab0"; <- added

        max-throttle=
        "fjpfca0t0:128",
        "fjpfca1t3:240"; <- added
        :
      3. Execute the following command to make the fjpfca driver take in the added part.
        # /usr/sbin/FJSVpfca/fc_config -v <Return>

      [ Fibre Channel driver (PW028FC3*/PW028FC4*/PW028FC5*) ]

      Configuration is not required.

      Compare the output of prtpicl(1M) command with the output saved at 4.b and find the instance number of the added card.

    2. Add device driver configuration

      Add device driver configuration with the following procedures according to the I/O device type.

      [ Disk/Tape devices(Solaris OE : sd/st) ]

      This example is a case where a disk device of target=2, lun=0 is pre-defined in procedure 1. and added in procedure 6. If the device is a tape device, re-read "sd" as "st" in this procedure and perform the same operation.

      1. Execute the following command to make the sd driver set up the added disk device.
        # /usr/sbin/devfsadm <Return>
      2. After saving the execution result of the prtpicl(1M) command, take the difference between the result and the information obtained in 4.a. and get the physical path name of additional disk device, then get the logical path name under /dev/dsk.

        In the following example, "c2t2d0" is the logical path name of the disk device added.

        # prtpicl -v > /tmp/prtpicl.post <Return>
        Disk device
        # diff /tmp/prtpicl.pre /tmp/prtpicl.post | grep devfs-path | grep sd <Return>
        > :devfs-path /pci@89,4000/scsi@2/sd@2,0
        # ls -l /dev/dsk | grep /pci@89,4000/scsi@2/sd@2,0 <Return>
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t2d0s0
        -> ../../devices/pci@89,4000/scsi@2/sd@2,0:a
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t2d0s1
        -> ../../devices/pci@89,4000/scsi@2/sd@2,0:b
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t2d0s2
        -> ../../devices/pci@89,4000/scsi@2/sd@2,0:c
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t2d0s3
        -> ../../devices/pci@89,4000/scsi@2/sd@2,0:d
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t2d0s4
        -> ../../devices/pci@89,4000/scsi@2/sd@2,0:e
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t2d0s5
        -> ../../devices/pci@89,4000/scsi@2/sd@2,0:f
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t2d0s6
        -> ../../devices/pci@89,4000/scsi@2/sd@2,0:g
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t2d0s7
        -> ../../devices/pci@89,4000/scsi@2/sd@2,0:h

        Tape device
        # diff /tmp/prtpicl.pre /tmp/prtpicl.post | grep devfs-path | grep st <Return>
        > :devfs-path /pci@89,4000/scsi@2/st@5,0
        # ls -l /dev/rmt | grep /pci@89,4000/scsi@2/st@5,0 <Return>
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 1
        -> ../../devices/pci@89,4000/scsi@2/st@5,0:
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 1b
        -> ../../devices/pci@89,4000/scsi@2/st@5,0:b
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 1bn
        -> ../../devices/pci@89,4000/scsi@2/st@5,0:bn
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 1c
        -> ../../devices/pci@89,4000/scsi@2/st@5,0:c
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 1cb
        -> ../../devices/pci@89,4000/scsi@2/st@5,0:cb
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 1cbn
        -> ../../devices/pci@89,4000/scsi@2/st@5,0:cbn
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 1cn
        -> ../../devices/pci@89,4000/scsi@2/st@5,0:cn
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 1h
        -> ../../devices/pci@89,4000/scsi@2/st@5,0:h
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 1hb
        -> ../../devices/pci@89,4000/scsi@2/st@5,0:hb
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 1hbn
        -> ../../devices/pci@89,4000/scsi@2/st@5,0:hbn
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 1hn
        -> ../../devices/pci@89,4000/scsi@2/st@5,0:hn
      3. If the device is a disk device, start the format(1M) command and configure labeling and partition(slice) information of the logical path name of added disks.

        If the device is a tape device, this procedure is not needed.

      [ Disk array device (hard disk driver : hddv) ]

      This example is a case where a disk array device of target=3, lun=0-2 is pre-defined in procedure 1. and added in procedure 5.

      1. There may be a change of status depending on the system operating state after pre-definition in procedure 1., so check the LUN definition conflict between sd.conf and hddv.conf again. In case of "Check NG", the definitions are conflicting, so go back to procedure 1.e.
        Check OK [ to procedure 9.b.2.]
        # /opt/FJSVhddv/bin/hddvchk_conf <Return>
        /kernel/drv/sd.conf and /kernel/drv/hddv.conf seems to be correct

        Check NG [ to procedure 1.e.]
        # /opt/FJSVhddv/bin/hddvchk_conf <Return>
        NG ID = [ 3 ]
      2. Execute the following command to make the hard disk driver set up the LUN.
        # /usr/sbin/hddvadmdr <Return>
      3. After saving execution result of the prtpicl command, get the physical path name of added LUN by taking the difference between the result and the information obtained in 4.a, and calculate the corresponding logical path name under /dev/dsk

        In the following example, "c3t3d0" is logical path name of added lun=0. The logical path name of lun=1,2 is set to "c3t3d1", "c3t3d2" respectively, because cXtYdZmeans X: PCI card number, Y: target number, Z: lun number .

        # prtpicl -v > /tmp/prtpicl.post <Return>
        # diff /tmp/prtpicl.pre /tmp/prtpicl.post | grep devfs-path | grep hddv <Return>
        > :devfs-path /pci@89,2000/fibre-channel@1/hddv@3,0
        > :devfs-path /pci@89,2000/fibre-channel@1/hddv@3,1
        > :devfs-path /pci@89,2000/fibre-channel@1/hddv@3,2
        # ls -l /dev/dsk | grep /pci@89,2000/fibre-channel@1/hddv@3,0 <Return>
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s0
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:a
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s1
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:b
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s2
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:c
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s3
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:d
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s4
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:e
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s5
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:f
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s6
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:g
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s7
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:h
      4. Start the format(1M) command and configure labeling and partition(slice) information calculated in 9.b.3. to the logical path name of each extended LUN.
    3. Add configure of PCI card driver

      [ SCSI driver(Solaris OE: glm , FUJITSU PCI SCSI Adapter Driver) ]

      No procedure is necessary. Go to step 10.

      [ Fibre Channel driver (PW008FC2U/ W008FC2X/GP7B8FC1U/GP7B8FC1X) ]

      No procedure is necessary. Go to step 10.

      [ Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*)]

      Bind the auto mapped target connected with the card and modify the entries of lpfc.conf to match the requirements of the extended Fibre Channel Controller. These procedures are reflected on next boot time. In the following, indicate to bind the auto mapped target with the card.

      # /usr/sbin/lpfc/lputil <Return>
      LightPulse Common Utility for Solaris/SPARC. Version 1.6a8 (03/03/2004).
      Copyright (c) 2003, Emulex Network Systems, Inc.

      Emulex Fibre Channel Host Adapters Detected: 2
      Host Adapter 0 (lpfc11) is an LP9802 (Ready Mode)
      Host Adapter 1 (lpfc9) is an LP9802 (Ready Mode)

      MAIN MENU
      1. List Adapters
      2. Adapter Information
      3. Firmware Maintenance
      4. Reset Adapter
      5. Persistent Bindings
      0. Exit
      Enter choice => 5 <Return>

      PERSISTENT BINDINGS MENU
      1. Display Current Bindings
      2. Display All Nodes
      3. Duplicate Adapter Bindings
      4. Bind Target Manually
      5. Bind Automapped Targets
      6. Delete Binding(s)
      0. Return to Main Menu
      Enter choice => 5 <Return>
      0. lpfc11
      1. lpfc9
      Select an adapter => 0 <Return>

      Automapped FCP Nodes:
      Node Target WWPN WWNN
      ---- ------ --------- --------
      Automapped FCP Node 0 50-06-0e-80-00-10-04-92 50-06-0e-80-00-10-04-92
      Automapped FCP Node 1 50-06-0e-80-00-10-04-90 50-06-0e-80-00-10-04-90

      Press the Enter or Return key to continue:
      Bind all auto-mapped targets? (Y/N) => Y <Return>

      Select Binding Method:
      1. By Node Name
      2. By Port Name
      3. By D_ID
      0. Cancel
      Enter Choice => 2 <Return>
      :
  10. Setup applications

    Add configuration for the application using the logical path name determined in 9.b.

    [ Disk device(filesystem operation) ]

    Create file system.

    To mount at system start, add an entry in the /etc/vfstab file.

    # newfs /dev/rdsk/c2t2d0s6 <Return>
    # vi /etc/vfstab <Return>
    :
    /dev/dsk/c2t2d0s6 /dev/rdsk/c2t2d0s6 /mnt ufs 1 yes -
    :

    [ Disk device(raw access operation)/Tape device ]

    Configure applications.

    For details, see the manual of each application.

    [ Disk device(swap device) ]

    Add swap device.

    To make valid at system start, add an entry in the /etc/vfstab file.

    # swap -a /dev/dsk/c2t2d0s4 <Return>
    # vi /etc/vfstab <Return>
    :
    /dev/dsk/c2t2d0s4 - - swap - no -
    :
    # swap -l <Return>
    swapfile dev swaplo blocks free
    :
    /dev/dsk/c2t2d0s4 32,164 16 788384 683680
  11. Start applications

    Use the logical path name determined in 9.b. to start applications.

    [ Disk device(filesystem operation) ]

    Mount filesystem

    # mount -F ufs /dev/dsk/c2t2d0s6 /mnt <Return>

    [ Disk device(raw access operation)/Tape device ]

    Start applications.

    For details, see the manual of each application.

    [ Disk device(swap device) ]

    This procedure is not required.

3.2.1.2 Extension of PCI cards on new redundant system 

The procedure to add PCI cards to a new redundant system on a Solaris 8 OE system is explained with examples for the multipath software below.

-Multipath Disk Control (MPHD)

- GR Multipath Driver (GRMPD)

If other multipath software is being used, see the manual of each product.

  1. Setup devices which will be added to system

    This is an example where an disk array device at target=3, lun=0-2 is pre-defined.

    Extension using PCI Hot Plug is impossible without definition, so definition must be done before the system goes into operation.

    [ If you use Solaris OE:sd ]

    If this is the first time to pre-define a disk array device, begin with procedure 1.a. If this is not the first time to pre-define a disk array device, go to step 2.

    1. Modify /kernel/drv/sd.conf

      Ensure target=3,lun=0 target=3,lun=1 target=3,lun=3 are defined in /kernel/drv/sd.conf.

      If they are not defined or commented, modify /kernel/drv/sd.conf.

      # vi /kernel/drv/sd.conf <Return>
      :
      name="sd" class="scsi"
      target=0 lun=0;
      :
      name="sd" class="scsi"
      target=3 lun=0;
      name="sd" class="scsi"
      target=3 lun=1;
      name="sd" class="scsi"
      target=3 lun=2;:
    2. Boot the system and reconfigure the kernel.
      # touch /reconfigure <Return>
      # /usr/sbin/shutdown -y -i6 -g0 <Return>

    [ If you use hddv ]

    If this is the first time to pre-define a disk array device, begin with procedure 1.a. If this is not the first time to pre-define a disk array device, begin with procedure 1.e.

    1. Check if /kernel/drv/hddv.conf exists

      Check if /kernel/drv/hddv.conf exists. If it exists, the hard disk driver is being installed, so start from procedure 1.c. If it does not exist, the system is being operated with none-active extension procedures, so start from procedure 1.b.

      Not exist(new install) [ to procedure 1.c.]
      # ls /kernel/drv/hddv.conf <Return>
      /kernel/drv/hddv.conf: No such file or directory

      Exists(operation with non-active extension procedures) [ to procedure 1.b.]
      # ls /kernel/drv/hddv.conf <Return>
      /kernel/drv/hddv.conf
    2. When operating with non-active extension procedures, execute the following commands and confirm that /kernel/drv/hddv.conf is deleted.
      # hddvadm -i <Return>
      # ls /kernel/drv/hddv.conf <Return>
      /kernel/drv/hddv.conf: No such file or directory
    3. Use the following command to create hddv.conf file for the pre-defined extension function. The message examples for "operation with non-active extension procedures" means that disk array device GR720 currently used with taget=1(lun=0) is registered twice on the sd/hddv driver.
      New installation [ to procedure 1.e.]
      # hddvadm -M <Return>

      Operation with non-active extension procedures [ to procedure 1.d.]
      # hddvadm -M <Return>
      already ID= 1 exists. /devices/pci@1d,2000/fibre-channel@1/sd@1,0:a,raw : GR720
    4. When performing pre-defined extension, the same target number cannot be registered in sd.conf and hddv.conf, so delete or comment out the conflicting definition parts in sd.conf file when definition in the two files conflict with each other.

      The following example shows the case that target=1 is repeated.

      # vi /kernel/drv/sd.conf <Return>
      :
      name="sd" class="scsi"
      target=0 lun=0;
      :
      #name="sd" class="scsi"
      target=1 lun=0;
      #name="sd" class="scsi" Delete or comment out
      target=1 lun=1; conflicting definition parts in sd/hddv
      #name="sd" class="scsi"
      target=1 lun=2;
      :

      A target number that is already used by the sd driver of the system disk or CD-ROM cannot be deleted. In this case, change the target number in hddv.conf and of the disk array device so that it does not conflict with sd.conf.

    5. Edit the /kernel/drv/hddv.conf file created automatically by the hddvadm -M command, and remove comments of the LUN definition to pre-define and the LUN definition currently being used with non-active extension procedures.

      If case active extension has been performed before and the LUN definition to extend the next procedures 1.f. - 1.g. are not needed.

      If the target number of each disk array controller of a redundant system differs, add definition for each target number.

      The following example shows target=3 , lun=0~2 for pre-defined extension and disk array device GR720 (target=1) in use with non-active extension procedures being registered.

      # ls /kernel/drv/hddv.conf <Return>
      /kernel/drv/hddv.conf
      # vi /kernel/drv/hddv.conf <Return>
      :
      #name="hddv" class="scsi" target=0 lun=0;
      :
      #name="hddv" class="scsi" target=0 lun=7;
      :
      name="hddv" class="scsi" target=1 lun=0;
      name="hddv" class="scsi" target=1 lun=1; Remove comments of the LUN in use
      name="hddv" class="scsi" target=1 lun=2;
      :
      name="hddv" class="scsi" target=3 lun=0;
      name="hddv" class="scsi" target=3 lun=1; Remove comments of extended LUN
      name="hddv" class="scsi" target=3 lun=2;
      #name="hddv" class="scsi" target=3 lun=3;
      :
    6. Check the LUN definition overlap between sd.conf and hddv.conf by executing hddvchk_conf command.

      The message example for "Check NG" means that target=3 is defined in both files,

      and the procedures must be performed again from procedure 1.d.

      The followings are examples of execution of the FJSVhddv package installation to /opt. The path of the hddvchk_conf command depends on the installed directory.

      Check OK [ to procedure 1.g.]
      # /opt/FJSVhddv/bin/hddvchk_conf <Return>
      /kernel/drv/sd.conf and /kernel/drv/hddv.conf seems to be correct

      Check NG [ to procedure 1.d.]
      # /opt/FJSVhddv/bin/hddvchk_conf <Return>
      NG ID = [ 3 ]
    7. Reconfigure the kernel , restart the system, and then start system operation.
      # touch /reconfigure <Return>
      # /usr/sbin/shutdown -y -i6 -g0 <Return>
  2. Stop the machine administration monitoring daemon

    Use the following command to stop the hardware disorder monitoring daemon of machine administration.

    # /usr/sbin/FJSVmadm/prephp <Return>
  3. If you use the Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*), execute the following.

    The daemons will be stopped.

    # /etc/rc0.d/K10ElxRMSrv stop <Return>
    # /etc/rc0.d/K10ElxDiscSrv stop <Return>
  4. Add PCI cards

    Add PCI cards according to the following procedure.

    [ for PCI Fibre Channel(PW008FC3U/PW008FC2U/ GP7B8FC1U)]:

    If you use SAN management function of Systemwalker StorageMGR/Softek SANView for ETERNUS (except for Vixel) /SP5000 SRM Facility

    No procedure is necessary. Go to step 5.

    If you don't use SAN management function of the above products

    To add PCI cards with the following configurations, Fibre Channel switch SN200 series and disk array device ETERNUS3000/ETERNUS6000/GR700/800 series need to be reconfigured individually.

    -If zone configuration by WWPN(World Wide Port Name) is done on SN200 series.

    -If the Host Affinity function of ETERNUS3000/ETERNUS6000/GR700/800 series is used.

    For details, see "SN200 Series Affinity User's Guide" or "ETERNUS3000/ETERNUS6000/GR700/800 series GRmgr User's Guide." If you use the Fibre Channel switch or disk array device other than described above, see the document of each product.

    To perform the above reconfiguration, the WWPN(a 16-digit number) of the replacement card is needed. The WWPN of the PCI card can be known from the eight characters shown on a label on the front plate of the card. These characters represent the bottom eight digit of the WWPN in hexadecimal form. The top eight digit are fixed to 10000000 in hexadecimal form.

    For example, if the following label is shown on the front plate of the card, the WWPN of the replacement card is 100000000e244061.

    0e24
    4061

    [ for Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*)]:

    To replace PCI cards with the following configurations, Fibre Channel switch and disk array device need to be reconfigured individually.

    -If zone configuration by WWPN (World Wide Port Name) is done on Fibre Channel switch.

    -If the Host Zoning function of disk array device is used.

    For details, see the document of each product.

    To perform the above reconfiguration, the WWPN(a 16-digit number) of the replacement card is needed. The WWPN of the PCI card can be known from the twelve characters shown on a label on the back of the card. These characters represent the bottom twelve digit of the WWPN in hexadecimal form. The top four digit are fixed to 1000 in hexadecimal form.

    For example, if the following label is shown on the back of the card, the WWPN of the replacement card is 10000000c9366037.

    IEEE:0000c9366037

    Note:

    When changing Affinity configuration on SN200 series or other Fibre Channel switch, I/O to other devices is effected by the change, and may result in temporal errors.

    I/O to disk array devices recovers normally because of retry processes, but on Fibre Channel tape devices, backup processes may end in errors. Stop backup before changing Affinity configuration.

    1. Before extension of PCI cards, save the current execution result of the prtpicl(1M) command (current configuration at present.)
      # prtpicl -v > /tmp/prtpicl.pre <Return>
    2. Check the status of the slot.

      The relationship of the position of a PCI card and format of "Ap_Id" is shown below.

      Location of PCI slot Definition of slot Format of "Ap_Id"
      Inside chassis of
      PRIMEPOWER
      900/1500/2500
      Chassis No.: X
      System board No.: Y
      Physical slot No.: ZZ
      • Solaris 8 OE and Solaris 9 OE
        pcipsyN:CXM0Y-PCI#slotZZ
      • Solaris 10 OE
        CXM0Y-PCI#ZZ
      PCI/Disk BOX I/O Chassis No.: X
      PCI/Disk BOX No.: Y
      Physical slot No.: ZZ
      • Solaris 8 OE and Solaris 9 OE
        pcipsyN:RXB0Y-PCI#slotZZ
      • Solaris 10 OE
        RXB0Y-PCI#ZZ
      Other Physical slot No.: ZZ
      • Solaris 8 OE and Solaris 9 OE
        pcipsyN:PCI#slotZZ
      • Solaris 10 OE
        PCI#ZZ

      Below, an example of adding a PCI card to chassis number : 0, PCI/Disk BOX number : 1, physical slot number : 02, slot in PCI/Disk BOX(R0B01-PCI#slot02) is explained.

      Check the status of the target slot is "empty unconfigured."

      # cfgadm | grep R0B01-PCI#slot02 <Return>
      pcipsy18:R0B01-PCI#slot02 unknown empty unconfigured unknown
    3. To confirm the slot position at extension operation, blink the ALARM LED of Ap_Id displayed in procedure 4.b.
      # cfgadm -x led=fault,mode=blink pcipsy18:R0B01-PCI#slot02 <Return>
    4. After checking that the READY LED on the PCI slot to add the PCI card is off, add PCI cards and connect cable to devices. This operation is performed by our customer support.
    5. Use the cfgadm(1M) command to check that the slot status of the added PCI card changed to "disconnected unconfigured".
      # cfgadm pcipsy18:R0B01-PCI#slot02 <Return>
      Ap_Id Type Receptacle Occupant Condition
      pcipsy18:R0B01-PCI#slot02 unknown disconnected unconfigured unknown
  5. Connect the PCI card

    Connect the replaced PCI card using the cfgadm(1M) command with the configure option, or by pushing the button corresponding to the extension slot position. Note that the push button is only effective in multiuser mode. After the new PCI card is connected, use the cfgadm(1M) command and confirm that the slot status has changed to "connected configured."

    If a large-scale configuration of I/O devices is connected to the PCI card in the target slot, command execution for status confirmation may take time.

    # cfgadm -c configure pcipsy18:R0B01-PCI#slot02 <Return>
    # cfgadm pcipsy18:R0B01-PCI#slot02 <Return>
    Ap_Id Type Receptacle Occupant Condition
    pcipsy18:R0B01-PCI#slot02 mult/hp connected configured ok
  6. If you use the Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*), execute the following.

    The daemons will be started.

    # /etc/rc2.d/S99ElxRMSrv start <Return>
    # /etc/rc2.d/S99ElxDiscSrv start <Return>
  7. Update hardware configuration information of machine administration/ Start the hardware monitoring daemon.

    Execute the following commands to update hardware configuration information of machine administration and to restart the hardware monitoring daemon.

    # /usr/sbin/FJSVmadm/postphp <Return>
  8. Preparation to recognize devices for Fibre Channel

    When the Host Affinity function is being used with Fibre Channel SN200 series, Host Affinity must be configured for the device added.

    [ for PCI Fibre Channel(PW008FC3U/PW008FC2U/ GP7B8FC1U)]:

    If you use SAN management function of Systemwalker StorageMGR/Softek SANView for ETERNUS (except for Vixel) /SP5000 SRM Facility

    WWPN zoning of SN200 series, Host affinity of Array disk device ETERNUS3000/ETERNUS6000/GR700/800 series and binding setting for fjpfca.conf/sd.conf of host can be added from SAN management window automatically.
    See "Operation guide" or "User's Manual" of each product for details.
    Before using this function, the following procedure 9.a.1. and 9.a.2. [except fcp-bind-target setting] are necessary.
    Moreover, begin from procedure 9.a.3. after executing this function.

    If you don't use SAN management function of the above products

    No procedure is necessary. Go to step 9.

    [ for Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*)]:

    No procedure is necessary. Go to step 9.

    Note:

    When changing Affinity configuration on SN200 series, I/O to other devices is effected by the change, and may result in temporal errors.

    I/O to disk array devices recovers normally because of retry processes, but on Fibre Channel tape devices, backup processes may end in errors. Stop backup before changing Affinity configuration.

  9. Set up driver

    Add the configuration of driver according to the following operation.

    As for file related drivers, additional configuration of the PCI card driver and the device driver is needed.

    1. Add configurations of PCI card driver

      [ SCSI driver(Solaris OE: glm , FUJITSU PCI SCSI Adapter Driver) ]

      Configuration is not required.

      [ Fibre Channel driver (PW008FC2U/ W008FC2X/GP7B8FC1U/GP7B8FC1X) ]

      1. After saving execution result of prtpicl (1M) command, take the difference between the result and the information obtained in procedure 4.a. to calculate the instance number of the fjpfca driver of the PCI card added.

        In the following example, instance number is 1, indicating that the interface name of the PCI card added is fjpfca1.

        Making the LED of the PCI card blink using the fc_adm command can specify the relation between the instance number and the slot position. The relation between the WWPN and instance number of the PCI card added can be known from the "fc_info -a" command.

        # prtpicl -v > /tmp/prtpicl.pfca <Return>
        # diff /tmp/prtpicl.pre /tmp/prtpicl.pfca | more <Return>
        :
        > :status okay
        > :devfs-path /pci@8d,2000/fibre-channel@1
        > :driver-name fjpfca
        > :binding-name pci10cf,105c
        > :bus-addr 1
        > :instance 1
        > :_class obp-device
        > :name fibre-channel
        :
        # /usr/sbin/FJSVpfca/fc_adm -l fjpfca1 <Return>
      2. Edit /kernel/drv/fjpfca.conf and add the following property definitions to the instance number from 9.a.1.

        port (topology classification , link speed)

        Define this property when PCI cards using Fabric connection are added.

        This definition is not needed with FC-AL connection.

        fcp-bind-target(connection of target ID and WWN)

        Define this property when disk array equipment using Fabric connections are added.

        This definition is not needed with FC-AL connection.

        max-throttle(number of maximum SCSI command issue)

        This property is defined for restriction on simultaneous issue of SCSI commands.

        The necessity and value of this definition depends on the specification of the disk array device, so refer to the user's manual of each equipment.

        The following example shows the definition of a fjpfca1 PCI card (instance number : 1) and disk array equipment( target = 3) being added.

        # vi /kernel/drv/fjpfca.conf <Return>
        :
        port=
        "fjpfca0:nport",
        "fjpfca1:nport:sp1"; <- added

        fcp-bind-target=
        "fjpfca0t0:0x50000e1000002501",
        "fjpfca1t3:0x50000e1000002ab0"; <- added

        max-throttle=
        "fjpfca0t0:128",
        "fjpfca1t3:240"; <- added
        :
      3. Execute the following command to make the fjpfca driver take in the added part.
        # /usr/sbin/FJSVpfca/fc_config -v <Return>

      [ Fibre Channel driver (PW028FC3*/PW028FC4*/PW028FC5*) ]

      Configuration is not required.

      Compare the output of prtpicl(1M) command with the output saved at 4.b and find the instance number of the added card.

    2. Add device driver configuration

      Add device driver configuration with the following procedures according to the I/O device type.

      [ Disk/Tape devices(Solaris OE : sd/st) ]

      This example is a case where a disk device of target=2, lun=0 is pre-defined in procedure 1. and added in procedure 5. If the device is a tape device, re-read "sd" as "st" in this procedure and perform the same operation.

      1. Execute the following command to make the sd driver set up the added disk device.
        # /usr/sbin/devfsadm <Return>
      2. After saving the execution result of the prtpicl(1M) command, take the difference between the result and the information obtained in 4.a. and get the physical path name of additional disk device, then get the logical path name under /dev/dsk.

        In the following example, "c2t2d0" is the logical path name of the disk device added.

        # prtpicl -v > /tmp/prtpicl.post <Return>

        Disk device
        # diff /tmp/prtpicl.pre /tmp/prtpicl.post | grep devfs-path | grep sd <Return>
        > :devfs-path /pci@89,4000/scsi@2/sd@2,0
        # ls -l /dev/dsk | grep /pci@89,4000/scsi@2/sd@2,0 <Return>
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t2d0s0
        -> ../../devices/pci@89,4000/scsi@2/sd@2,0:a
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t2d0s1
        -> ../../devices/pci@89,4000/scsi@2/sd@2,0:b
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t2d0s2
        -> ../../devices/pci@89,4000/scsi@2/sd@2,0:c
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t2d0s3
        -> ../../devices/pci@89,4000/scsi@2/sd@2,0:d
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t2d0s4
        -> ../../devices/pci@89,4000/scsi@2/sd@2,0:e
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t2d0s5
        -> ../../devices/pci@89,4000/scsi@2/sd@2,0:f
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t2d0s6
        -> ../../devices/pci@89,4000/scsi@2/sd@2,0:g
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t2d0s7
        -> ../../devices/pci@89,4000/scsi@2/sd@2,0:h

        Tape device
        # diff /tmp/prtpicl.pre /tmp/prtpicl.post | grep devfs-path | grep st <Return>
        > :devfs-path /pci@89,4000/scsi@2/st@5,0
        # ls -l /dev/rmt | grep /pci@89,4000/scsi@2/st@5,0 <Return>
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 1
        -> ../../devices/pci@89,4000/scsi@2/st@5,0:
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 1b
        -> ../../devices/pci@89,4000/scsi@2/st@5,0:b
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 1bn
        -> ../../devices/pci@89,4000/scsi@2/st@5,0:bn
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 1c
        -> ../../devices/pci@89,4000/scsi@2/st@5,0:c
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 1cb
        -> ../../devices/pci@89,4000/scsi@2/st@5,0:cb
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 1cbn
        -> ../../devices/pci@89,4000/scsi@2/st@5,0:cbn
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 1cn
        -> ../../devices/pci@89,4000/scsi@2/st@5,0:cn
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 1h
        -> ../../devices/pci@89,4000/scsi@2/st@5,0:h
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 1hb
        -> ../../devices/pci@89,4000/scsi@2/st@5,0:hb
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 1hbn
        -> ../../devices/pci@89,4000/scsi@2/st@5,0:hbn
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 1hn
        -> ../../devices/pci@89,4000/scsi@2/st@5,0:hn
      3. If the device is a disk device, start the format(1M) command and configure labeling and partition(slice) information of the logical path name of added disks.

        If the device is a tape device, this procedure is not needed.

      [ Disk array device (hard disk driver : hddv) ]

      This example is a case where a disk array device of target=3, lun=0-2 is pre-defined in procedure 1. and added in procedure 5.

      1. There may be a change of status depending on the system operating state after pre-definition in procedure 1., so check the LUN definition conflict between sd.conf and hddv.conf again. In case of "Check NG", the definitions are conflicting, so go back to procedure 1.e.
        Check OK [ to procedure 9.b.2.]
        # /opt/FJSVhddv/bin/hddvchk_conf <Return>
        /kernel/drv/sd.conf and /kernel/drv/hddv.conf seems to be correct

        Check NG [ to procedure 1.e.]
        # /opt/FJSVhddv/bin/hddvchk_conf <Return>
        NG ID = [ 3 ]
      2. Execute the following command to make the hard disk driver set up the LUN.
        # /usr/sbin/hddvadmdr <Return>
      3. After saving execution result of the prtpicl command, get the physical path name of added LUN by taking the difference between the result and the information obtained in 4.a, and calculate the corresponding logical path name under /dev/dsk

        In the following example, "c3t3d0" is logical path name of added lun=0. The logical path name of lun=1,2 is set to "c3t3d1", "c3t3d2" respectively, because cXtYdZmeans X: PCI card number, Y: target number, Z: lun number .

        # prtpicl -v > /tmp/prtpicl.post <Return>
        # diff /tmp/prtpicl.pre /tmp/prtpicl.post | grep devfs-path | grep hddv <Return>
        > :devfs-path /pci@89,2000/fibre-channel@1/hddv@3,0
        > :devfs-path /pci@89,2000/fibre-channel@1/hddv@3,1
        > :devfs-path /pci@89,2000/fibre-channel@1/hddv@3,2
        # ls -l /dev/dsk | grep /pci@89,2000/fibre-channel@1/hddv@3,0 <Return>
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s0
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:a
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s1
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:b
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s2
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:c
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s3
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:d
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s4
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:e
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s5
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:f
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s6
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:g
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s7
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:h
      4. Start the format(1M) command and configure labeling and partition(slice) information calculated in 9.b.3. to the logical path name of each extended LUN.
    3. Add configure of PCI card driver

      [ SCSI driver(Solaris OE: glm , FUJITSU PCI SCSI Adapter Driver) ]

      No procedure is necessary. Go to step 10.

      [ Fibre Channel driver (PW008FC2U/ W008FC2X/GP7B8FC1U/GP7B8FC1X) ]

      No procedure is necessary. Go to step 10.

      [ Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*)]

      Bind the auto mapped target connected with the card and modify the entries of lpfc.conf to match the requirements of the extended Fibre Channel Controller. These procedures are reflected on next boot time. In the following, indicate to bind the auto mapped target with the card..

      # /usr/sbin/lpfc/lputil <Return>
      LightPulse Common Utility for Solaris/SPARC. Version 1.6a8 (03/03/2004).
      Copyright (c) 2003, Emulex Network Systems, Inc.

      Emulex Fibre Channel Host Adapters Detected: 2
      Host Adapter 0 (lpfc11) is an LP9802 (Ready Mode)
      Host Adapter 1 (lpfc9) is an LP9802 (Ready Mode)

      MAIN MENU
      1. List Adapters
      2. Adapter Information
      3. Firmware Maintenance
      4. Reset Adapter
      5. Persistent Bindings
      0. Exit
      Enter choice => 5 <Return>

      PERSISTENT BINDINGS MENU
      1. Display Current Bindings
      2. Display All Nodes
      3. Duplicate Adapter Bindings
      4. Bind Target Manually
      5. Bind Automapped Targets
      6. Delete Binding(s)
      0. Return to Main Menu
      Enter choice => 5 <Return>
      0. lpfc11
      1. lpfc9
      Select an adapter => 0 <Return>
      Automapped FCP Nodes:
      Node Target WWPN WWNN
      ---- ------ --------- --------
      Automapped FCP Node 0 50-06-0e-80-00-10-04-92 50-06-0e-80-00-10-04-92
      Automapped FCP Node 1 50-06-0e-80-00-10-04-90 50-06-0e-80-00-10-04-90

      Press the Enter or Return key to continue:
      Bind all auto-mapped targets? (Y/N) => Y <Return>

      Select Binding Method:
      1. By Node Name
      2. By Port Name
      3. By D_ID
      0. Cancel
      Enter Choice => 2 <Return>
      :
  10. Setup redundant system

    Execute the following commands to create a new redundant system.

    The following procedure is described for MPHD/GRMPHD. If you use the redundancy software other than MPHD/GRMPHD, see the document for each product.

    In case of MPHD, the commands need to be executed for each LUN, specifying the path name determined in 9.b.3 as a parameter.

    The italic letters below are the new multipath names, and the underlined letters are the path names that compose the multipath.

    MPHD
    # /usr/opt/FJSViomp/bin/iompadm -c mphd new /dev/rdsk/c3t3d0s2 /dev/rdsk/c4t3d0s2 <Return>
    IOMP: /dev/FJSVmphd/fiomp/adm2
    Node:
    /dev/FJSVmphd/rdsk/mphd2s0
    /dev/FJSVmphd/rdsk/mphd2s1
    /dev/FJSVmphd/rdsk/mphd2s2
    /dev/FJSVmphd/rdsk/mphd2s3
    /dev/FJSVmphd/rdsk/mphd2s4
    /dev/FJSVmphd/rdsk/mphd2s5
    /dev/FJSVmphd/rdsk/mphd2s6
    /dev/FJSVmphd/rdsk/mphd2s7

    GRMPD
    # /usr/sbin/mplbconfig -nv <Return>
    *** Phase 1: checking /dev/rdsk ***
    *** Phase 2: checking multi path ***
    *** Phase 3: analyzing path ***
    Path : Action : Element path : LUN : Storage : Zone
    mplb0 : exist : c1t4d0s2 c2t3d0s2 : 0 : GR720464f-12340001 :
    :
    mplb8 : new : c3t3d0s2 c4t3d0s2 : 0 : GR8404550- 000213 :
    :
    # /usr/sbin/mplbconfig -av <Return>
    :
    IOMP: /dev/FJSVmplb/fiomp/adm8
    Node:
    /dev/FJSVmplb/rdsk/mplb8s0
    /dev/FJSVmplb/rdsk/mplb8s1
    /dev/FJSVmplb/rdsk/mplb8s2
    /dev/FJSVmplb/rdsk/mplb8s3
    /dev/FJSVmplb/rdsk/mplb8s4
    /dev/FJSVmplb/rdsk/mplb8s5
    /dev/FJSVmplb/rdsk/mplb8s6
    /dev/FJSVmplb/rdsk/mplb8s7
    :
  11. Setup applications

    Add configuration for the application using the logical path name determined in 9.

    [ filesystem operation ]

    Create file system.

    To mount at system start, add an entry in the /etc/vfstab file.

    # newfs /dev/FJSVmphd/rdsk/mphd2s6 <Return>
    # vi /etc/vfstab <Return>
    :
    /dev/FJSVmphd/dsk/mphd2s6 /dev/FJSVmphd/rdsk/mphd2s6 /mnt ufs 1 yes -
    :

    [ raw access operation ]

    Configure applications.

    For details, see the manual of each application.

    [ swap device ]

    Add swap device.

    To make valid at system start, add an entry in the /etc/vfstab file.

    # swap -a /dev/FJSVmphd/dsk/mphd2s4 <Return>
    # vi /etc/vfstab <Return>
    :
    /dev/FJSVmphd/dsk/mphd2s4 - - swap - no -
    :
    # swap -l <Return>
    swapfile dev swaplo blocks free
    :
    /dev/FJSVmphd/dsk/mphd2s4 32,164 16 788384 683680
  12. Start applications

    Use the logical path name determined in 9. to start applications.

    [ filesystem operation ]

    Mount filesystem

    # mount -F ufs /dev/FJSVmphd/dsk/mphd2s6 /mnt <Return>

    [ raw access operation ]

    Start applications.

    For details, see the manual of each application.

    [ swap device ]

    This procedure is not required.

3.2.1.3 Extension of PCI cards to redundant system 

The procedure to add PCI cards to a redundant system in order to extend redundancy on a Solaris 8 OE system is explained with an example with GR multipath driver (GRMPD).

There are no additional patterns for Multipath Disk Control (MPHD).

If multipath software products other than GRMPD are being used, see the manual of each product.

No configuration is needed for applications when adding PCI cards to an existing redundant system.

  1. Setup devices which will be added to system

    This is an example where a disk array controller of target=3, lun=0 is added with pre-definition to an existing disk array device. Extension using PCI Hot Plug cannot be done without definition, so devices must be defined before the system goes into operation.

    [ If you use Solaris OE:sd ]

    If this is the first time to pre-define a disk array device, begin with procedure 1.a. If this is not the first time to pre-define a disk array device, go to step 2.

    1. Modify /kernel/drv/sd.conf

      Ensure target=3,lun=0 target=3,lun=1 target=3,lun=3 are defined in /kernel/drv/sd.conf.

      If they are not defined or commented, modify /kernel/drv/sd.conf.

      # vi /kernel/drv/sd.conf <Return>
      :name="sd" class="scsi"
      target=0 lun=0;
      :
      name="sd" class="scsi"
      target=3 lun=0;
      name="sd" class="scsi"
      target=3 lun=1;
      name="sd" class="scsi"
      target=3 lun=2;
      :
    2. Boot the system and reconfigure the kernel.
      # touch /reconfigure <Return>
      # /usr/sbin/shutdown -y -i6 -g0 <Return>

    [ If you use hddv ]

    If this is the first time to pre-define a disk array device, begin with procedure 1.a. If this is not the first time to pre-define a disk array device, begin with procedure 1.e.

    1. Check if /kernel/drv/hddv.conf exists

      Check if /kernel/drv/hddv.conf exists. If it exists, the hard disk driver is being installed, so start from procedure 1.c. If it does not exist, the system is being operated with none-active extension procedures, so start from procedure 1.b.

      Not exist(new install) [ to procedure 1.c.]
      # ls /kernel/drv/hddv.conf <Return>
      /kernel/drv/hddv.conf: No such file or directory

      Exists(operation with non-active extension procedures) [ to procedure 1.b.]
      # ls /kernel/drv/hddv.conf <Return>
      /kernel/drv/hddv.conf
    2. When operating with non-active extension procedures, execute the following commands and confirm that /kernel/drv/hddv.conf is deleted.
      # hddvadm -i <Return>
      # ls /kernel/drv/hddv.conf <Return>
      /kernel/drv/hddv.conf: No such file or directory
    3. Use the following command to create hddv.conf file for the pre-defined extension function. The message examples for "operation with non-active extension procedures" means that disk array device GR720 currently used with taget=1(lun=0) is registered twice on the sd/hddv driver.
      New installation [ to procedure 1.e.]
      # hddvadm -M <Return>

      Operation with non-active extension procedures [ to procedure 1.d.]
      # hddvadm -M <Return>
      already ID= 1 exists. /devices/pci@1d,2000/fibre-channel@1/sd@1,0:a,raw : GR720
    4. When performing pre-defined extension, the same target number cannot be registered in sd.conf and hddv.conf, so delete or comment out the conflicting definition parts in sd.conf file when definition in the two files conflict with each other.

      The following example shows the case that target=1 is repeated.

      # vi /kernel/drv/sd.conf <Return>
      :
      name="sd" class="scsi"
      target=0 lun=0;
      :
      #name="sd" class="scsi"
      target=1 lun=0;
      #name="sd" class="scsi" Delete or comment out
      target=1 lun=1; conflicting definition parts in sd/hddv
      #name="sd" class="scsi"
      target=1 lun=2;
      :

      A target number that is already used by the sd driver of the system disk or CD-ROM cannot be deleted. In this case, change the target number in hddv.conf and of the disk array device so that it does not conflict with sd.conf.

    5. Edit the /kernel/drv/hddv.conf file created automatically by the hddvadm -M command, and remove comments of the LUN definition to pre-define and the LUN definition currently being used with non-active extension procedures.

      If case active extension has been performed before and the LUN definition to extend the next procedures 1.f. - 1.g. are not needed.

      If the target number of each disk array controller of a redundant system differs, add definition for each target number.

      The following example shows target=3 , lun=0~2 for pre-defined extension and disk array device GR720 (target=1) in use with non-active extension procedures being registered.

      # ls /kernel/drv/hddv.conf <Return>
      /kernel/drv/hddv.conf
      # vi /kernel/drv/hddv.conf <Return>
      :
      #name="hddv" class="scsi" target=0 lun=0;
      :
      #name="hddv" class="scsi" target=0 lun=7;
      :
      name="hddv" class="scsi" target=1 lun=0;
      name="hddv" class="scsi" target=1 lun=1; Remove comments of the LUN in use
      name="hddv" class="scsi" target=1 lun=2;
      :
      name="hddv" class="scsi" target=3 lun=0;
      name="hddv" class="scsi" target=3 lun=1; Remove comments of extended LUN
      name="hddv" class="scsi" target=3 lun=2;
      #name="hddv" class="scsi" target=3 lun=3;
      :
    6. Check the LUN definition overlap between sd.conf and hddv.conf by executing hddvchk_conf command.

      The message example for "Check NG" means that target=3 is defined in both files,

      and the procedures must be performed again from procedure 1.d.

      The followings are examples of execution of the FJSVhddv package installation to /opt. The path of the hddvchk_conf command depends on the installed directory.

      Check OK [ to procedure 1.g.]
      # /opt/FJSVhddv/bin/hddvchk_conf <Return>
      /kernel/drv/sd.conf and /kernel/drv/hddv.conf seems to be correct

      Check NG [ to procedure 1.d.]
      # /opt/FJSVhddv/bin/hddvchk_conf <Return>
      NG ID = [ 3 ]
    7. Reconfigure the kernel , restart the system, and then start system operation.
      # touch /reconfigure <Return>
      # /usr/sbin/shutdown -y -i6 -g0 <Return>
  2. Stop the machine administration hardware monitoring daemon

    Execute the following commands to stop the machine administration hardware monitoring daemon.

    # /usr/sbin/FJSVmadm/prephp <Return>
  3. If you use the Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*), execute the following.

    The daemons will be stopped.

    # /etc/rc0.d/K10ElxRMSrv stop <Return>
    # /etc/rc0.d/K10ElxDiscSrv stop <Return>
  4. Add PCI cards

    Add PCI cards according to the following procedure.

    [ for PCI Fibre Channel(PW008FC3U/PW008FC2U/ GP7B8FC1U)]:

    If you use SAN management function of Systemwalker StorageMGR/Softek SANView for ETERNUS (except for Vixel) /SP5000 SRM Facility

    No procedure is necessary. Go to step 5.

    If you don't use SAN management function of the above products

    To add PCI cards with the following configurations, Fibre Channel switch SN200 series and disk array device ETERNUS3000/ETERNUS6000/GR700/800 series need to be reconfigured individually.

    -If zone configuration by WWPN(World Wide Port Name) is done on SN200 series.

    -If the Host Affinity function of ETERNUS3000/ETERNUS6000/GR700/800 series is used.

    For details, see "SN200 Series Affinity User's Guide" or "ETERNUS3000/ETERNUS6000/GR700/800 series GRmgr User's Guide." If you use the Fibre Channel switch or disk array device other than described above, see the document of each product.

    To perform the above reconfiguration, the WWPN(a 16-digit number) of the replacement card is needed. The WWPN of the PCI card can be known from the eight characters shown on a label on the front plate of the card. These characters represent the bottom eight digit of the WWPN in hexadecimal form. The top eight digit are fixed to 10000000 in hexadecimal form.

    For example, if the following label is shown on the front plate of the card, the WWPN of the replacement card is 100000000e244061.

    0e24
    4061

    [ for Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*)]:

    To replace PCI cards with the following configurations, Fibre Channel switch and disk array device need to be reconfigured individually.

    -If zone configuration by WWPN (World Wide Port Name) is done on Fibre Channel switch.

    -If the Host Zoning function of disk array device is used.

    For details, see the document of each product.

    To perform the above reconfiguration, the WWPN(a 16-digit number) of the replacement card is needed. The WWPN of the PCI card can be known from the twelve characters shown on a label on the back of the card. These characters represent the bottom twelve digit of the WWPN in hexadecimal form. The top four digit are fixed to 1000 in hexadecimal form.

    For example, if the following label is shown on the back of the card, the WWPN of the replacement card is 10000000c9366037.

    IEEE:0000c9366037

    Note:

    When changing Affinity configuration on SN200 series or other Fibre Channel switch, I/O to other devices is effected by the change, and may result in temporal errors.

    I/O to disk array devices recovers normally because of retry processes, but on Fibre Channel tape devices, backup processes may end in errors. Stop backup before changing Affinity configuration.

    1. Before extension of PCI cards, save the current execution result of the prtpicl(1M) command (current configuration at present.)
      # prtpicl -v > /tmp/prtpicl.pre <Return>
    2. Check the status of the slot.

      When adding PCI cards to a redundant configuration, several PCI cards may be added to the system. In this case, perform to procedures 4. on each card.

      The relationship of the position of a PCI card and format of "Ap_Id" is shown below.

      Location of PCI slot Definition of slot Format of "Ap_Id"
      Inside chassis of
      PRIMEPOWER
      900/1500/2500
      Chassis No.: X
      System board No.: Y
      Physical slot No.: ZZ
      • Solaris 8 OE and Solaris 9 OE
        pcipsyN:CXM0Y-PCI#slotZZ
      • Solaris 10 OE
        CXM0Y-PCI#ZZ
      PCI/Disk BOX I/O Chassis No.: X
      PCI/Disk BOX No.: Y
      Physical slot No.: ZZ
      • Solaris 8 OE and Solaris 9 OE
        pcipsyN:RXB0Y-PCI#slotZZ
      • Solaris 10 OE
        RXB0Y-PCI#ZZ
      Other Physical slot No.: ZZ
      • Solaris 8 OE and Solaris 9 OE
        pcipsyN:PCI#slotZZ
      • Solaris 10 OE
        PCI#ZZ

      Below, an example of adding a PCI card to chassis number : 0, PCI/Disk BOX number : 1, physical slot number : 02, slot in PCI/Disk BOX(R0B01-PCI#slot02) is explained.

      Check the status of the target slot is "empty unconfigured."

      # cfgadm | grep R0B01-PCI#slot02 <Return>
      pcipsy18:R0B01-PCI#slot02 unknown empty unconfigured unknown
    3. To confirm the slot position at extension operation, blink the ALARM LED of Ap_Id displayed in procedure 4.b.
      # cfgadm -x led=fault,mode=blink pcipsy18:R0B01-PCI#slot02 <Return>
    4. After checking that the READY LED on the PCI slot to add the PCI card is off, add PCI cards and connect cable to devices. This operation is performed by our customer support.
    5. Use the cfgadm(1M) command to check that the slot status of the added PCI card changed to "disconnected unconfigured".
      # cfgadm pcipsy18:R0B01-PCI#slot02 <Return>
      Ap_Id Type Receptacle Occupant Condition
      pcipsy18:R0B01-PCI#slot02 unknown disconnected unconfigured unknown
  5. Connect PCI cards

    Connect the replaced PCI card using the cfgadm(1M) command with the configure option, or by pushing the button corresponding to the extension slot position. Note that the push button is only effective in multi-user mode. After the new PCI card is connected, use the cfgadm(1M) command and confirm that the slot status has changed to "connected configured."

    If a large-scale configuration of I/O devices is connected to the PCI card in the target slot, command execution for status confirmation may take time.

    # cfgadm -c configure pcipsy18:R0B01-PCI#slot02 <Return>
    # cfgadm pcipsy18:R0B01-PCI#slot02 <Return>
    Ap_Id Type Receptacle Occupant Condition
    pcipsy18:R0B01-PCI#slot02 mult/hp connected configured ok
  6. If you use the Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*), execute the following.

    The daemons will be started.

    # /etc/rc2.d/S99ElxRMSrv start <Return>
    # /etc/rc2.d/S99ElxDiscSrv start <Return>
  7. Update hardware configuration information of machine administration/ Start the hardware monitoring daemon.

    Execute the following commands to update hardware configuration information of machine administration and to restart the hardware monitoring daemon.

    # /usr/sbin/FJSVmadm/postphp <Return>
  8. Preparation to recognize devices for Fibre Channel

    When the Host Affinity function is being used with Fibre Channel SN200 series, Host Affinity must be configured for the device added.

    [ for PCI Fibre Channel(PW008FC3U/PW008FC2U/ GP7B8FC1U)]:

    If you use SAN management function of Systemwalker StorageMGR/Softek SANView for ETERNUS (except for Vixel) /SP5000 SRM Facility

    WWPN zoning of SN200 series, Host affinity of Array disk device ETERNUS3000/ETERNUS6000/GR700/800 series and binding setting for fjpfca.conf/sd.conf of host can be added from SAN management window automatically.
    See "Operation guide" or "User's Manual" of each product for details.
    Before using this function, the following procedure 9.a.1. and 9.a.2. [except fcp-bind-target setting] are necessary.
    Moreover, begin from procedure 9.a.3. after executing this function.

    If you don't use SAN management function of the above products

    No procedure is necessary. Go to step 9.

    [ for Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*)]:

    No procedure is necessary. Go to step 9

    Note:

    When changing Affinity configuration on SN200 series, I/O to other devices is effected by the change, and may result in temporal errors.

    I/O to disk array devices recovers normally because of retry processes, but on Fibre Channel tape devices, backup processes may end in errors. Stop backup before changing Affinity configuration.

  9. Set up driver

    Add the configuration of driver according to the following operation.

    As for file related drivers, additional configuration of the PCI card driver and the device driver is needed.

    1. Add configurations of PCI card driver

      [ SCSI driver(Solaris OE: glm , FUJITSU PCI SCSI Adapter Driver) ]

      Configuration is not required.

      [ Fibre Channel driver (PW008FC2U/ W008FC2X/GP7B8FC1U/GP7B8FC1X) ]

      1. After saving execution result of prtpicl (1M) command, take the difference between the result and the information obtained in procedure 4.a. to calculate the instance number of the fjpfca driver of the PCI card added.

        In the following example, instance number is 1, indicating that the interface name of the PCI card added is fjpfca1.

        Making the LED of the PCI card blink using the fc_adm command can specify the relation between the instance number and the slot position. The relation between the WWPN and instance number of the PCI card added can be known from the "fc_info -a" command.

        # prtpicl -v > /tmp/prtpicl.pfca <Return>
        # diff /tmp/prtpicl.pre /tmp/prtpicl.pfca | more <Return>
        :
        > :status okay
        > :devfs-path /pci@8d,2000/fibre-channel@1
        > :driver-name fjpfca
        > :binding-name pci10cf,105c
        > :bus-addr 1
        > :instance 1
        > :_class obp-device
        > :name fibre-channel
        :# /usr/sbin/FJSVpfca/fc_adm -l fjpfca1 <Return>
      2. Edit /kernel/drv/fjpfca.conf and add the following property definitions to the instance number from 9.a.1.

        port (topology classification , link speed)

        Define this property when PCI cards using Fabric connection are added.

        This definition is not needed with FC-AL connection.

        fcp-bind-target(connection of target ID and WWN)

        Define this property when disk array equipment using Fabric connections are added.

        This definition is not needed with FC-AL connection.

        max-throttle(number of maximum SCSI command issue)

        This property is defined for restriction on simultaneous issue of SCSI commands.

        The necessity and value of this definition depends on the specification of the disk array device, so refer to the user's manual of each equipment.

        The following example shows the definition of a fjpfca1 PCI card (instance number : 1) and disk array equipment( target = 3) being added.

        # vi /kernel/drv/fjpfca.conf <Return>
        :
        port=
        "fjpfca0:nport",
        "fjpfca1:nport:sp1"; <- added

        fcp-bind-target=
        "fjpfca0t0:0x50000e1000002501",
        "fjpfca1t3:0x50000e1000002ab0"; <- added

        max-throttle=
        "fjpfca0t0:128",
        "fjpfca1t3:240"; <- added
        :
      3. Execute the following command to make the fjpfca driver take in the added part.
        # /usr/sbin/FJSVpfca/fc_config -v <Return>

      [ Fibre Channel driver (PW028FC3*/PW028FC4*/PW028FC5*) ]

      Configuration is not required.

      Compare the output of prtpicl(1M) command with the output saved at 4.b and find the instance number of the added card.

    2. Add device driver configuration

      Add device driver configuration with the following procedures according to the I/O device type.

      [ Disk/Tape devices(Solaris OE : sd/st) ]

      This example is a case where a disk device of target=2, lun=0 is pre-defined in procedure 1. and added in procedure 5. If the device is a tape device, re-read "sd" as "st" in this procedure and perform the same operation.

      1. Execute the following command to make the sd driver set up the added disk device.
        # /usr/sbin/devfsadm <Return>
      2. After saving the execution result of the prtpicl(1M) command, take the difference between the result and the information obtained in 4.a. and get the physical path name of additional disk device, then get the logical path name under /dev/dsk.

        In the following example, "c2t2d0" is the logical path name of the disk device added.

        # prtpicl -v > /tmp/prtpicl.post <Return>

        Disk device
        # diff /tmp/prtpicl.pre /tmp/prtpicl.post | grep devfs-path | grep sd <Return>
        > :devfs-path /pci@89,4000/scsi@2/sd@2,0
        # ls -l /dev/dsk | grep /pci@89,4000/scsi@2/sd@2,0 <Return>
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t2d0s0
        -> ../../devices/pci@89,4000/scsi@2/sd@2,0:a
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t2d0s1
        -> ../../devices/pci@89,4000/scsi@2/sd@2,0:b
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t2d0s2
        -> ../../devices/pci@89,4000/scsi@2/sd@2,0:c
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t2d0s3
        -> ../../devices/pci@89,4000/scsi@2/sd@2,0:d
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t2d0s4
        -> ../../devices/pci@89,4000/scsi@2/sd@2,0:e
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t2d0s5
        -> ../../devices/pci@89,4000/scsi@2/sd@2,0:f
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t2d0s6
        -> ../../devices/pci@89,4000/scsi@2/sd@2,0:g
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t2d0s7
        -> ../../devices/pci@89,4000/scsi@2/sd@2,0:h

        Tape device
        # diff /tmp/prtpicl.pre /tmp/prtpicl.post | grep devfs-path | grep st <Return>
        > :devfs-path /pci@89,4000/scsi@2/st@5,0
        # ls -l /dev/rmt | grep /pci@89,4000/scsi@2/st@5,0 <Return>
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 1
        -> ../../devices/pci@89,4000/scsi@2/st@5,0:
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 1b
        -> ../../devices/pci@89,4000/scsi@2/st@5,0:b
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 1bn
        -> ../../devices/pci@89,4000/scsi@2/st@5,0:bn
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 1c
        -> ../../devices/pci@89,4000/scsi@2/st@5,0:c
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 1cb
        -> ../../devices/pci@89,4000/scsi@2/st@5,0:cb
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 1cbn
        -> ../../devices/pci@89,4000/scsi@2/st@5,0:cbn
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 1cn-> ../../devices/pci@89,4000/scsi@2/st@5,0:cn
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 1h
        -> ../../devices/pci@89,4000/scsi@2/st@5,0:h
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 1hb
        -> ../../devices/pci@89,4000/scsi@2/st@5,0:hb
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 1hbn
        -> ../../devices/pci@89,4000/scsi@2/st@5,0:hbn
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 1hn
        -> ../../devices/pci@89,4000/scsi@2/st@5,0:hn
      3. If the device is a disk device, start the format(1M) command and configure labeling and partition(slice) information of the logical path name of added disks.

        If the device is a tape device, this procedure is not needed.

      [ Disk array device (hard disk driver : hddv) ]

      This example is a case where a disk array device of target=3, lun=0-2 is pre-defined in procedure 1. and added in procedure 5.

      1. There may be a change of status depending on the system operating state after pre-definition in procedure 1., so check the LUN definition conflict between sd.conf and hddv.conf again. In case of "Check NG", the definitions are conflicting, so go back to procedure 1.e.
        Check OK [ to procedure 9.b.2.]
        # /opt/FJSVhddv/bin/hddvchk_conf <Return>
        /kernel/drv/sd.conf and /kernel/drv/hddv.conf seems to be correct

        Check NG [ to procedure 1.e.]
        # /opt/FJSVhddv/bin/hddvchk_conf <Return>
        NG ID = [ 3 ]
      2. Execute the following command to make the hard disk driver set up the LUN.
        # /usr/sbin/hddvadmdr <Return>
      3. After saving execution result of the prtpicl command, get the physical path name of added LUN by taking the difference between the result and the information obtained in 4.a, and calculate the corresponding logical path name under /dev/dsk

        In the following example, "c3t3d0" is logical path name of added lun=0. The logical path name of lun=1,2 is set to "c3t3d1", "c3t3d2" respectively, because cXtYdZmeans X: PCI card number, Y: target number, Z: lun number .

        # prtpicl -v > /tmp/prtpicl.post <Return>
        # diff /tmp/prtpicl.pre /tmp/prtpicl.post | grep devfs-path | grep hddv <Return>
        > :devfs-path /pci@89,2000/fibre-channel@1/hddv@3,0
        > :devfs-path /pci@89,2000/fibre-channel@1/hddv@3,1
        > :devfs-path /pci@89,2000/fibre-channel@1/hddv@3,2
        # ls -l /dev/dsk | grep /pci@89,2000/fibre-channel@1/hddv@3,0 <Return>
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s0
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:a
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s1
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:b
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s2
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:c
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s3
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:d
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s4
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:e
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s5
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:f
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s6
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:g
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s7
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:h
      4. Start the format(1M) command and configure labeling and partition(slice) information calculated in 9.b.3. to the logical path name of each extended LUN.
    3. Add configure of PCI card driver

      [ SCSI driver(Solaris OE: glm , FUJITSU PCI SCSI Adapter Driver) ]

      No procedure is necessary. Go to step 10.

      [ Fibre Channel driver (PW008FC2U/ W008FC2X/GP7B8FC1U/GP7B8FC1X) ]

      No procedure is necessary. Go to step 10.

      [ Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*)]

      Bind the auto mapped target connected with the card and modify the entries of lpfc.conf to match the requirements of the extended Fibre Channel Controller. These procedures are reflected on next boot time. In the following, indicate to bind the auto mapped target with the card.

      # /usr/sbin/lpfc/lputil <Return>
      LightPulse Common Utility for Solaris/SPARC. Version 1.6a8 (03/03/2004).
      Copyright (c) 2003, Emulex Network Systems, Inc.

      Emulex Fibre Channel Host Adapters Detected: 2
      Host Adapter 0 (lpfc11) is an LP9802 (Ready Mode)
      Host Adapter 1 (lpfc9) is an LP9802 (Ready Mode)

      MAIN MENU
      1. List Adapters
      2. Adapter Information
      3. Firmware Maintenance
      4. Reset Adapter
      5. Persistent Bindings
      0. Exit
      Enter choice => 5 <Return>

      PERSISTENT BINDINGS MENU
      1. Display Current Bindings
      2. Display All Nodes
      3. Duplicate Adapter Bindings
      4. Bind Target Manually
      5. Bind Automapped Targets
      6. Delete Binding(s)
      0. Return to Main Menu
      Enter choice => 5 <Return>
      0. lpfc11
      1. lpfc9
      Select an adapter => 0 <Return>

      Automapped FCP Nodes:
      Node Target WWPN WWNN
      ---- ------ --------- --------
      Automapped FCP Node 0 50-06-0e-80-00-10-04-92 50-06-0e-80-00-10-04-92
      Automapped FCP Node 1 50-06-0e-80-00-10-04-90 50-06-0e-80-00-10-04-90

      Press the Enter or Return key to continue:
      Bind all auto-mapped targets? (Y/N) => Y <Return>

      Select Binding Method:
      1. By Node Name
      2. By Port Name
      3. By D_ID
      0. Cancel
      Enter Choice => 2 <Return>
      :
  10. Connect to redundant system

    The following procedure is described for MPHD/GRMPHD. If you use the redundancy software other than MPHD/GRMPHD, see the document for each product.

    To connect the added PCI card to the redundant system, execute the following command and confirm that the path name determined in procedure 9.b.3. (in underlined letters) is added.

    # /usr/sbin/mplbconfig -nv <Return>
    *** Phase 1: checking /dev/rdsk ***
    *** Phase 2: checking multi path ***
    *** Phase 3: analyzing path ***
    Path : Action : Element path : LUN :Storage : Zone
    mplb0 : exist : c1t4d0s2 c2t3d0s2 : 0 : GR720464f-12340001 :
    :
    mplb8 : add : c3t3d0s2 c4t3d0s2 c5t3d0s2: 0 : GR8404550- 000213 :
    :
    # /usr/sbin/mplbconfig -av <Return>
    :
    IOMP: /dev/FJSVmplb/fiomp/adm8
    Node:
    /dev/FJSVmplb/rdsk/mplb8s0
    /dev/FJSVmplb/rdsk/mplb8s1
    /dev/FJSVmplb/rdsk/mplb8s2
    /dev/FJSVmplb/rdsk/mplb8s3
    /dev/FJSVmplb/rdsk/mplb8s4
    /dev/FJSVmplb/rdsk/mplb8s5
    /dev/FJSVmplb/rdsk/mplb8s6
    /dev/FJSVmplb/rdsk/mplb8s7
    :

3.2.2 Solaris 9 OE or later 

This section describes the procedures to extend PCI cards on Solaris 9 OE or later systems.

If you use Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*), ensure the automap function is effective. Boot the system after modifying lpfc.conf.

# vi /kernel/drv/lpfc.conf <Return>

:
# If automap is set, SCSI IDs for all FCP nodes without
# persistent bindings will be automatically generated.
# If new FCP devices are added to the network when the system is down,
# there is no guarantee that these SCSI IDs will remain the same
# when the system is booted again.
# The bind method of the port is used as the binding method of
# automap devices to preserve SCSI IDs between link down and link up.
# If automap is 0, only devices with persistent bindings will be
# recognized by the system.
automap=1; <-- set to 1
:

3.2.2.1 Extension of PCI cards on non-redundant system 

Procedures to add PCI cards without making them redundant on a Solaris 9 OE or later system are explained here.

  1. Stop the machine administration hardware monitoring daemon Execute the following commands to stop the machine administration hardware error monitoring daemon.
    # /usr/sbin/FJSVmadm/prephp <Return>

  2. If you use the Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*), execute the following. The daemons will be stopped.
    # /etc/rc0.d/K10ElxRMSrv stop <Return>
    # /etc/rc0.d/K10ElxDiscSrv stop <Return>

  3. Add PCI cards

    Add PCI cards according to the following procedure.

    [ for PCI Fibre Channel(PW008FC3U/PW008FC2U/ GP7B8FC1U)]:

    If you use SAN management function of Systemwalker StorageMGR/Softek SANView for ETERNUS (except for Vixel) /SP5000 SRM Facility

    No procedure is necessary. Go to step 7.

    If you don't use SAN management function of the above products

    To add PCI cards with the following configurations, Fibre Channel switch SN200 series and disk array device ETERNUS3000/ETERNUS6000/GR700/800 series need to be reconfigured individually.

    -If zone configuration by WWPN(World Wide Port Name) is done on SN200 series.

    -If the Host Affinity function of ETERNUS3000/ETERNUS6000/GR700/800 series is used.

    For details, see "SN200 Series Affinity User's Guide" or "ETERNUS3000/ETERNUS6000/GR700/800 series GRmgr User's Guide." If you use the Fibre Channel switch or disk array device other than described above, see the document of each product.

    To perform the above reconfiguration, the WWPN(a 16-digit number) of the replacement card is needed. The WWPN of the PCI card can be known from the eight characters shown on a label on the front plate of the card. These characters represent the bottom eight digit of the WWPN in hexadecimal form. The top eight digit are fixed to 10000000 in hexadecimal form.

    For example, if the following label is shown on the front plate of the card, the WWPN of the replacement card is 100000000e244061.

    0e24
    4061

    [ for Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*)]:

    To replace PCI cards with the following configurations, Fibre Channel switch and disk array device need to be reconfigured individually.

    -If zone configuration by WWPN (World Wide Port Name) is done on Fibre Channel switch.

    -If the Host Zoning function of disk array device is used.

    For details, see the document of each product.

    To perform the above reconfiguration, the WWPN(a 16-digit number) of the replacement card is needed. The WWPN of the PCI card can be known from the twelve characters shown on a label on the back of the card. These characters represent the bottom twelve digit of the WWPN in hexadecimal form. The top four digit are fixed to 1000 in hexadecimal form.

    For example, if the following label is shown on the back of the card, the WWPN of the replacement card is 10000000c9366037.

    IEEE:0000c9366037

    Note:

    When changing Affinity configuration on SN200 series or other Fibre Channel switch, I/O to other devices is effected by the change, and may result in temporal errors.

    I/O to disk array devices recovers normally because of retry processes, but on Fibre Channel tape devices, backup processes may end in errors. Stop backup before changing Affinity configuration.

    1. Before extension of PCI cards, save the current execution result of prtpicl(1M) command (composition information at present)

      # prtpicl -v > /tmp/prtpicl.pre <Return>

    2. Check the status of the slot.

      The relationship of the position of a PCI card and format of "Ap_Id" is shown below.

      Location of PCI slot

      Definition of slot

      Format of "Ap_Id"

      Inside chassis of PRIMEPOWER 900/1500/2500

      Chassis No.: X
      System board No.: Y
      Physical slot No.: ZZ

      • Solaris 8 OE and Solaris 9 OE
        pcipsyN:CXM0Y-PCI#slotZZ
      • Solaris 10 OE
        CXM0Y-PCI#ZZ

      PCI/Disk BOX

      I/O Chassis No.: X
      PCI/Disk BOX No.: Y
      Physical slot No.: ZZ

      • Solaris 8 OE and Solaris 9 OE
        pcipsyN:RXB0Y-PCI#slotZZ
      • Solaris 10 OE
        RXB0Y-PCI#ZZ

      Other

      Physical slot No.: ZZ

      • Solaris 8 OE and Solaris 9 OE
        pcipsyN:PCI#slotZZ
      • Solaris 10 OE
        PCI#ZZ

      Below, an example of adding a PCI card to chassis number : 0, system board number : 1, physical slot number : 02,slot in chassis(C0M01-PCI#slot02) is explained.

      Check the status of the target slot is "empty unconfigured."

      # cfgadm | grep C0M01-PCI#slot02 <Return>
      pcipsy15:C0M01-PCI#slot02 unknown empty unconfigured unknown

    3. To confirm the slot position at extension operation, blink the ALARM LED of Ap_Id displayed in procedure 3.b.

      # cfgadm -x led=fault,mode=blink pcipsy15:C0M01-PCI#slot02 <Return>

    4. After checking that the READY LED on the PCI slot to add the PCI card is off, add PCI cards and connect cable to devices. This operation is performed by our customer support.

    5. Use the cfgadm(1M) command to check that the slot status of the added PCI card changed to "disconnected unconfigured".

      # cfgadm pcipsy15:C0M01-PCI#slot02 <Return>
      Ap_Id Type Receptacle Occupant Condition
      pcipsy15:C0M01-PCI#slot02 unknown disconnected unconfigured unknown

  4. Connect PCI cards

    Connect the replaced PCI card using the cfgadm(1M) command with the configure option, or by pushing the button corresponding to the extension slot position. Note that the push button is only effective in multiuser mode. After the new PCI card is connected, use the cfgadm(1M) command and confirm that the slot status has changed to "connected configured."

    If a large-scale configuration of I/O devices is connected to the PCI card in the target slot, command execution for status confirmation may take time.

    # cfgadm -c configure pcipsy15:C0M01-PCI#slot02 <Return>
    # cfgadm pcipsy15:C0M01-PCI#slot02 <Return>
    Ap_Id Type Receptacle Occupant Condition
    pcipsy15:C0M01-PCI#slot02 mult/hp connected configured ok

  5. If you use the Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*), execute the following.

    The daemons will be started.

    # /etc/rc2.d/S99ElxRMSrv start <Return>
    # /etc/rc2.d/S99ElxDiscSrv start <Return>

  6. Update hardware configuration information of machine administration/ Start the hardware monitoring daemon.

    Execute the following commands to update hardware configuration information of machine administration and to restart the hardware monitoring daemon.

    # /usr/sbin/FJSVmadm/postphp <Return>

  7. Preparation to recognize devices for Fibre Channel

    When the Host Affinity function is being used with Fibre Channel SN200 series, Host Affinity must be configured for the device added.

    [ for PCI Fibre Channel(PW008FC3U/PW008FC2U/ GP7B8FC1U)]:

    If you use SAN management function of Systemwalker StorageMGR/Softek SANView for ETERNUS (except for Vixel) /SP5000 SRM Facility

    WWPN zoning of SN200 series, Host affinity of Array disk device ETERNUS3000/ETERNUS6000/GR700/800 series and binding setting for fjpfca.conf/sd.conf of host can be added from SAN management window automatically.
    See "Operation guide" or "User's Manual" of each product for details.
    Before using this function, the following procedure 8.a.1. and 8.a.2. [except fcp-bind-target setting] are necessary.
    Moreover, begin from procedure 8.a.3. after executing this function.

    If you don't use SAN management function of the above products

    No procedure is necessary. Go to step 8.

    [ for Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*)]:

    No procedure is necessary. Go to step 8.

    Note:

    When changing Affinity configuration on SN200 series, I/O to other devices is effected by the change, and may result in temporal errors.

    I/O to disk array devices recovers normally because of retry processes, but on Fibre Channel tape devices, backup processes may end in errors. Stop backup before changing Affinity configuration.

  8. Set up driver

    Add the configuration of driver according to the following operation.

    As for file related drivers, additional configuration of the PCI card driver and the device driver is needed.

    1. Add the configuration of PCI cards driver

      [ SCSI driver (Solaris OE: glm , FUJITSU PCI SCSI Adapter Driver) ]

      Configuration is not required.

      [Fibre Channel driver (PW008FC3U/PW008FC2U/ GP7B8FC1U) ]

      1. After saving execution result of prtpicl (1M) command, take the difference between the result and the information obtained in procedure 3.a. to calculate the instance number of the fjpfca driver of the PCI card added. In the following example, instance number is 1, indicating that the interface name of the PCI card added is fjpfca1. Making the LED of the PCI card blink using the fc_adm command can specify the relation between the instance number and the slot position. The relation between the WWPN and instance number of the PCI card added can be known from the "fc_info -a" command

        # prtpicl -v > /tmp/prtpicl.pfca <Return>
        # diff /tmp/prtpicl.pre /tmp/prtpicl.pfca | more <Return>
        :
        > :status okay
        > :devfs-path /pci@8d,2000/fibre-channel@1
        > :driver-name fjpfca
        > :binding-name pci10cf,105c
        > :bus-addr 1
        > :instance 1
        > :_class obp-device
        > :name fibre-channel
        :
        # /usr/sbin/FJSVpfca/fc_adm -l fjpfca1 <Return>

      2. Edit /kernel/drv/fjpfca.conf and add the following property definitions to the instance number from 8.a.1.

        port (topology classification , link speed)

        Define this property when PCI cards using Fabric connection are added.

        This definition is not needed with FC-AL connection.

        fcp-bind-target(connection of target ID and WWN)

        Define this property when disk array equipment using Fabric connections are added.

        This definition is not needed with FC-AL connection.

        max-throttle(number of maximum SCSI command issue)

        This property is defined for restriction on simultaneous issue of SCSI commands.

        The necessity and value of this definition depends on the specification of the disk array device, so refer to the user's manual of each equipment.

        The following example shows the definition of a fjpfca1 PCI card (instance number : 1) and disk array equipment( target = 3) being added.

        # vi /kernel/drv/fjpfca.conf <Return>
        :
        port=
        "fjpfca0:nport",
        "fjpfca1:nport:sp1"; <- added

        fcp-bind-target=
        "fjpfca0t0:0x50000e1000002501",
        "fjpfca1t3:0x50000e1000002ab0"; <- added

        max-throttle=
        "fjpfca0t0:128",
        "fjpfca1t3:240"; <- added
        :

      3. Execute the following command to make the fjpfca driver take in the added part.

        # /usr/sbin/FJSVpfca/fc_config -v <Return>

      [ Fibre Channel driver (PW028FC3*/PW028FC4*/PW028FC5*) ]

      Configuration is not required.

      Compare the output of prtpicl(1M) command with the output saved at 3.b and find the instance number of the added card.

    2. Add device driver configuration

      Add device driver configuration with the following procedures according to the I/O device type.

      [ Disk/Tape devices(Solaris OE : sd/st) ]

      This example is a case where a disk device of target=2, lun=0 is pre-defined in procedure 1. and added in procedure 4. If the device is a tape device, re-read "sd" as "st" in this procedure and perform the same operation.

      1. See /kernel/drv/sd.conf and check that target=2 and lun=0 are defined. When this definition does not exist, add the definition. If it is commented out, remove comments.

        Disk equipment
        # vi /kernel/drv/sd.conf <Return>
        :
        name="sd" class="scsi"
        target=0 lun=0;
        :
        name="sd" class="scsi" In case the definition does not exist,
        target=2 lun=0; add or remove comments

        Tape equipment
        # vi /kernel/drv/st.conf <Return>
        :
        name="st" class="scsi"
        target=0 lun=0;
        :
        name="st" class="scsi" In case the definition does not exist,
        target=5 lun=0; add or remove comments

      2. To make the sd driver set up the disk device added, execute the following commands. When the commands are executed, the following messages are displayed but do not effect performance. Similar messages are displayed when a tape device already exists when a tape device is added, but does not effect performance.

        Disk device
        # update_drv -f sd <Return>
        Cannot unload module: sd
        Will be unloaded upon reboot.
        Forcing update of sd.conf.

        Tape device
        # update_drv -f st <Return>

      3. After saving the execution result of the prtpicl(1M) command, take the difference between the result and the information obtained in 3.a. and get the physical path name of additional disk device, then get the logical path name under /dev/dsk. In the following example, "c2t2d0" is the logical path name of the disk device added.

        # prtpicl -v > /tmp/prtpicl.post <Return>
        Disk equipment
        # diff /tmp/prtpicl.pre /tmp/prtpicl.post | grep devfs-path | grep sd <Return>
        > :devfs-path /pci@89,4000/scsi@2/sd@2,0
        # ls -l /dev/dsk | grep /pci@89,4000/scsi@2/sd@2,0 <Return>
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t2d0s0
        -> ../../devices/pci@89,4000/scsi@2/sd@2,0:a
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t2d0s1
        -> ../../devices/pci@89,4000/scsi@2/sd@2,0:b
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t2d0s2
        -> ../../devices/pci@89,4000/scsi@2/sd@2,0:c
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t2d0s3
        -> ../../devices/pci@89,4000/scsi@2/sd@2,0:d
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t2d0s4
        -> ../../devices/pci@89,4000/scsi@2/sd@2,0:e
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t2d0s5
        -> ../../devices/pci@89,4000/scsi@2/sd@2,0:f
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t2d0s6
        -> ../../devices/pci@89,4000/scsi@2/sd@2,0:g
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t2d0s7
        -> ../../devices/pci@89,4000/scsi@2/sd@2,0:h

        Tape equipment
        # diff /tmp/prtpicl.pre /tmp/prtpicl.post | grep devfs-path | grep st <Return>
        > :devfs-path /pci@89,4000/scsi@2/st@5,0
        # ls -l /dev/rmt | grep /pci@89,4000/scsi@2/st@5,0 <Return>
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 1
        -> ../../devices/pci@89,4000/scsi@2/st@5,0:
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 1b
        -> ../../devices/pci@89,4000/scsi@2/st@5,0:b
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 1bn
        -> ../../devices/pci@89,4000/scsi@2/st@5,0:bn
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 1c
        -> ../../devices/pci@89,4000/scsi@2/st@5,0:c

        lrwxrwxrwx 1 root root 41 Sep 20 22:53 1cb
        -> ../../devices/pci@89,4000/scsi@2/st@5,0:cb
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 1cbn
        -> ../../devices/pci@89,4000/scsi@2/st@5,0:cbn
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 1cn
        -> ../../devices/pci@89,4000/scsi@2/st@5,0:cn
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 1h
        -> ../../devices/pci@89,4000/scsi@2/st@5,0:h
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 1hb
        -> ../../devices/pci@89,4000/scsi@2/st@5,0:hb
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 1hbn
        -> ../../devices/pci@89,4000/scsi@2/st@5,0:hbn
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 1hn
        -> ../../devices/pci@89,4000/scsi@2/st@5,0:hn

      4. If the device is a disk device, start the format(1M) command and configure labeling and partition(slice) information of the logical path name of added disks. If the device is a tape device, this procedure is not needed.

      [ Disk array devices(hard disk driver : hddv) ]

      This is an example where an disk array device at target=3, lun=0-2 is pre-defined in procedure 4.

      If this is the first time to pre-define a disk array device, begin with procedure 8.b.1. If this is not the first time to pre-define a disk array device, begin with procedure 8.b.5.

      1. Check if /kernel/drv/hddv.conf exists

        Check if /kernel/drv/hddv.conf exists. If it exists, the hard disk driver is being installed, so start from procedure 8.b.3. If it does not exist, the system is being operated with none-active extension procedures, so start from procedure 8.b.2.

        Not exist (new install) [ to procedure 8.b.3.]
        # ls /kernel/drv/hddv.conf <Return>
        /kernel/drv/hddv.conf: No such file or directory

        Exists(operation with non-active extension procedures) [ to procedure 8.b.2.]
        # ls /kernel/drv/hddv.conf <Return>
        /kernel/drv/hddv.conf

      2. When operating with non-active extension procedures, execute the following commands and confirm that /kernel/drv/hddv.conf is deleted.

        # hddvadm -i <Return>
        # ls /kernel/drv/hddv.conf <Return>
        /kernel/drv/hddv.conf: No such file or directory

      3. Use the following command to create hddv.conf file for the active extension function. The message examples for "operation with non-active extension procedures" means that disk array device GR720 currently used with taget=1(lun=0) is registered twice on the sd/hddv driver.

        New installation [ to procedure 8.b.5.]
        # hddvadm -M <Return>

        Operation with non-active extension procedures [ to procedure 8.b.4.]
        # hddvadm -M <Return>
        already ID= 1 exists. /devices/pci@1d,2000/fibre-channel@1/sd@1,0:a,raw : GR720

      4. When performing pre-defined extension, the same target number cannot be registered in sd.conf and hddv.conf, so delete or comment out the conflicting definition parts in sd.conf file when definition in the two files conflict with each other. The following example shows the case that target=1 is repeated.

        # vi /kernel/drv/sd.conf <Return>
        :
        name="sd" class="scsi"
        target=0 lun=0;
        :
        #name="sd" class="scsi"
        target=1 lun=0;
        #name="sd" class="scsi" Delete or comment out
        target=1 lun=1; the duplication definition parts in sd/hddv
        #name="sd" class="scsi"
        target=1 lun=2;
        :

        A target number that is already used by the sd driver of the system disk or CD-ROM cannot be deleted. In this case, change the target number in hddv.conf and of the disk array device so that it does not conflict with sd.conf.

      5. Edit the /kernel/drv/hddv.conf file created automatically by the hddvadm -M command, and remove comments of the LUN definition to pre-define and the LUN definition currently being used with non-active extension procedures. If case active extension has been performed before and the LUN definition to extend the next procedures 1.f. - 1.g. are not needed. The following example shows target=3 , lun=0~2 for pre-defined extension and disk array device GR720 (target=1) in use with non-active extension procedures being registered.

        # ls /kernel/drv/hddv.conf <Return>
        /kernel/drv/hddv.conf
        # vi /kernel/drv/hddv.conf <Return>
        :
        #name="hddv" class="scsi" target=0 lun=0;
        :
        #name="hddv" class="scsi" target=0 lun=7;
        :
        name="hddv" class="scsi" target=1 lun=0;
        name="hddv" class="scsi" target=1 lun=1; Remove comments of the LUN in use
        name="hddv" class="scsi" target=1 lun=2;
        :
        name="hddv" class="scsi" target=3 lun=0;
        name="hddv" class="scsi" target=3 lun=1; Remove comments of extended LUN
        name="hddv" class="scsi" target=3 lun=2;
        #name="hddv" class="scsi" target=3 lun=3;
        :

      6. Check the LUN definition overlap between sd.conf and hddv.conf by executing hddvchk_conf command. The message example for "Check NG" means that target=3 is defined in both files, and the procedures must be performed again from procedure 8.b.4. The followings are examples of execution of the FJSVhddv package installation to /opt. The path of the hddvchk_conf command depends on the installed directory.

        Check OK [ to procedure 8.b.7.]
        # /opt/FJSVhddv/bin/hddvchk_conf <Return>
        /kernel/drv/sd.conf and /kernel/drv/hddv.conf seems to be correct

        Check NG [ to procedure 8.b.4.]
        # /opt/FJSVhddv/bin/hddvchk_conf <Return>
        NG ID = [ 3 ]

      7. Execute the following command in order to make the hard disk driver set up the LUN added.

        # update_drv -f hddv <Return>

        In case disk array equipment currently used exists, the following messages are displayed but do not effect performance.

        # update_drv -f hddv <Return>
        Cannot unload module: hddv
        Will be unloaded upon reboot.
        Forcing update of hddv.conf.

      8. After saving execution result of the prtpicl(1M) command, get the physical path name of added LUN by taking the difference between the result and the information obtained in 3.a, and calculate the corresponding logical path name under /dev/dsk. In the following example, "c3t3d0" is logical path name of added lun=0. The logical path name of lun=1,2 is set to "c3t3d1", "c3t3d2" respectively, because cXtYdZmeans X: PCI card number, Y: target number, Z: lun number .

        # prtpicl -v > /tmp/prtpicl.post <Return>
        # diff /tmp/prtpicl.pre /tmp/prtpicl.post | grep devfs-path | grep hddv <Return>
        > :devfs-path /pci@89,2000/fibre-channel@1/hddv@3,0
        > :devfs-path /pci@89,2000/fibre-channel@1/hddv@3,1
        > :devfs-path /pci@89,2000/fibre-channel@1/hddv@3,2
        # ls -l /dev/dsk | grep /pci@89,2000/fibre-channel@1/hddv@3,0 <Return>
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s0
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:a
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s1
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:b
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s2
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:c
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s3
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:d
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s4
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:e
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s5
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:f
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s6
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:g
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s7
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:h

      9. Start the format(1M) command and configure labeling and partition(slice) information calculated in 8.b.8. to the logical path name of each extended LUN.

    3. Add configure of PCI card driver

      [ SCSI driver(Solaris OE: glm , FUJITSU PCI SCSI Adapter Driver) ]

      No procedure is necessary. Go to step 9.

      [Fibre Channel driver (PW008FC3U/PW008FC2U/ GP7B8FC1U) ]

      No procedure is necessary. Go to step 9.

      [ Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*)]

      Bind the auto mapped target connected with the card and modify the entries of lpfc.conf to match the requirements of the extended Fibre Channel Controller. These procedures are reflected on next boot time. In the following, indicate to bind the auto mapped target with the card..

      # /usr/sbin/lpfc/lputil<Return>
      LightPulse Common Utility for Solaris/SPARC. Version 1.6a8 (03/03/2004).
      Copyright (c) 2003, Emulex Network Systems, Inc.

      Emulex Fibre Channel Host Adapters Detected: 2
      Host Adapter 0 (lpfc11) is an LP9802 (Ready Mode)
      Host Adapter 1 (lpfc9) is an LP9802 (Ready Mode)

      MAIN MENU
      1. List Adapters
      2. Adapter Information
      3. Firmware Maintenance
      4. Reset Adapter
      5. Persistent Bindings
      0. Exit
      Enter choice => 5 <Return>

      PERSISTENT BINDINGS MENU
      1. Display Current Bindings
      2. Display All Nodes
      3. Duplicate Adapter Bindings
      4. Bind Target Manually
      5. Bind Automapped Targets
      6. Delete Binding(s)
      0. Return to Main Menu
      Enter choice => 5 <Return>
      0. lpfc11
      1. lpfc9
      Select an adapter => 0 <Return>

      Automapped FCP Nodes:
      Node Target WWPN WWNN
      ---- ------ --------- --------
      Automapped FCP Node 0 50-06-0e-80-00-10-04-92 50-06-0e-80-00-10-04-92
      Automapped FCP Node 1 50-06-0e-80-00-10-04-90 50-06-0e-80-00-10-04-90

      Press the Enter or Return key to continue:
      Bind all auto-mapped targets? (Y/N) => Y <Return>

      Select Binding Method:
      1. By Node Name
      2. By Port Name
      3. By D_ID
      0. Cancel
      Enter Choice => 2 <Return>
      :

  9. Setup applications

    Add configuration for the application using the logical path name determined in 8.b.

    [ Disk device(filesystem operation) ]

    Create file system.

    To mount at system start, add an entry in the /etc/vfstab file.

    # newfs /dev/rdsk/c2t2d0s6 <Return>
    # vi /etc/vfstab <Return>
    :
    /dev/dsk/c2t2d0s6 /dev/rdsk/c2t2d0s6 /mnt ufs 1 yes -
    :

    [ Disk device(raw access operation)/Tape device ]

    Configure applications.

    For details, see the manual of each application.

    [ Disk device(swap device) ]

    Add swap device.

    To make valid at system start, add an entry in the /etc/vfstab file.

    # swap -a /dev/dsk/c2t2d0s4 <Return>
    # vi /etc/vfstab <Return>
    :
    /dev/dsk/c2t2d0s4 - - swap - no -
    :
    # swap -l <Return>
    swapfile dev swaplo blocks free
    :
    /dev/dsk/c2t2d0s4 32,164 16 788384 683680

  10. Start applications

    Use the logical path name determined in 8.b. to start applications.

    [ Disk device(filesystem operation) ]

    Mount filesystem

    # mount -F ufs /dev/dsk/c2t2d0s6 /mnt <Return>

    [ Disk device(raw access operation)/Tape device ]

    Start applications.

    For details, see the manual of each application.

    [ Disk device(swap device) ]

    This procedure is not required.

3.2.2.2 Extension of PCI cards with new redundant system 

The procedure to add PCI cards to a new redundant system on a Solaris 9 OE or later system is explained with examples for the multipath software below.

*Multipath Disk Control (MPHD)

*GR Multipath Driver (GRMPD)

an example with GR multipath driver (GRMPD).

If other redundant software is used, see the manual of each product.

  1. Stop the machine administration hardware monitoring daemon

    Use the following command to stop the hardware monitoring daemon of machine administration.

    # /usr/sbin/FJSVmadm/prephp <Return>

  2. If you use the Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*), execute the following.

    The daemons will be stopped.

    # /etc/rc0.d/K10ElxRMSrv stop <Return>

    # /etc/rc0.d/K10ElxDiscSrv stop <Return>

  3. Add PCI cards

    Add PCI cards according to the following procedure.

    [ for PCI Fibre Channel(PW008FC3U/PW008FC2U/ GP7B8FC1U)]:

    If you use SAN management function of Systemwalker StorageMGR/Softek SANView for ETERNUS (except for Vixel) /SP5000 SRM Facility

    No procedure is necessary. Go to step 4.

    If you don't use SAN management function of the above products

    To add PCI cards with the following configurations, Fibre Channel switch SN200 series and disk array device ETERNUS3000/ETERNUS6000/GR700/800 series need to be reconfigured individually.

    -If zone configuration by WWPN(World Wide Port Name) is done on SN200 series.

    -If the Host Affinity function of ETERNUS3000/ETERNUS6000/GR700/800 series is used.

    For details, see "SN200 Series Affinity User's Guide" or "ETERNUS3000/ETERNUS6000/GR700/800 series GRmgr User's Guide." If you use the Fibre Channel switch or disk array device other than described above, see the document of each product.

    To perform the above reconfiguration, the WWPN(a 16-digit number) of the replacement card is needed. The WWPN of the PCI card can be known from the eight characters shown on a label on the front plate of the card. These characters represent the bottom eight digit of the WWPN in hexadecimal form. The top eight digit are fixed to 10000000 in hexadecimal form.

    For example, if the following label is shown on the front plate of the card, the WWPN of the replacement card is 100000000e244061.

    0e24
    4061

    [ for Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*)]:

    To replace PCI cards with the following configurations, Fibre Channel switch and disk array device need to be reconfigured individually.

    -If zone configuration by WWPN (World Wide Port Name) is done on Fibre Channel switch.

    -If the Host Zoning function of disk array device is used.

    For details, see the document of each product.

    To perform the above reconfiguration, the WWPN(a 16-digit number) of the replacement card is needed. The WWPN of the PCI card can be known from the twelve characters shown on a label on the back of the card. These characters represent the bottom twelve digit of the WWPN in hexadecimal form. The top four digit are fixed to 1000 in hexadecimal form.

    For example, if the following label is shown on the back of the card, the WWPN of the replacement card is 10000000c9366037.

    IEEE:0000c9366037

    Note:

    When changing Affinity configuration on SN200 series or other Fibre Channel switch, I/O to other devices is effected by the change, and may result in temporal errors.

    I/O to disk array devices recovers normally because of retry processes, but on Fibre Channel tape devices, backup processes may end in errors. Stop backup before changing Affinity configuration.

    1. Before extension of PCI cards, save the current execution result of the prtpicl(1M) command (current configuration at present.)

      # prtpicl -v > /tmp/prtpicl.pre <Return>

    2. Check the status of the slot.

      The relationship of the position of a PCI card and format of "Ap_Id" is shown below.

      Location of PCI slot

      Definition of slot

      Format of "Ap_Id"

      Inside chassis of PRIMEPOWER 900/1500/2500

      Chassis No.: X

      System board No.: Y

      Physical slot No.: ZZ

      • Solaris 8 OE and Solaris 9 OE
        pcipsyN:CXM0Y-PCI#slotZZ
      • Solaris 10 OE
        CXM0Y-PCI#ZZ

      PCI/Disk BOX

      I/O Chassis No.: X

      PCI/Disk BOX No.: Y

      Physical slot No.: ZZ

      • Solaris 8 OE and Solaris 9 OE
        pcipsyN:RXB0Y-PCI#slotZZ
      • Solaris 10 OE
        RXB0Y-PCI#ZZ

      Other

      Physical slot No.: ZZ

      • Solaris 8 OE and Solaris 9 OE
        pcipsyN:PCI#slotZZ
      • Solaris 10 OE
        PCI#ZZ

      Below, an example of adding a PCI card to chassis number : 0, PCI/Disk BOX number : 1, physical slot number : 02, slot in PCI/Disk BOX(R0B01-PCI#slot02) is explained.

      Check the status of the target slot is "empty unconfigured."

      # cfgadm | grep R0B01-PCI#slot02 <Return>
      pcipsy18:R0B01-PCI#slot02 unknown empty unconfigured unknown

    3. To confirm the slot position at extension operation, blink the ALARM LED of Ap_Id displayed in procedure 3.b.

      # cfgadm -x led=fault,mode=blink pcipsy18:R0B01-PCI#slot02 <Return>

    4. After checking that the READY LED on the PCI slot to add the PCI card is off, add PCI cards and connect cable to devices. This operation is performed by our customer support.

    5. Use the cfgadm(1M) command to check that the slot status of the added PCI card changed to "disconnected unconfigured".

      # cfgadm pcipsy18:R0B01-PCI#slot02 <Return>
      Ap_Id Type Receptacle Occupant Condition
      pcipsy18:R0B01-PCI#slot02 unknown disconnected unconfigured unknown

  4. Connect the PCI card

    Connect the replaced PCI card using the cfgadm(1M) command with the configure option, or by pushing the button corresponding to the extension slot position. Note that the push button is only effective in multiuser mode. After the new PCI card is connected, use the cfgadm(1M) command and confirm that the slot status has changed to "connected configured."

    If a large-scale configuration of I/O devices is connected to the PCI card in the target slot, command execution for status confirmation may take time.

    # cfgadm -c configure pcipsy18:R0B01-PCI#slot02 <Return>
    # cfgadm pcipsy18:R0B01-PCI#slot02 <Return>
    Ap_Id Type Receptacle Occupant Condition
    pcipsy18:R0B01-PCI#slot02 mult/hp connected configured ok

  5. If you use the Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*), execute the following.

    The daemons will be started.

    # /etc/rc2.d/S99ElxRMSrv start <Return>
    # /etc/rc2.d/S99ElxDiscSrv start <Return>

  6. Update hardware configuration information of machine administration/ Start the hardware monitoring daemon. Execute the following commands to update hardware configuration information of machine administration and to restart the hardware monitoring daemon.

    # /usr/sbin/FJSVmadm/postphp <Return>

  7. Preparation to recognize devices for Fibre Channel

    When the Host Affinity function is being used with Fibre Channel SN200 series, Host Affinity must be configured for the device added.

    [ for PCI Fibre Channel(PW008FC3U/PW008FC2U/ GP7B8FC1U)]:

    If you use SAN management function of Systemwalker StorageMGR/Softek SANView for ETERNUS (except for Vixel) /SP5000 SRM Facility

    WWPN zoning of SN200 series, Host affinity of Array disk device ETERNUS3000/ETERNUS6000/GR700/800 series and binding setting for fjpfca.conf/sd.conf of host can be added from SAN management window automatically.
    See "Operation guide" or "User's Manual" of each product for details.
    Before using this function, the following procedure 8.a.1. and 8.a.2. [except fcp-bind-target setting] are necessary.
    Moreover, begin from procedure 8.a.3. after executing this function.

    If you don't use SAN management function of the above products

    No procedure is necessary. Go to step 8.

    [ for Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*)]:

    No procedure is necessary. Go to step 8.

    Note:

    When changing Affinity configuration on SN200 series, I/O to other devices is effected by the change, and may result in temporal errors.

    I/O to disk array devices recovers normally because of retry processes, but on Fibre Channel tape devices, backup processes may end in errors. Stop backup before changing Affinity configuration.

  8. Set up driver

    Add driver configurations with the operation below.

    With file related drivers, configuration for the PCI card driver and the device driver is needed.

    1. Add configuration for PCI card driver

      [ SCSI driver(Solaris OE:glm, FUJITSU PCI SCSI Adapter Driver) ]

      Configuration is not required.

      [Fibre Channel driver (PW008FC3U/PW008FC2U/ GP7B8FC1U) ]

      1. After saving execution result of prtpicl (1M) command, take the difference between the result and the information obtained in procedure 3.a. to calculate the instance number of the fjpfca driver of the PCI card added.

        In the following example, instance number is 1, indicating that the interface name of the PCI card added is fjpfca1.

        Making the LED of the PCI card blink using the fc_adm command can specify the relation between the instance number and the slot position. The relation between the WWPN and instance number of the PCI card added can be known from the "fc_info -a" command.

        # prtpicl -v > /tmp/prtpicl.pfca <Return>
        # diff /tmp/prtpicl.pre /tmp/prtpicl.pfca | more <Return>
        :
        > :status okay
        > :fru PCI slot(PCI#02 at Rack#0,IO_BOX#1)
        > :component-name R0B01-PCI#02
        > :devfs-path /pci@8d,2000/fibre-channel@1
        > :driver-name fjpfca
        > :binding-name pci10cf,105c
        > :bus-addr 1
        > :instance 1
        > :_class obp-device
        > :name fibre-channel
        :

      2. Edit the /kernel/drv/fjpfca.conf file, and add the following property definitions to the instance number determined in 8.a.1.

        port(type of topology, link speed)

        Defined when adding PCI cards with Fabric connection.

        This definition is not needed with FC-AL connection.

        fcp-bind-target(connection between target ID and WWN)

        Defined when adding disk array with Fabric connection.

        This definition is not needed with FC-AL connection.

        max-throttle(maximum issued SCSI commands)

        Defined to limit the number of SCSI commands issued at the same time.

        The need and value of this definition depends on the disk array device, so refer to the manuals of each disk array device. Below is an example where definitions for a PCI card at fjpfca1(instance number 1) and a disk array device at target=3 is added.

        # vi /kernel/drv/fjpfca.conf <Return>
        :
        port=
        "fjpfca0:nport",
        "fjpfca1:nport:sp1"; <- add

        fcp-bind-target=
        "fjpfca0t0:0x50000e1000002501",
        "fjpfca1t3:0x50000e1000002ab0"; <- add

        max-throttle=
        "fjpfca0t0:128",
        "fjpfca1t3:240"; <- add
        :

      3. Execute the following command to make the fjpfca driver set up the additional part.

        # /usr/sbin/FJSVpfca/fc_config -v <Return>

        [ Fibre Channel driver (PW028FC3*/PW028FC4*/PW028FC5*) ]

        Configuration is not required.

        Compare the output of prtpicl(1M) command with the output saved at 3.b and find the instance number of the added card.

    2. Add configuration of device driver

      This is an example where an disk array device in redundant configuration at target=3, lun=0-2 is added in procedure 4.

      If this is the first time to pre-define a disk array device, begin with procedure 8.b.1. If this is not the first time to pre-define a disk array device, begin with procedure 8.b.5.

      1. Check if /kernel/drv/hddv.conf exists

        Check if /kernel/drv/hddv.conf exists. If it exists, the hard disk driver is being installed, so start from procedure 8.b.3. If it does not exist, the system is being operated with none-active extension procedures, so start from procedure 8.b.2.

        Not exist(new install) [ to Procedure 8.b.3.]
        # ls /kernel/drv/hddv.conf <Return>
        /kernel/drv/hddv.conf: No such file or directory

        Exist(operation with non-active extension procedures) [ to Procedure 8.b.2. ]
        # ls /kernel/drv/hddv.conf <Return>
        /kernel/drv/hddv.conf

      2. When operating with non-active extension procedures, execute the following commands and confirm that /kernel/drv/hddv.conf is deleted.

        # hddvadm -i <Return>
        # ls /kernel/drv/hddv.conf <Return>
        /kernel/drv/hddv.conf: No such file or directory

      3. Use the following command to create hddv.conf file for the pre-defined extension function. The message examples for "operation with non-active extension procedures" means that disk array device GR720 currently used with taget=1(lun=0) is registered twice on the sd/hddv driver.

        New installation [ to procedure 8.b.5.]
        # hddvadm -M <Return>

        Operation with non-active extension procedures [ to procedure 8.b.4.]
        # hddvadm -M <Return>
        already ID= 1 exists. /devices/pci@1d,2000/fibre-channel@1/sd@1,0:a,raw : GR720

      4. When performing active extension, the same target number cannot be registered in sd.conf and hddv.conf, so delete or comment out the conflicting definition parts in sd.conf file when definition in the two files conflict with each other.

        The following example shows the case that target=1 is repeated

        # vi /kernel/drv/sd.conf <Return>
        :
        name="sd" class="scsi"
        target=0 lun=0;
        :
        #name="sd" class="scsi"
        target=1 lun=0;
        #name="sd" class="scsi" delete or comment out
        target=1 lun=1; the duplication definition parts in sd/hddv
        #name="sd" class="scsi"
        target=1 lun=2;
        :

        A target number that is already used by the sd driver of the system disk or CD-ROM cannot be deleted. In this case, change the target number in hddv.conf and of the disk array device so that it does not conflict with sd.conf.

      5. Edit the /kernel/drv/hddv.conf file created automatically by the hddvadm -M command, and remove comments of the LUN definition to pre-define and the LUN definition currently being used with non-active extension procedures.

        If case active extension has been performed before and the LUN definition to extend the next procedures 1.f. - 1.g. are not needed.

        The following example shows target=3, lun=0~2 for pre-defined extension and disk array device GR720 (target=1) in use with non-active extension procedures being registered.

        # ls /kernel/drv/hddv.conf <Return>
        /kernel/drv/hddv.conf
        # vi /kernel/drv/hddv.conf <Return>
        :
        #name="hddv" class="scsi" target=0 lun=0;
        :
        #name="hddv" class="scsi" target=0 lun=7;
        :
        name="hddv" class="scsi" target=1 lun=0;
        name="hddv" class="scsi" target=1 lun=1; remove comments of the LUN in use
        name="hddv" class="scsi" target=1 lun=2;
        :
        name="hddv" class="scsi" target=3 lun=0;
        name="hddv" class="scsi" target=3 lun=1; remove comments of extended LUN
        name="hddv" class="scsi" target=3 lun=2;
        #name="hddv" class="scsi" target=3 lun=3;
        :

      6. Check the LUN definition overlap between sd.conf and hddv.conf by executing hddvchk_conf command.

        The message example for "Check NG" means that target=3 is defined in both files, and the procedures must be performed again from procedure 8.b.4.

        The followings are examples of execution of the FJSVhddv package installation to /opt. The path of the hddvchk_conf command depends on the installed directory.

        Check OK [ to procedure 8.b.7.]
        # /opt/FJSVhddv/bin/hddvchk_conf <Return>
        /kernel/drv/sd.conf and /kernel/drv/hddv.conf seems to be correct

        Check NG [ to procedure 8.b.4]
        # /opt/FJSVhddv/bin/hddvchk_conf <Return>
        NG ID = [ 3 ]

      7. Execute the following command in order to make the hard disk driver set up the LUN added.

        # update_drv -f hddv <Return>

        In case disk array equipment currently used exists, the following messages are displayed but do not effect performance.

        # update_drv -f hddv <Return>
        Cannot unload module: hddv
        Will be unloaded upon reboot.
        Forcing update of hddv.conf.

      8. After saving execution result of the prtpicl(1M) command, get the physical path name of added LUN by taking the difference between the result and the information obtained in 3.a, and calculate the corresponding logical path name under /dev/dsk

        In the following example, "c3t3d0" is logical path name of added lun=0. The logical path name of lun=1,2 is set to "c3t3d1", "c3t3d2" respectively, because cXtYdZmeans X: PCI card number, Y: target number, Z: lun number

        # prtpicl -v > /tmp/prtpicl.post <Return>
        # diff /tmp/prtpicl.pre /tmp/prtpicl.post | grep devfs-path | grep hddv <Return>
        > :devfs-path /pci@89,2000/fibre-channel@1/hddv@3,0
        > :devfs-path /pci@89,2000/fibre-channel@1/hddv@3,1
        > :devfs-path /pci@89,2000/fibre-channel@1/hddv@3,2
        # ls -l /dev/dsk | grep /pci@89,2000/fibre-channel@1/hddv@3,0 <Return>
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s0
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:a
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s1
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:b
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s2
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:c
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s3
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:d
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s4
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:e
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s5
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:f
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s6
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:g
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s7
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:h

      9. Start the format(1M) command and configure labeling and partition(slice) information calculated in 8.b.8. to the logical path name of each extended LUN.
    3. Add configure of PCI card driver

      [ SCSI driver(Solaris OE: glm , FUJITSU PCI SCSI Adapter Driver) ]

      No procedure is necessary. Go to step 9.

      [ Fibre Channel driver (PW008FC3U/PW008FC2U/ GP7B8FC1U) ]

      No procedure is necessary. Go to step 9.

      [ Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*)]

      Bind the auto mapped target connected with the card and modify the entries of lpfc.conf to match the requirements of the extended Fibre Channel Controller. These procedures are reflected on next boot time. In the following, indicate to bind the auto mapped target with the card.

      # /usr/sbin/lpfc/lputil<Return>
      LightPulse Common Utility for Solaris/SPARC. Version 1.6a8 (03/03/2004).
      Copyright (c) 2003, Emulex Network Systems, Inc.

      Emulex Fibre Channel Host Adapters Detected: 2
      Host Adapter 0 (lpfc11) is an LP9802 (Ready Mode)
      Host Adapter 1 (lpfc9) is an LP9802 (Ready Mode)

      MAIN MENU
      1. List Adapters
      2. Adapter Information
      3. Firmware Maintenance
      4. Reset Adapter
      5. Persistent Bindings
      0. Exit
      Enter choice => 5 <Return>

      PERSISTENT BINDINGS MENU
      1. Display Current Bindings
      2. Display All Nodes
      3. Duplicate Adapter Bindings
      4. Bind Target Manually
      5. Bind Automapped Targets
      6. Delete Binding(s)
      0. Return to Main Menu
      Enter choice => 5 <Return>
      0. lpfc11
      1. lpfc9
      Select an adapter => 0 <Return>

      Automapped FCP Nodes:
      Node Target WWPN WWNN
      ---- ------ --------- --------
      Automapped FCP Node 0 50-06-0e-80-00-10-04-92 50-06-0e-80-00-10-04-92
      Automapped FCP Node 1 50-06-0e-80-00-10-04-90 50-06-0e-80-00-10-04-90

      Press the Enter or Return key to continue:
      Bind all auto-mapped targets? (Y/N) => Y <Return>

      Select Binding Method:
      1. By Node Name
      2. By Port Name
      3. By D_ID

      0. Cancel
      Enter Choice => 2 <Return>
      :

  9. Setup redundant system

    Execute the following commands to create a new redundant system.

    The following procedure is described for MPHD/GRMPHD. If you use the redundancy software other than MPHD/GRMPHD, see the document for each product.

    In case of MPHD, the commands need to be executed for each LUN, specifying the path name determined in 8.b.8 as a parameter.

    The italic letters below are the new multipath names, and the underlined letters are the path names that compose the multipath.

    MPHD
    # /usr/opt/FJSViomp/bin/iompadm -c mphd new /dev/rdsk/c3t3d0s2 /dev/rdsk/c4t3d0s2 <Return>
    IOMP: /dev/FJSVmphd/fiomp/adm2
    Node:
    /dev/FJSVmphd/rdsk/mphd2s0
    /dev/FJSVmphd/rdsk/mphd2s1
    /dev/FJSVmphd/rdsk/mphd2s2
    /dev/FJSVmphd/rdsk/mphd2s3
    /dev/FJSVmphd/rdsk/mphd2s4
    /dev/FJSVmphd/rdsk/mphd2s5
    /dev/FJSVmphd/rdsk/mphd2s6
    /dev/FJSVmphd/rdsk/mphd2s7

    GRMPD
    # /usr/sbin/mplbconfig -nv <Return>
    *** Phase 1: checking /dev/rdsk ***
    *** Phase 2: checking multi path ***
    *** Phase 3: analyzing path ***
    Path : Action : Element path : LUN : Storage : Zone
    mplb0 : exist : c1t4d0s2 c2t3d0s2 : 0 : GR720464f-12340001 :
    :
    mplb8 : new : c3t3d0s2 c4t3d0s2 : 0 : GR8404550- 000213 :
    :
    # /usr/sbin/mplbconfig -av <Return>
    :
    IOMP: /dev/FJSVmplb/fiomp/adm8
    Node:
    /dev/FJSVmplb/rdsk/mplb8s0
    /dev/FJSVmplb/rdsk/mplb8s1
    /dev/FJSVmplb/rdsk/mplb8s2
    /dev/FJSVmplb/rdsk/mplb8s3
    /dev/FJSVmplb/rdsk/mplb8s4
    /dev/FJSVmplb/rdsk/mplb8s5
    /dev/FJSVmplb/rdsk/mplb8s6
    /dev/FJSVmplb/rdsk/mplb8s7
    :

  10. Setup applications

    Add configuration for the application using the logical path name determined in 8.

    [ filesystem operation ]

    Create file system.

    To mount at system start, add an entry in the /etc/vfstab file.

    # newfs /dev/FJSVmphd/rdsk/mphd2s6 <Return>
    # vi /etc/vfstab <Return>
    :
    /dev/FJSVmphd/dsk/mphd2s6 /dev/FJSVmphd/rdsk/mphd2s6 /mnt ufs 1 yes -
    :

    [ raw access operation ]

    Configure applications.

    For details, see the manual of each application.

    [ swap device ]

    Add swap device.

    To make valid at system start, add an entry in the /etc/vfstab file.

    # swap -a /dev/FJSVmphd/dsk/mphd2s4 <Return>
    # vi /etc/vfstab <Return>
    :
    /dev/FJSVmphd/dsk/mphd2s4- - swap - no -
    :
    # swap -l <Return>
    swapfile dev swaplo blocks free
    :
    /dev/FJSVmphd/dsk/mphd2s4 32,164 16 788384 683680

  11. Start applications

    Use the logical path name determined in 8. to start applications.

    [ filesystem operation ]

    Mount filesystem

    # mount -F ufs /dev/FJSVmphd/dsk/mphd2s6 /mnt <Return>

    [ raw access operation ]

    Start applications.

    For details, see the manual of each application.

    [ swap device ]

    This procedure is not required.

3.2.2.3 Extension of PCI cards to redundant system 

The procedure to add PCI cards to a redundant system in order to extend redundancy on a Solaris 9 OE or later system is explained with an example with GR multipath driver (GRMPD).

There is no additional pattern for Multipath Disk Control (MPHD).

If multipath software products other than GRMPD are being used, see the manual of each product.

No configuration is needed for applications when adding PCI cards to an existing redundant system.

  1. Stop the machine administration hardware monitoring daemon

    Use the following command to stop the hardware monitoring daemon of machine administration.

    # /usr/sbin/FJSVmadm/prephp <Return>

  2. If you use the Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*), execute the following.

    The daemons will be stopped.

    # /etc/rc0.d/K10ElxRMSrv stop <Return>
    # /etc/rc0.d/K10ElxDiscSrv stop <Return>

  3. Add PCI cards

    Add PCI cards according to the following procedure.

    [ for PCI Fibre Channel(PW008FC3U/PW008FC2U/ GP7B8FC1U)]:

    If you use SAN management function of Systemwalker StorageMGR/Softek SANView for ETERNUS (except for Vixel) /SP5000 SRM Facility

    No procedure is necessary. Go to step 4.

    If you don't use SAN management function of the above products

    To add PCI cards with the following configurations, Fibre Channel switch SN200 series and disk array device ETERNUS3000/ETERNUS6000/GR700/800 series need to be reconfigured individually.

    -If zone configuration by WWPN(World Wide Port Name) is done on SN200 series.

    -If the Host Affinity function of ETERNUS3000/ETERNUS6000/GR700/800 series is used.

    For details, see "SN200 Series Affinity User's Guide" or "ETERNUS3000/ETERNUS6000/GR700/800 series GRmgr User's Guide." If you use the Fibre Channel switch or disk array device other than described above, see the document of each product.

    To perform the above reconfiguration, the WWPN(a 16-digit number) of the replacement card is needed. The WWPN of the PCI card can be known from the eight characters shown on a label on the front plate of the card. These characters represent the bottom eight digit of the WWPN in hexadecimal form. The top eight digit are fixed to 10000000 in hexadecimal form.

    For example, if the following label is shown on the front plate of the card, the WWPN of the replacement card is 100000000e244061.

    0e24
    4061

    [ for Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*)]:

    To replace PCI cards with the following configurations, Fibre Channel switch and disk array device need to be reconfigured individually.

    -If zone configuration by WWPN (World Wide Port Name) is done on Fibre Channel switch.

    -If the Host Zoning function of disk array device is used.

    For details, see the document of each product.

    To perform the above reconfiguration, the WWPN(a 16-digit number) of the replacement card is needed. The WWPN of the PCI card can be known from the twelve characters shown on a label on the back of the card. These characters represent the bottom twelve digit of the WWPN in hexadecimal form. The top four digit are fixed to 1000 in hexadecimal form.

    For example, if the following label is shown on the back of the card, the WWPN of the replacement card is 10000000c9366037.

    IEEE:0000c9366037

    Note:

    When changing Affinity configuration on SN200 series or other Fibre Channel switch, I/O to other devices is effected by the change, and may result in temporal errors.

    I/O to disk array devices recovers normally because of retry processes, but on Fibre Channel tape devices, backup processes may end in errors. Stop backup before changing Affinity configuration.

    1. Before extension of PCI cards, save the current execution result of the prtpicl(1M) command (current configuration at present.)

      # prtpicl -v > /tmp/prtpicl.pre <Return>

    2. Check the status of the slot.

      When adding PCI cards to a redundant configuration, several PCI cards may be added to the system. In this case, perform procedures to 3. on each card.

      The relationship of the position of a PCI card and format of "Ap_Id" is shown below.

      Location of PCI slot

      Definition of slot

      Format of "Ap_Id"

      Inside chassis of PRIMEPOWER 900/1500/2500

      Chassis No.: X
      System board No.: Y
      Physical slot No.: ZZ

      • Solaris 8 OE and Solaris 9 OE
        pcipsyN:CXM0Y-PCI#slotZZ
      • Solaris 10 OE
        CXM0Y-PCI#ZZ

      PCI/Disk BOX

      I/O Chassis No.: X
      PCI/Disk BOX No.: Y
      Physical slot No.: ZZ

      • Solaris 8 OE and Solaris 9 OE
        pcipsyN:RXB0Y-PCI#slotZZ
      • Solaris 10 OE
        RXB0Y-PCI#ZZ

      Other

      Physical slot No.: ZZ

      • Solaris 8 OE and Solaris 9 OE
        pcipsyN:PCI#slotZZ
      • Solaris 10 OE
        PCI#ZZ

      Below, an example of adding a PCI card to chassis number : 0, PCI/Disk BOX number : 1, physical slot number : 02, slot in PCI/Disk BOX(R0B01-PCI#slot02) is explained.

      Check the status of the target slot is "empty unconfigured."

      # cfgadm | grep R0B01-PCI#slot02 <Return>
      pcipsy18:R0B01-PCI#slot02 unknown empty unconfigured unknown

    3. To confirm the slot position at extension operation, blink the ALARM LED of Ap_Id displayed in procedure 3.b.

      # cfgadm -x led=fault,mode=blink pcipsy18:R0B01-PCI#slot02 <Return>

    4. After checking that the READY LED on the PCI slot to add the PCI card is off, add PCI cards and connect cable to devices. This operation is performed by our customer support.

    5. Use the cfgadm(1M) command to check that the slot status of the added PCI card changed to "disconnected unconfigured".

      # cfgadm pcipsy18:R0B01-PCI#slot02 <Return>
      Ap_Id Type Receptacle Occupant Condition
      pcipsy18:R0B01-PCI#slot02 unknown disconnected unconfigured unknown

  4. Connect the PCI card

    Connect the replaced PCI card using the cfgadm(1M) command with the configure option, or by pushing the button corresponding to the extension slot position. Note that the push button is only effective in multiuser mode. After the new PCI card is connected, use the cfgadm(1M) command and confirm that the slot status has changed to "connected configured."

    If a large-scale configuration of I/O devices is connected to the PCI card in the target slot, command execution for status confirmation may take time.

    # cfgadm -c configure pcipsy18:R0B01-PCI#slot02 <Return>
    # cfgadm pcipsy18:R0B01-PCI#slot02 <Return>
    Ap_Id Type Receptacle Occupant Condition
    pcipsy18:R0B01-PCI#slot02 mult/hp connected configured ok

  5. If you use the Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*), execute the following.

    The daemons will be started.

    # /etc/rc2.d/S99ElxRMSrv start <Return>
    # /etc/rc2.d/S99ElxDiscSrv start <Return>

  6. Update hardware configuration information of machine administration/ Start the hardware monitoring daemon.

    Execute the following commands to update hardware configuration information of machine administration and to restart the hardware monitoring daemon.

    # /usr/sbin/FJSVmadm/postphp <Return>

  7. Preparation to recognize devices for Fibre Channel

    When the Host Affinity function is being used with Fibre Channel SN200 series, Host Affinity must be configured for the device added.

    [ for PCI Fibre Channel(PW008FC3U/PW008FC2U/ GP7B8FC1U)]:

    If you use SAN management function of Systemwalker StorageMGR/Softek SANView for ETERNUS (except for Vixel) /SP5000 SRM Facility

    WWPN zoning of SN200 series, Host affinity of Array disk device ETERNUS3000/ETERNUS6000/GR700/800 series and binding setting for fjpfca.conf/sd.conf of host can be added from SAN management window automatically.
    See "Operation guide" or "User's Manual" of each product for details.
    Before using this function, the following procedure 8.a.1. and 8.a.2. [except fcp-bind-target setting] are necessary.
    Moreover, begin from procedure 8.a.3. after executing this function.

    If you don't use SAN management function of the above products

    No procedure is necessary. Go to step 8.

    [ for Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*)]:

    No procedure is necessary. Go to step 8.

    Note:

    When changing Affinity configuration on SN200 series, I/O to other devices is effected by the change, and may result in temporal errors.

    I/O to disk array devices recovers normally because of retry processes, but on Fibre Channel tape devices, backup processes may end in errors. Stop backup before changing Affinity configuration.

  8. Set up driver

    Add driver configurations with the operation below.

    With file related drivers, configuration for the PCI card driver and the device driver is needed.

    1. Add configuration for PCI card driver

      [SCSI driver(Solaris OE:glm, FUJITSU PCI SCSI Adapter Driver) ]

      Configuration is not required.

      [ Fibre Channel driver (PW008FC3U/PW008FC2U/ GP7B8FC1U) ]

      1. After saving execution result of prtpicl (1M) command, take the difference between the result and the information obtained in procedure 3.a. to calculate the instance number of the fjpfca driver of the PCI card added.

        In the following example, instance number is 1, indicating that the interface name of the PCI card added is fjpfca1.

        Making the LED of the PCI card blink using the fc_adm command can specify the relation between the instance number and the slot position. The relation between the WWPN and instance number of the PCI card added can be known from the "fc_info -a" command.

        # prtpicl -v > /tmp/prtpicl.pfca <Return>
        # diff /tmp/prtpicl.pre /tmp/prtpicl.pfca | more <Return>
        :
        > :status okay
        > :fru PCI slot(PCI#02 at Rack#0,IO_BOX#1)
        > :component-name R0B01-PCI#02
        > :devfs-path /pci@8d,2000/fibre-channel@1
        > :driver-name fjpfca
        > :binding-name pci10cf,105c
        > :bus-addr 1
        > :instance 1
        > :_class obp-device
        > :name fibre-channel
        :

      2. Edit the /kernel/drv/fjpfca.conf file, and add the following property definitions to the instance number determined in 8.a.1.

        port(type of topology, link speed)

        Defined when adding PCI cards with Fabric connection.

        This definition is not needed with FC-AL connection.

        fcp-bind-target(connection between target ID and WWN)

        Defined when adding disk array with Fabric connection.

        This definition is not needed with FC-AL connection.

        max-throttle(maximum issued SCSI commands)

        Defined to limit the number of SCSI commands issued at the same time.

        The need and value of this definition depends on the disk array device, so refer to the manuals of each disk array device.

        Below is an example where definitions for a PCI card at fjpfca1(instance number 1) and a disk array device at target=3 is added.

        # vi /kernel/drv/fjpfca.conf <Return>
        :
        port=
        "fjpfca0:nport",
        "fjpfca1:nport:sp1"; <- add

        fcp-bind-target=
        "fjpfca0t0:0x50000e1000002501",
        "fjpfca1t3:0x50000e1000002ab0"; <- add

        max-throttle=
        "fjpfca0t0:128",
        "fjpfca1t3:240"; <- add
        :

      3. Execute the following command to make the fjpfca driver set up the additional part.

        # /usr/sbin/FJSVpfca/fc_config -v <Return>

        [ Fibre Channel driver (PW028FC3*/PW028FC4*/PW028FC5*) ]

        Configuration is not required.

        Compare the output of prtpicl(1M) command with the output saved at 3.b and find the instance number of the added card.

    2. Add configuration of device driver

      [ Disk/Tape devices(Solaris OE : sd/st) ]

      This example is a case where a disk device of target=2, lun=0 is pre-defined in procedure 1. and added in procedure 4. If the device is a tape device, re-read "sd" as "st" in this procedure and perform the same operation.

      1. Execute the following command to make the sd driver set up the added disk device.

        # /usr/sbin/devfsadm <Return>

      2. After saving the execution result of the prtpicl(1M) command, take the difference between the result and the information obtained in 3.a. and get the physical path name of additional disk device, then get the logical path name under /dev/dsk.

        In the following example, "c3t3d0","c4t3d0" is the logical path name of the disk device added.

        # prtpicl -v > /tmp/prtpicl.post <Return>
        # diff /tmp/prtpicl.pre /tmp/prtpicl.post | grep devfs-path | grep hddv <Return>
        > :devfs-path /pci@8d,2000/fibre-channel@1/sd@3,0
        > :devfs-path /pci@8d,2000/fibre-channel@1/sd@3,1
        > :devfs-path /pci@8d,2000/fibre-channel@1/sd@3,2
        > :devfs-path /pci@89,2000/fibre-channel@1/sd@3,0
        > :devfs-path /pci@89,2000/fibre-channel@1/sd@3,1
        > :devfs-path /pci@89,2000/fibre-channel@1/sd@3,2
        # ls -l /dev/dsk | grep /pci@8d,2000/fibre-channel@1/sd@3,0 <Return>
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s0
        -> ../../devices/pci@8d,2000/fibre-channel@1/sd@3,0:a
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s1
        -> ../../devices/pci@8d,2000/fibre-channel@1/sd@3,0:b
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s2
        -> ../../devices/pci@8d,2000/fibre-channel@1/sd@3,0:c
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s3
        -> ../../devices/pci@8d,2000/fibre-channel@1/sd@3,0:d
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s4
        -> ../../devices/pci@8d,2000/fibre-channel@1/sd@3,0:e
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s5
        -> ../../devices/pci@8d,2000/fibre-channel@1/sd@3,0:f
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s6
        -> ../../devices/pci@8d,2000/fibre-channel@1/sd@3,0:g
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s7
        -> ../../devices/pci@8d,2000/fibre-channel@1/sd@3,0:h
        # ls -l /dev/dsk | grep /pci@89,2000/fibre-channel@1/sd@3,0 <Return>
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c4t3d0s0
        -> ../../devices/pci@89,2000/fibre-channel@1/sd@3,0:a

      [ Disk array device (hard disk driver : hddv) ]

      This example is a case where a disk array device of target=3, lun=0-2 is pre-defined in procedure 1. and added in procedure 4.

      1. Check if /kernel/drv/hddv.conf exists.

        Check if /kernel/drv/hddv.conf exists. If it exists, the hard disk driver is being installed, so start from procedure 8.b.3. If it does not exist, the system is being operated with none-active extension procedures, so start from procedure 8.b.2.

        Not exist(new install) [ to procedure 8.b.3.]
        # ls /kernel/drv/hddv.conf <Return>
        /kernel/drv/hddv.conf: No such file or directory

        Exist(operation with non-active extension procedures) [ to procedure 8.b.2.]
        # ls /kernel/drv/hddv.conf <Return>
        /kernel/drv/hddv.conf

      2. When operating with non-active extension procedures, execute the following commands and confirm that /kernel/drv/hddv.conf is deleted.

        # hddvadm -i <Return>
        # ls /kernel/drv/hddv.conf <Return>
        /kernel/drv/hddv.conf: No such file or directory

      3. Use the following command to create hddv.conf file for the pre-defined extension function. The message examples for "operation with non-active extension procedures" means that disk array device GR720 currently used with taget=1(lun=0) is registered twice on the sd/hddv driver.

        New installation [ to procedure 8.b.5.]
        # hddvadm -M <Return>

        Set up with a non-active extension procedure [ to procedure 8.b.4.]
        # hddvadm -M <Return>
        already ID= 1 exists. /devices/pci@1d,2000/fibre-channel@1/sd@1,0:a,raw : GR720

      4. In activity extension, because same target number can't be registered in sd.conf and hddv.conf, delete or comment out the duplication definition parts in sd.conf file when both definition are overlapped. The following example shows the case that target=1 is repeated.

        # vi /kernel/drv/sd.conf <Return>
        :
        name="sd" class="scsi"
        target=0 lun=0;
        :
        #name="sd" class="scsi"
        target=1 lun=0;
        #name="sd" class="scsi" delete or comment out
        target=1 lun=1; the duplication definition parts in sd/hddv
        #name="sd" class="scsi"
        target=1 lun=2;
        :

        A target number that is already used by the sd driver of the system disk or CD-ROM cannot be deleted. In this case, change the target number in hddv.conf and of the disk array device so that it does not conflict with sd.conf.

      5. Edit the /kernel/drv/hddv.conf file created automatically by the hddvadm -M command, and remove comments of the LUN definition to pre-define and the LUN definition currently being used with non-active extension procedures.

        If case active extension has been performed before and the LUN definition to extend the next procedures 1.f. - 1.g. are not needed.

        The following example shows target=3 , lun=0~2 for pre-defined extension and disk array device GR720 (target=1) in use with non-active extension procedures being registered.

        # ls /kernel/drv/hddv.conf <Return>
        /kernel/drv/hddv.conf
        # vi /kernel/drv/hddv.conf <Return>
        :
        #name="hddv" class="scsi" target=0 lun=0;
        :
        #name="hddv" class="scsi" target=0 lun=7;
        :
        name="hddv" class="scsi" target=1 lun=0;
        name="hddv" class="scsi" target=1 lun=1; remove comments of the LUN in use
        name="hddv" class="scsi" target=1 lun=2;
        :
        name="hddv" class="scsi" target=3 lun=0;
        name="hddv" class="scsi" target=3 lun=1; remove comments of extended LUN
        name="hddv" class="scsi" target=3 lun=2;
        #name="hddv" class="scsi" target=3 lun=3;
        :

      6. Check the LUN definition overlap between sd.conf and hddv.conf by executing hddvchk_conf command.

        The message example for "Check NG" means that target=3 is defined in both files, and the procedures must be performed again from procedure 8.b.4.

        The followings are examples of execution of the FJSVhddv package installation to /opt. The path of the hddvchk_conf command depends on the installed directory.

        Check OK [ to procedure 8.b.7.]
        # /opt/FJSVhddv/bin/hddvchk_conf <Return>
        /kernel/drv/sd.conf and /kernel/drv/hddv.conf seems to be correct

        Check NG [ to procedure 8.b.4.]
        # /opt/FJSVhddv/bin/hddvchk_conf <Return>
        NG ID = [ 3 ]

      7. Execute the following command in order to make the hard disk driver set up the LUN added.

        # update_drv -f hddv <Return>

        In case disk array equipment currently used exists, the following messages are displayed but do not effect performance.

        # update_drv -f hddv <Return>
        Cannot unload module: hddv
        Will be unloaded upon reboot.
        Forcing update of hddv.conf.

      8. After saving execution result of the prtpicl(1M) command, get the physical path name of added LUN by taking the difference between the result and the information obtained in 3.a, and calculate the corresponding logical path name under /dev/dsk.

        In the following example, "c3t3d0" is logical path name of added lun=0. The logical path name of lun=1,2 is set to "c3t3d1", "c3t3d2" respectively, because cXtYdZmeans X: PCI card number, Y: target number, Z: lun number

        # prtpicl -v > /tmp/prtpicl.post <Return>
        # diff /tmp/prtpicl.pre /tmp/prtpicl.post | grep devfs-path | grep hddv <Return>
        > :devfs-path /pci@89,2000/fibre-channel@1/hddv@3,0
        > :devfs-path /pci@89,2000/fibre-channel@1/hddv@3,1
        > :devfs-path /pci@89,2000/fibre-channel@1/hddv@3,2
        # ls -l /dev/dsk | grep /pci@89,2000/fibre-channel@1/hddv@3,0 <Return>
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:a
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s1
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:b
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s2
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:c
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s3
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:d
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s4
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:e
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s5
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:f
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s6
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:g
        lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s7
        -> ../../devices/pci@89,2000/fibre-channel@1/hddv@3,0:h

      9. Start format command and configure labeling and partition (slice) information calculated at 8.b.8 to the logical path name of each extended LUN.

    3. Add configure of PCI card driver

      [ SCSI driver(Solaris OE: glm , FUJITSU PCI SCSI Adapter Driver) ]

      No procedure is necessary. Go to step 9.

      [ Fibre Channel driver (PW008FC3U/PW008FC2U/ GP7B8FC1U) ]

      No procedure is necessary. Go to step 9.

      [ Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*)]

      Bind the auto mapped target connected with the card and modify the entries of lpfc.conf to match the requirements of the extended Fibre Channel Controller. These procedures are reflected on next boot time. In the following, indicate to bind the auto mapped target with the card.

      # /usr/sbin/lpfc/lputil<Return>
      LightPulse Common Utility for Solaris/SPARC. Version 1.6a8 (03/03/2004).
      Copyright (c) 2003, Emulex Network Systems, Inc.

      Emulex Fibre Channel Host Adapters Detected: 2
      Host Adapter 0 (lpfc11) is an LP9802 (Ready Mode)
      Host Adapter 1 (lpfc9) is an LP9802 (Ready Mode)

      MAIN MENU
      1. List Adapters
      2. Adapter Information
      3. Firmware Maintenance
      4. Reset Adapter
      5. Persistent Bindings
      0. Exit
      Enter choice => 5 <Return>

      PERSISTENT BINDINGS MENU
      1. Display Current Bindings
      2. Display All Nodes
      3. Duplicate Adapter Bindings
      4. Bind Target Manually
      5. Bind Automapped Targets
      6. Delete Binding(s)
      0. Return to Main Menu
      Enter choice => 5 <Return>
      0. lpfc11
      1. lpfc9
      Select an adapter => 0 <Return>

      Automapped FCP Nodes:
      Node Target WWPN WWNN
      ---- ------ --------- --------
      Automapped FCP Node 0 50-06-0e-80-00-10-04-92 50-06-0e-80-00-10-04-92
      Automapped FCP Node 1 50-06-0e-80-00-10-04-90 50-06-0e-80-00-10-04-90
      Press the Enter or Return key to continue:
      Bind all auto-mapped targets? (Y/N) => Y <Return>
      Select Binding Method:
      1. By Node Name
      2. By Port Name
      3. By D_ID
      0. Cancel
      Enter Choice => 2 <Return>
      :

  9. Connect to redundant system

    The following procedure is described for MPHD/GRMPHD. If you use the redundancy software other than MPHD/GRMPHD, see the document for each product.

    To connect the added PCI card to the redundant system, execute the following command and confirm that the path name determined in procedure 8.b.8 (in underlined letters) is added.

    # /usr/sbin/mplbconfig -nv <Return>
    *** Phase 1: checking /dev/rdsk ***
    *** Phase 2: checking multi path ***
    *** Phase 3: analyzing path ***
    Path : Action : Element path : LUN : Storage : Zone
    mplb0 : exist : c1t4d0s2 c2t3d0s2 : 0 : GR720464f-12340001 :
    :
    mplb8 : add : c3t3d0s2 c4t3d0s2 c5t3d0s2: 0 : GR8404550- 000213 :
    :
    # /usr/sbin/mplbconfig -av <Return>
    :
    IOMP: /dev/FJSVmplb/fiomp/adm8
    Node:
    /dev/FJSVmplb/rdsk/mplb8s0
    /dev/FJSVmplb/rdsk/mplb8s1
    /dev/FJSVmplb/rdsk/mplb8s2
    /dev/FJSVmplb/rdsk/mplb8s3
    /dev/FJSVmplb/rdsk/mplb8s4
    /dev/FJSVmplb/rdsk/mplb8s5
    /dev/FJSVmplb/rdsk/mplb8s6
    /dev/FJSVmplb/rdsk/mplb8s7
    :


Contents PreviousNext

All Rights Reserved, Copyright(C) FUJITSU LIMITED 2005