| Dynamic Reconfiguration User's Guide I/O device edition - for Solaris (TM) Operating System - |
|
Contents
|
| Chapter 3 File Devices | > 3.2 Expansion Procedure | > 3.2.1 Solaris 8 |
This section shows the flow of the procedure PCI cards expansion for non-redundant configuration on Solaris 8 system.
Using the following procedures for each device type, reserve the definition of I/O devices that are expected to expand.
When the I/O devices are not defined, it is not possible to expand by using DR.
So please define them before system operation is started.
[Disk Devices/Tape Devices (Solaris:sd/st)]
In the following, the procedure is explained by taking the reserved definition of the disk device (target=2 lun=0) as an example.
For tape devices, execute the same operations with replacing "sd" of the following procedures to "st".
|
| # touch /reconfigure <Return> # /usr/sbin/shutdown -y -i6 -g0 <Return> |
[Disk Array devices (Hard Disk Driver:hddv)]
In the following, the procedure is explained by taking the reserved definition of the disk array device (target=3 lun=0 to 2) as an example.
| [In case of not existing (first installation)] --> go to step 1.c # ls /kernel/drv/hddv.conf <Return> /kernel/drv/hddv.conf: No such file or directory [In case of existing (non-hot expansion settings)] --> go to step 1.b # ls /kernel/drv/hddv.conf <Return> /kernel/drv/hddv.conf |
| # hddvadm -i <Return> # ls /kernel/drv/hddv.conf <Return> /kernel/drv/hddv.conf: No such file or directory |
|
| # vi /kernel/drv/sd.conf <Return> : name="sd" class="scsi" target=0 lun=0; : #name="sd" class="scsi" <------ target=1 lun=0; #name="sd" class="scsi" Delete or comment out target=1 lun=1; the duplicated definitions of sd and hddv drivers. #name="sd" class="scsi" target=1 lun=2; <------ : |
But it is impossible to delete the target IDs being recognized as a system volume a CD-ROM device etc. in sd driver.
In this case, change the target ID on hddv.conf and the disk array device side to unduplicated definition of sd.conf.
| # ls /kernel/drv/hddv.conf <Return> /kernel/drv/hddv.conf # vi /kernel/drv/hddv.conf <Return> : #name="hddv" class="scsi" target=0 lun=0; : #name="hddv" class="scsi" target=0 lun=7; : name="hddv" class="scsi" target=1 lun=0; <------ name="hddv" class="scsi" target=1 lun=1; Remove the comment symbols of using LUNs name="hddv" class="scsi" target=1 lun=2; <------ : name="hddv" class="scsi" target=3 lun=0; <------ name="hddv" class="scsi" target=3 lun=1; Remove the comment symbols of name="hddv" class="scsi" target=3 lun=2; <------ LUNs that are expected to expand #name="hddv" class="scsi" target=3 lun=3; : |
|
| # touch /reconfigure <Return> # /usr/sbin/shutdown -y -i6 -g0 <Return> |
If you use the Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*), execute the following.
| # /etc/rc0.d/K10ElxRMSrv stop <Return> # /etc/rc0.d/K10ElxDiscSrv stop <Return> |
In case of expanding PCI cards on the existing system boards, choose the system boards of expansion target and detach the system boards according to step 3 to 4 in Section 3.1.1 or step 4 in Section 3.1.2.
This procedure is not required when expanding the new system boards together.
Using the following procedures, expand the PCI cards. A certified service engineer takes charge of this work.
When expanding the fibre channel PCI cards, the steps below are required.
[for PCI Fibre Channel(PW008FC3U/PW008FC2U/ GP7B8FC1U)]:
When using any of SAN management function of Systemwalker/StorageMGR, Softek SANView for ETERNUS (except for Vixel) and SP5000 SRM Facility
No procedure is necessary. Go to step 6.
When not using SAN management function of the above products
In case of replacing of the fibre channel cards of the below environment setting, it is necessary to change the setting of the fibre channel switch SN200 series and the disk array device ETERNUS3000/ETERNUS6000/GR700/800 series.
For details, refer to the manual "Zoning User's Guide" and "ETERNUSmgr User Guide/GRmgr User Guide". If you use the Fibre Channel switch or disk array device other than described above, see the document of each product.
When changing the above setting, WWPNs (Figures of sixteen digits in all) of the replacement parts are needed. The WWPN of the expansion fibre channel card is determined by using eight-digit number printed on the seal of the card on front plate.
This figure of eight digit represents the lower eight digit of WWPN by the hexadecimal number. The upper eight digit of WWPN is 10000000 by a fixed hexadecimal number.
For instance, when the following seal puts on the expansion fibre channel card, the WWPN of the expansion card is 100000000e244061.
| 0e24 4061 |
[ for Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*)]:
To replace PCI cards with the following configurations, Fibre Channel switch and disk array device need to be reconfigured individually.
For details, see the document of each product.
To perform the above reconfiguration, the WWPN(a 16-digit number) of the replacement card is needed. The WWPN of the PCI card can be known from the twelve characters shown on a label on the back of the card. These characters represent the bottom twelve digits of the WWPN in hexadecimal form. The top four digits are fixed to 1000 in hexadecimal form.
For example, if the following label is shown on the back of the card, the WWPN of the replacement card is 10000000c9366037.
| IEEE:0000c9366037 |
| # /opt/FJSVhwr/sbin/drcstat -device > /tmp/drcstat.pre <Return> |
Note
When the zoning setting is changed in SN200 series or other Fibre Channel switch, executing I/O requests to other devices sometimes terminate abnormally with temporary errors by the setting changes. I/O requests to disk array devices and the like are recovered normally by retry processes, but backup processes of fibre channel tape devices sometimes terminate abnormally. Please execute the change of the zoning setting after stopping backup processes.
Attach the system boards equipped with the expanded PCI cards by using the drc(1M) command, and check if the status of the system boards is changed to "Configured".
In case of detaching the system boards in step 3, restart the stopped high-level products and check the multipath status according to step 7 in Section 3.1.1 or step 2.c in Section 3.1.2.
| # /opt/FJSVhwr/sbin/drc -connect sb00 <Return> # /opt/FJSVhwr/sbin/drcstat -board sb00 <Return> SB BN Status (Detail) PID Next_PID Board_Type CPU(MHz) -- -- --------------- --- -------- ---------- -------- 00 0 Configured 00 00 10 300 |
If you use the Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*), execute the following.
| # /etc/rc2.d/S99ElxRMSrv start <Return> # /etc/rc2.d/S99ElxDiscSrv start <Return> |
When the zoning function is used on fiber channel switch SN200 series configuration, it is necessary to set zoning to the expanded devices.
[for PCI Fibre Channel(PW008FC3U/PW008FC2U/ GP7B8FC1U)]:
When using any of SAN management function of Systemwalker/StorageMGR, Softek SANView for ETERNUS (except for Vixel) and SP5000 SRM Facility
WWPN zoning of SN200 series, Host affinity of Array disk device ETERNUS3000/ETERNUS6000/GR700/800 series and binding setting for fjpfca.conf/sd.conf of host can be added from SAN management window automatically.
See "Operation guide" or "User's Manual" of each product for details.
Before using this function, the following step 8.a.1) and 8.a.2) [except fcp-bind-target setting] are necessary.
Moreover, begin from step 8.a.3) after executing this function.
When not using SAN management function of the above products
No procedure is necessary.
[ for Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*)]:
No procedure is necessary.
Note
When the zoning setting is changed in SN200 series, executing I/O requests to other devices sometimes terminate abnormally with temporary errors by the setting changes.
I/O requests to disk array devices and the like are recovered normally by retry processes, but backup processes of fibre channel tape devices sometimes terminate abnormally.
Please execute the change of the zoning setting after stopping backup processes.
Using the steps below, add the setting of drivers.
For file drivers, it is necessary to add the setting of PCI card drivers and device drivers.
[SCSI drivers (Solaris:glm, FUJITSU PCI SCSI Adapter Driver)]
[PCI Fibre Channel(PW008FC3U/PW008FC2U/GP7B8FC1U)]
1) Save the results of execution of the drcstat(1M) command in a file, and find a difference between the data in step 4.a and the current result, then determine the instance numbers of fjpfca driver for the expanded PCI cards.
The following example shows that fjpfca1 (instance number: 1) is added.
If the relation between the instance number and the slot position is necessary, locate it by blinking LED of the expanded PCI card with using the fc_adm command.
Also, check by using the "fc_info -a" command about the relation between the WWPN of expanded card and the instance number.
# /opt/FJSVhwr/sbin/drcstat -device > /tmp/drcstat.pfca <Return>
# diff /tmp/drcstat.pre /tmp/drcstat.pfca | grep fjpfca <Return>
:
> 00-PCI#0B "/pci@80,4000/fibre-channel@1" 1 "fjpfca"
# /usr/sbin/FJSVpfca/fc_adm -l fjpfca1 <Return>
2) Edit the setting file "/kernel/drv/fjpfca.conf" and add the below definitions of property for instance numbers in step 8.a.1).port (topology types, link speed)
In case of expanding the PCI cards of fabric connection, define this property.
In FC-AL connection, this definition is not required.fcp-bind-target (binding target ID and WWN)
In case of expanding the disk array devices of fabric connection, define this property.
In FC-AL connection, this definition is not required.max-throttle (the number of maximum SCSI command issues)
This is the property to limit the number of SCSI commands (tag) that can be accepted at the same time.
The necessity and setting value of this property depend on the specification of the disk array devices, so refer to the manual of each disk array device.
The following example is the setting addition of the PCI card for fjpfca1 (instance number: 1) and the disk array device of target=3.
# vi /kernel/drv/fjpfca.conf <Return>
:
port=
"fjpfca0:nport",
"fjpfca1:nport:sp1"; <------ added
fcp-bind-target=
"fjpfca0t0:0x50000e1000002501",
"fjpfca1t3:0x50000e1000002ab0"; <------ added
max-throttle=
"fjpfca0t0:128",
"fjpfca1t3:240"; <------ added
:
3) Notify the fjpfca driver of the attachment of the expansion parts by using the following command.
# /usr/sbin/FJSVpfca/fc_config -v <Return>
[ Fibre Channel driver (PW028FC3*/PW028FC4*/PW028FC5*) ]
Configuration is not required.Compare the output of prtpicl(1M) command with the output saved at 4.a and find the instance number of the added card.
Using the following procedures for each I/O device type, add the setting of the device drivers.
[Disk Devices/Tape Devices (Solaris:sd/st)]
In the following, the procedure is explained by taking the expansion in step 5 after the reserved definition of the disk device (target=2 lun=0) in step 1 as an example.
For tape devices, execute the same operations with replacing "sd" of the following procedures to "st".
1) Notify the sd driver of the attachment of the expanded disk devices by using the following command.
# /usr/sbin/devfsadm <Return>
2) Save the results of execution of the drcstat(1M) command in a file, and find a difference between the data in step 3.a and the current result, then determine the physical path names and the logical path names under /dev/dsk of the expanded disk device.
The following example shows that "c2t2d0" is the logical path name of the expanded disk device.
# /opt/FJSVhwr/sbin/drcstat -device > /tmp/drcstat.post <Return>
- [Disk Devices]
# diff /tmp/drcstat.pre /tmp/drcstat.post | grep sd <Return>
:
> 00-PCI#0B "/pci@80,4000/scsi@2/sd@2,0" 10 "sd"
# ls -l /dev/dsk | grep /pci@80,4000/scsi@2/sd@2,0 <Return>
lrwxrwxrwx 1 root roo 41 Sep 20 22:53 c2t2d0s0
-> ../../devices/pci@80,4000/scsi@2/sd@2,0:a
lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t2d0s1
-> ../../devices/pci@80,4000/scsi@2/sd@2,0:b
lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t2d0s2
-> ../../devices/pci@80,4000/scsi@2/sd@2,0:c
lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t2d0s3
-> ../../devices/pci@80,4000/scsi@2/sd@2,0:d
lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t2d0s4
-> ../../devices/pci@80,4000/scsi@2/sd@2,0:e
lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t2d0s5
-> ../../devices/pci@80,4000/scsi@2/sd@2,0:f
lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t2d0s6
-> ../../devices/pci@80,4000/scsi@2/sd@2,0:g
lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t2d0s7
-> ../../devices/pci@80,4000/scsi@2/sd@2,0:h
- [Tape Devices]
# diff /tmp/drcstat.pre /tmp/drcstat.post | grep st <Return>
:
> 00-PCI#0B "/pci@80,4000/scsi@2/st@5,0" 1 "st"
# ls -l /dev/rmt | grep /pci@80,4000/scsi@2/st@5,0 <Return>
lrwxrwxrwx 1 root root 41 Sep 20 22:53 1
-> ../../devices/pci@80,4000/scsi@2/st@5,0:
lrwxrwxrwx 1 root root 41 Sep 20 22:53 1b
-> ../../devices/pci@80,4000/scsi@2/st@5,0:b
lrwxrwxrwx 1 root root 41 Sep 20 22:53 1bn
-> ../../devices/pci@80,4000/scsi@2/st@5,0:bn
lrwxrwxrwx 1 root root 41 Sep 20 22:53 1c
-> ../../devices/pci@80,4000/scsi@2/st@5,0:c
lrwxrwxrwx 1 root root 41 Sep 20 22:53 1cb
-> ../../devices/pci@80,4000/scsi@2/st@5,0:cb
lrwxrwxrwx 1 root root 41 Sep 20 22:53 1cbn
-> ../../devices/pci@80,4000/scsi@2/st@5,0:cbn
lrwxrwxrwx 1 root root 41 Sep 20 22:53 1cn
-> ../../devices/pci@80,4000/scsi@2/st@5,0:cn
lrwxrwxrwx 1 root root 41 Sep 20 22:53 1h
-> ../../devices/pci@80,4000/scsi@2/st@5,0:h
lrwxrwxrwx 1 root root 41 Sep 20 22:53 1hb
-> ../../devices/pci@80,4000/scsi@2/st@5,0:hb
lrwxrwxrwx 1 root root 41 Sep 20 22:53 1hbn
-> ../../devices/pci@80,4000/scsi@2/st@5,0:hbn
lrwxrwxrwx 1 root root 41 Sep 20 22:53 1hn
-> ../../devices/pci@80,4000/scsi@2/st@5,0:hn
3) For disk devices, label a disk and perform the setting of partitions (slices) for the logical path names in step 8.b.2) by using the format command.
For tape devices, this procedure is not required.
[Disk Array devices (Hard Disk Driver:hddv)]
In the following, the procedure is explained by taking the expansion in step 5 after the reserved definition of the disk array device (target=3 lun=0 to 2) in step 1 as an example.
1) Check if no duplicate definitions exist in the sd.conf and hddv.conf files again, because the situation could be changed since step 1.
The message example in "Check NG" shows the duplicate definition, so go back to step 1.d.
- [Check OK] --> go to step 8.b.2)
# /opt/FJSVhddv/bin/hddvchk_conf <Return>
/kernel/drv/sd.conf and /kernel/drv/hddv.conf seems to be correct
- [Check NG] --> go back to step 1.d
# /opt/FJSVhddv/bin/hddvchk_conf <Return>
NG ID = [ 3 ]
2) Notify the hard disk driver of the attachment of the expanded LUNs by using the following command.
# /usr/sbin/hddvadmdr <Return>
3) Save the results of execution of the drcstat(1M) command in a file, and find a difference between the data in step 4.a and the current result, then determine the physical path names and the logical path names under /dev/dsk of the expanded LUNs.
The following example shows that "c3t3d0" is the logical path name of the expanded lun=0. Then the logical path names of lun=1 to 2 are "c3t3d1" and "c3t3d2" (cXtYdZ; X:PCI card number, Y:target ID, Z:lun number).
# /opt/FJSVhwr/sbin/drcstat -device > /tmp/drcstat.post <Return>
# diff /tmp/drcstat.pre /tmp/drcstat.post | grep hddv <Return>
:
> 00-PCI#0B "/pci@80,4000/fibre-channel@1/hddv@3,0" 10 "hddv"
> 00-PCI#0B "/pci@80,4000/fibre-channel@1/hddv@3,1" 11 "hddv"
> 00-PCI#0B "/pci@80,4000/fibre-channel@1/hddv@3,2" 12 "hddv"
# ls -l /dev/dsk | grep /pci@80,4000/fibre-channel@1/hddv@3,0 <Return>
lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s0
-> ../../devices/pci@80,4000/fibre-channel@1/hddv@3,0:a
lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s1
-> ../../devices/pci@80,4000/fibre-channel@1/hddv@3,0:b
lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s2
-> ../../devices/pci@80,4000/fibre-channel@1/hddv@3,0:c
lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s3
-> ../../devices/pci@80,4000/fibre-channel@1/hddv@3,0:d
lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s4
-> ../../devices/pci@80,4000/fibre-channel@1/hddv@3,0:e
lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s5
-> ../../devices/pci@80,4000/fibre-channel@1/hddv@3,0:f
lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s6
-> ../../devices/pci@80,4000/fibre-channel@1/hddv@3,0:g
lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s7
-> ../../devices/pci@80,4000/fibre-channel@1/hddv@3,0:h
4) Label a disk and perform the setting of partitions (slices) for the logical path names in step 8.b.3) by using the format(1M) command.
[ SCSI driver(Solaris: glm , FUJITSU PCI SCSI Adapter Driver) ]
No procedure is necessary.
[ Fibre Channel driver (PW008FC3U/PW008FC2U/ GP7B8FC1U) ]
No procedure is necessary.
[ Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*)]
Bind the auto mapped target connected with the card and modify the entries of lpfc.conf to match the requirements of the extended Fibre Channel Controller. These procedures are reflected on next boot time. In the following, indicate to bind the auto mapped target with the card.
| # /usr/sbin/lpfc/lputil <Return> LightPulse Common Utility for Solaris/SPARC. Version 1.6a8 (03/03/2004). Copyright (c) 2003, Emulex Network Systems, Inc. Emulex Fibre Channel Host Adapters Detected: 1 Host Adapter 0 (lpfc11) is an LP9802 (Ready Mode) MAIN MENU 1. List Adapters 2. Adapter Information 3. Firmware Maintenance 4. Reset Adapter 5. Persistent Bindings 0. Exit Enter choice => 5 <Return> PERSISTENT BINDINGS MENU 1. Display Current Bindings 2. Display All Nodes 3. Duplicate Adapter Bindings 4. Bind Target Manually 5. Bind Automapped Targets 6. Delete Binding(s) 0. Return to Main Menu Enter choice => 5 <Return> Automapped FCP Nodes: Node Target WWPN WWNN ---- ------ --------- -------- Automapped FCP Node 2 50-06-0e-80-00-10-04-92 50-06-0e-80-00-10-04-92 Press the Enter or Return key to continue: Bind all auto-mapped targets? (Y/N) => Y <Return> Select Binding Method: 1. By Node Name 2. By Port Name 3. By D_ID 0. Cancel Enter Choice => 2 <Return> |
Add the setting of the high-level products (services) by using the logical path name in step 8.b.
[Disk and Disk Array devices (in file system operation)]
| # newfs /dev/rdsk/c2t2d0s6 <Return> # vi /etc/vfstab <Return> : /dev/dsk/c2t2d0s6 /dev/rdsk/c2t2d0s6 /mnt ufs 1 yes - : |
[Disk and Disk Array devices (in raw access operation)/Tape devices]
[Disk and Disk Array devices (swap device)]
| # swap -a /dev/dsk/c2t2d0s4 <Return> # vi /etc/vfstab <Return> : /dev/dsk/c2t2d0s4 - - swap - no - : # swap -l <Return> swapfile dev swaplo blocks free : /dev/dsk/c2t2d0s4 32,164 16 788384 683680 |
Start the high-level products (services) by using the logical path name in step 8.b.
[Disk and Disk Array devices (in file system operation)]
| # mount -F ufs /dev/dsk/c2t2d0s6 /mnt <Return> |
[Disk and Disk Array devices (in raw access operation)/Tape devices]
[Disk and Disk Array devices (swap device)]
|
Contents
|