| Dynamic Reconfiguration User's Guide I/O device edition - for Solaris (TM) Operating System - |
|
Contents
|
| Chapter 3 File Devices | > 3.2 Expansion Procedure | > 3.2.1 Solaris 8 |
This section shows the flow of the procedure of PCI cards expansion for existing redundant configuration to reinforce redundancy on Solaris 8 system by taking the following redundant software products as examples.
The expansion pattern of this section is not supported in MPHD.
In case of using the redundant software products except MPLB/GRMPD, refer to the manual for each product.
The procedures of the high-level products are not required for the expansion of PCI cards to existing redundant configuration.
In the following, the procedure is explained by taking the reserved definition of the expansion of disk array device controller (target=3 lun=0 to 2) to the existing disk array device as an example.
When the disk array devices are not defined, it is not possible to expand by using DR.
So please define them before system operation is started.
[ If you use Solaris:sd ]
| # vi /kernel/drv/sd.conf<Return> : name="sd" class="scsi" target=0 lun=0; : name="sd" class="scsi" <------ If undefined, add this definition target=3 lun=0; <------ or remove the comment symbols. name="sd" class="scsi" target=3 lun=1; name="sd" class="scsi" target=3 lun=2; : |
| # touch /reconfigure <Return> # /usr/sbin/shutdown -y -i6 -g0 <Return> |
[ If you use hddv ]
|
| # hddvadm -i <Return> # ls /kernel/drv/hddv.conf <Return> /kernel/drv/hddv.conf: No such file or directory |
|
| # vi /kernel/drv/sd.conf <Return> : name="sd" class="scsi" target=0 lun=0; : #name="sd" class="scsi" <------ target=1 lun=0; #name="sd" class="scsi" Delete or comment out target=1 lun=1; the duplicated definitions of sd and hddv drivers. #name="sd" class="scsi" target=1 lun=2; <------ : |
But it is impossible to delete the target IDs being recognized as a system volume a CD-ROM device etc. in sd driver.
In this case, change the target ID on hddv.conf and the disk array device side to unduplicated definition of sd.conf.
| # ls /kernel/drv/hddv.conf <Return> /kernel/drv/hddv.conf # vi /kernel/drv/hddv.conf <Return> : #name="hddv" class="scsi" target=0 lun=0; : #name="hddv" class="scsi" target=0 lun=7; : name="hddv" class="scsi" target=1 lun=0; <------ name="hddv" class="scsi" target=1 lun=1; Remove the comment symbols of using LUNs name="hddv" class="scsi" target=1 lun=2; <------ : name="hddv" class="scsi" target=3 lun=0; <------ name="hddv" class="scsi" target=3 lun=1; Remove the comment symbols of name="hddv" class="scsi" target=3 lun=2; <------ LUNs that are expected to expand #name="hddv" class="scsi" target=3 lun=3; : |
|
| # touch /reconfigure <Return> # /usrt/sbin/shutdown -y -i6 -g0 <Return> |
If you use the Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*), execute the following.
| # /etc/rc0.d/K10ElxRMSrv stop <Return> # /etc/rc0.d/K10ElxDiscSrv stop <Return> |
In case of expanding PCI cards on the existing system boards, choose the system boards of expansion target and detach the system boards according to step 3 to 4 in Section 3.1.1 or step 4 in Section 3.1.2.
This procedure is not required when expanding the new system boards together.
Using the following procedures, expand the PCI cards. A certified service engineer takes charge of this work.
In expansion of PCI cards to redundant configuration, it is possible to expand two or more PCI cards on different system boards, then excute from step 4 to 5 every expansion PCI cards.
When expanding the fibre channel PCI cards, the steps below are required.
[for PCI Fibre Channel(PW008FC3U/PW008FC2U/ GP7B8FC1U)]:
When using any of SAN management function of Systemwalker/StorageMGR, Softek SANView for ETERNUS (except for Vixel) and SP5000 SRM Facility
No procedure is necessary. Go to step 6.
When not using SAN management function of the above products
In case of replacing of the fibre channel cards of the below environment setting, it is necessary to change the setting of the fibre channel switch SN200 series and the disk array device ETERNUS3000/ETERNUS6000/GR700/800 series.
For details, refer to the manual "Zoning User's Guide" and "ETERNUSmgr User Guide/GRmgr User Guide". If you use the Fibre Channel switch or disk array device other than described above, see the document of each product.
When changing the above setting, WWPNs (Figures of sixteen digits in all) of the replacement parts are needed.The WWPN of the expansion fibre channel card is determined by using eight-digit number printed on the seal of the card on front plate.
This figure of eight digit represents the lower eight digit of WWPN by the hexadecimal number. The upper eight digit of WWPN is 10000000 by a fixed hexadecimal number.
For instance, when the following seal puts on the expansion fibre channel card, the WWPN of the expansion card is 100000000e244061.
| 0e24 4061 |
[ for Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*)]:
To replace PCI cards with the following configurations, Fibre Channel switch and disk array device need to be reconfigured individually.
For details, see the document of each product.
To perform the above reconfiguration, the WWPN(a 16-digit number) of the replacement card is needed. The WWPN of the PCI card can be known from the twelve characters shown on a label on the back of the card. These characters represent the bottom twelve digits of the WWPN in hexadecimal form. The top four digits are fixed to 1000 in hexadecimal form.
For example, if the following label is shown on the back of the card, the WWPN of the replacement card is 10000000c9366037.
| IEEE:0000c9366037 |
| # /opt/FJSVhwr/sbin/drcstat -device > /tmp/drcstat.pre <Return> |
Note
When the zoning setting is changed in SN200 series or other Fibre Channel switch, executing I/O requests to other devices sometimes terminate abnormally with temporary errors by the setting changes. I/O requests to disk array devices and the like are recovered normally by retry processes, but backup processes of fibre channel tape devices sometimes terminate abnormally. Please execute the change of the zoning setting after stopping backup processes.
Attach the system boards equipped with the expanded PCI cards by using the drc(1M) command, and check if the status of the system boards is changed to "Configured".
In case of detaching the system boards in step 3, restart the stopped high-level products and check the multipath status according to step 7 in Section 3.1.1 or step 6.c in Section 3.1.2.
| # /opt/FJSVhwr/sbin/drc -connect sb00 <Return> # /opt/FJSVhwr/sbin/drcstat -board sb00 <Return> SB BN Status (Detail) PID Next_PID Board_Type CPU(MHz) -- -- --------------- --- -------- ---------- -------- 00 0 Configured 00 00 10 300 |
If you use the Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*), execute the following.
| # /etc/rc2.d/S99ElxRMSrv start <Return> # /etc/rc2.d/S99ElxDiscSrv start <Return> |
When the zoning function is used on fiber channel switch SN200 series configuration, it is necessary to set zoning to the expanded devices.
[for PCI Fibre Channel(PW008FC3U/PW008FC2U/ GP7B8FC1U)]:
When using any of SAN management function of Systemwalker/StorageMGR, Softek SANView for ETERNUS (except for Vixel) and SP5000 SRM Facility
WWPN zoning of SN200 series, Host affinity of Array disk device ETERNUS3000/ETERNUS6000/GR700/800 series and binding setting for fjpfca.conf/sd.conf of host can be added from SAN management window automatically.
See "Operation guide" or "User's Manual" of each product for details.
Before using this function, the following step 8.a.1) and 8.a.2) [except fcp-bind-target setting] are necessary.
Moreover, begin from step 8.a.3) after executing this function.
If you don't use SAN management function of the above products
No procedure is necessary.
[ for Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*)]:
No procedure is necessary.
Note
When the zoning setting is changed in SN200 series, executing I/O requests to other devices sometimes terminate abnormally with temporary errors by the setting changes.
I/O requests to disk array devices and the like are recovered normally by retry processes, but backup processes of fibre channel tape devices sometimes terminate abnormally.
Please execute the change of the zoning setting after stopping backup processes.
Using the steps below, add the setting of drivers.
For file drivers, it is necessary to add the setting of PCI card drivers and device drivers.
[SCSI drivers (Solaris:glm)]
[PCI Fibre Channel(PW008FC3U/PW008FC2U/GP7B8FC1U)]
1) Save the results of execution of the drcstat(1M) command in a file, and find a difference between the data in step 4.a and the current result, then determine the instance numbers of fjpfca driver for the expanded PCI cards.
The following example shows that fjpfca1 (instance number: 1) is added.
If the relation between the instance number and the slot position is necessary, locate it by blinking LED of the expanded PCI card with using the fc_adm command.
Also, check by using the "fc_info -a" command about the relation between the WWPN of expanded card and the instance number.
# /opt/FJSVhwr/sbin/drcstat -device > /tmp/drcstat.pfca <Return>
# diff /tmp/drcstat.pre /tmp/drcstat.pfca | grep fjpfca <Return>
:
> 00-PCI#0B "/pci@80,4000/fibre-channel@1" 1 "fjpfca"
:
# /usr/sbin/FJSVpfca/fc_adm -l fjpfca1 <Return>
2) Edit the setting file "/kernel/drv/fjpfca.conf" and add the below definitions of property for instance numbers in step 8.a.1).port (topology types, link speed)
In case of expanding the PCI cards of fabric connection, define this property.
In FC-AL connection, this definition is not required.fcp-bind-target (binding target ID and WWN)
In case of expanding the disk array devices of fabric connection, define this property.
In FC-AL connection, this definition is not required.max-throttle (the number of maximum SCSI command issues)
This is the property to limit the number of SCSI commands (tag) that can be accepted at the same time.
The necessity and setting value of this property depend on the specification of the disk array devices, so refer to the manual of each disk array device.
The following example is the setting addition of the PCI card for fjpfca1 (instance number: 1) and the disk array device of target=3.
# vi /kernel/drv/fjpfca.conf <Return>
:
port=
"fjpfca0:nport",
"fjpfca1:nport:sp1"; <------ added
fcp-bind-target=
"fjpfca0t0:0x50000e1000002501",
"fjpfca1t3:0x50000e1000002ab0"; <------ added
max-throttle=
"fjpfca0t0:128",
"fjpfca1t3:240"; <------ added
:
3) Notify the fjpfca driver of the attachment of the expansion parts by using the following command.
# /usr/sbin/FJSVpfca/fc_config -v <Return>
[ for Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*)]
No procedure is necessary.
Compare the output of prtpicl(1M) command with the output saved at 4.b and find the instance number of the added card.
In the following, the procedure is explained by taking the expansion in step 7 after the reserved definition of the expansion of disk array device controller (target=3 lun=0 to 2) to the existing disk array device in step 1 as an example.
[Solaris : sd ]
1) Execute the following command to make the sd driver set up the added disk device.
# /usr/sbin/devfsadm <Return>
2) After saving the execution result of the prtpicl(1M) command, take the difference between the result and the information obtained in 4.a. and get the physical path name of additional disk device, then get the logical path name under /dev/dsk. The following example shows that "c3t3d0" and "c4t3d0" are the logical path names of the expanded lun=0. Then the logical path names of lun=1 to 2 are "c3t3d1", "c4t3d1","c3t3d2" and "c4t3d2" (cXtYdZ; X:PCI card number, Y:target ID, Z:lun number).
# prtpicl -v > /tmp/prtpicl.post <Return>
# diff /tmp/prtpicl.pre /tmp/prtpicl.post | grep devfs-path | grep hddv <Return>
> :devfs-path /pci@8d,2000/fibre-channel@1/sd@3,0
> :devfs-path /pci@8d,2000/fibre-channel@1/sd@3,1
> :devfs-path /pci@8d,2000/fibre-channel@1/sd@3,2
> :devfs-path /pci@89,2000/fibre-channel@1/sd@3,0
> :devfs-path /pci@89,2000/fibre-channel@1/sd@3,1
> :devfs-path /pci@89,2000/fibre-channel@1/sd@3,2
# ls -l /dev/dsk | grep /pci@8d,2000/fibre-channel@1/sd@3,0 <Return>
lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s0
-> ../../devices/pci@8d,2000/fibre-channel@1/sd@3,0:a
lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s1
-> ../../devices/pci@8d,2000/fibre-channel@1/sd@3,0:b
lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s2
-> ../../devices/pci@8d,2000/fibre-channel@1/sd@3,0:c
lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s3
-> ../../devices/pci@8d,2000/fibre-channel@1/sd@3,0:d
lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s4
-> ../../devices/pci@8d,2000/fibre-channel@1/sd@3,0:e
lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s5
-> ../../devices/pci@8d,2000/fibre-channel@1/sd@3,0:f
lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s6
-> ../../devices/pci@8d,2000/fibre-channel@1/sd@3,0:g
lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s7
-> ../../devices/pci@8d,2000/fibre-channel@1/sd@3,0:h
# ls -l /dev/dsk | grep /pci@89,2000/fibre-channel@1/sd@3,0 <Return>
lrwxrwxrwx 1 root root 41 Sep 20 22:53 c4t3d0s0
-> ../../devices/pci@89,2000/fibre-channel@1/sd@3,0:a
:
3) Label a disk and perform the setting of partitions (slices) for the logical path names in step 6.b.3) by using the format(1M) command.
[ Disk array device (hard disk driver : hddv) ]
1) Check if no duplicate definitions exist in the sd.conf and hddv.conf files again, because the situation could be changed since step 1.
The message example in "Check NG" shows the duplicate definition, so go back to step 1.d.
- [Check OK] --> go to step 8.b.2)
# /opt/FJSVhddv/bin/hddvchk_conf <Return>
/kernel/drv/sd.conf and /kernel/drv/hddv.conf seems to be correct- [Check NG] --> go back to step 1.d
# /opt/FJSVhddv/bin/hddvchk_conf <Return>
NG ID = [ 3 ]
2) Notify the hard disk driver of the attachment of the expanded LUNs by using the following command.
# /usr/sbin/hddvadmdr <Return>
3) Save the results of execution of the drcstat(1M) command in a file, and find a difference between the data in step 4.a and the current result, then determine the physical path names and the logical path names under /dev/dsk of the expanded LUNs.
The following example shows that "c3t3d0" is the logical path name of the expanded lun=0. Then the logical path names of lun=1 to 2 are "c3t3d1" and "c3t3d2" (cXtYdZ; X:PCI card number, Y:target ID, Z:lun number).
# /opt/FJSVhwr/sbin/drcstat -device > /tmp/drcstat.post <Return>
# diff /tmp/drcstat.pre /tmp/drcstat.post | grep hddv <Return>
:
> 00-PCI#0B "/pci@80,4000/fibre-channel@1/hddv@3,0" 10 "hddv"
> 00-PCI#0B "/pci@80,4000/fibre-channel@1/hddv@3,1" 11 "hddv"
> 00-PCI#0B "/pci@80,4000/fibre-channel@1/hddv@3,2" 12 "hddv"
# ls -l /dev/dsk | grep /pci@80,4000/fibre-channel@1/hddv@3,0 <Return>
lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s0
-> ../../devices/pci@80,4000/fibre-channel@1/hddv@3,0:a
lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s1
-> ../../devices/pci@80,4000/fibre-channel@1/hddv@3,0:b
lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s2
-> ../../devices/pci@80,4000/fibre-channel@1/hddv@3,0:c
lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s3
-> ../../devices/pci@80,4000/fibre-channel@1/hddv@3,0:d
lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s4
-> ../../devices/pci@80,4000/fibre-channel@1/hddv@3,0:e
lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s5
-> ../../devices/pci@80,4000/fibre-channel@1/hddv@3,0:f
lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s6
-> ../../devices/pci@80,4000/fibre-channel@1/hddv@3,0:g
lrwxrwxrwx 1 root root 41 Sep 20 22:53 c3t3d0s7
-> ../../devices/pci@80,4000/fibre-channel@1/hddv@3,0:h
Label a disk and perform the setting of partitions (slices) for the logical path names in step 8.b.3) by using the format(1M) command.
It is sufficient to execute for one representative logical path name of each expanded LUN, so need not to execute for every all PCI cards.
[ SCSI driver(Solaris: glm , FUJITSU PCI SCSI Adapter Driver) ]
No procedure is necessary.
[ Fibre Channel driver (PW008FC3U/PW008FC2U/ GP7B8FC1U) ]
No procedure is necessary.
[ Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*)]
Bind the auto mapped target connected with the card and modify the entries of lpfc.conf to match the requirements of the extended Fibre Channel Controller. These procedures are reflected on next boot time. In the following, indicate to bind the auto mapped target with the card.
| # /usr/sbin/lpfc/lputil <Return> LightPulse Common Utility for Solaris/SPARC. Version 1.6a8 (03/03/2004). Copyright (c) 2003, Emulex Network Systems, Inc. Emulex Fibre Channel Host Adapters Detected: 2 Host Adapter 0 (lpfc11) is an LP9802 (Ready Mode) Host Adapter 1 (lpfc9) is an LP9802 (Ready Mode) MAIN MENU 1. List Adapters 2. Adapter Information 3. Firmware Maintenance 4. Reset Adapter 5. Persistent Bindings 0. Exit Enter choice => 5 <Return> PERSISTENT BINDINGS MENU 1. Display Current Bindings 2. Display All Nodes 3. Duplicate Adapter Bindings 4. Bind Target Manually 5. Bind Automapped Targets 6. Delete Binding(s) 0. Return to Main Menu Enter choice => 5 <Return> 0. lpfc11 1. lpfc9 Select an adapter => 0<Return> Automapped FCP Nodes: Node Target WWPN WWNN ---- ------ --------- -------- Automapped FCP Node 3 50-06-0e-80-00-10-04-92 50-06-0e-80-00-10-04-92 Press the Enter or Return key to continue: Bind all auto-mapped targets? (Y/N) => Y <Return> Select Binding Method: 1. By Node Name 2. By Port Name 3. By D_ID 0. Cancel Enter Choice => 2 <Return> |
Attach the paths connected expanded PCI cards with disk array device controllers to the existing redundant configuration by using the following command, and check if the logical path names in step 8.b.3) are added (the below red character strings part). The following procedure is described for MPLB/GRMPD. If you use the redundancy software other than MPLB/GRMPD, see the document for each product.
|
|
Contents
|