3.1.1 Replacement of non-redundant configuration

Dynamic Reconfiguration User's Guide I/O device edition - for Solaris (TM) Operating System -

3.1.1 Replacement of non-redundant configuration

When the PCI cards are needed to be replaced in non-redundant configuration without using software that controls the multipath, it is necessary to stop all the high-level products (services) as applications that are using these PCI cards on the system boards equipped with the PCI cards and the connected devices.

The replacement procedures of PCI cards for non-redundant configuration are below.

Stop the daemons
If you use the Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*), execute the following.

# /etc/rc0.d/K10ElxRMSrv stop <Return>
# /etc/rc0.d/K10ElxDiscSrv stop <Return>

Locating the PCI cards and the system boards

Using the following procedures, locate the system boards equipped with the faulty PCI cards and the all connected devices.

Locate the path interfaces connected the faulty PCI cards with I/O devices from the WARNING messages information output to the console.
The following example shows that glm2 is the path inteface connected the faulty PCI card with I/O device.

:
WARNING: /pci@80,4000/scsi@2 (glm2):
invalid intcode=fe00
:
WARNING: /pci@80,4000/scsi@2/sd@3,0 (sd20):
SCSI transport failed: reason 'reset': giving up
:

If you use the Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*), the messages are shown as follows.

:
WARNING: lpfc0:INe:Adapter Hardware Error
:WARNING: /pci@89,4000/fibre-channel@2/sd@3,0 (sd20):
SCSI transport failed: reason 'reset': giving up
:

Exec the following command and check the physical path name.

# grep lpfc /etc/path_to_inst <Return>
"/pci@89,4000/fibre-channel@2" 0 "lpfc"

The second value is instance number of each lpfc instance.

Locate the system board numbers for the path interfaces connected the faulty PCI cards with I/O devices in step 2.a (glm2) and display a list of devices under the system boards.
The system board number is the figure in the below red character part: XY (X represents the cabinet number, Y represents the system board slot number in the cabinet).
Also, it is understood that the hme0 of network device ("/pci@83,4000/network@1,1" 0 "hme") excluding file devices (sd1,sd20) exists on the relevant system board from the results of execution of the second command.
It is possible to ignore about se driver and scf driver (FJSVscfx) with the redundant configuration automatically.
Refer to "Enhanced Support Facility User's Guide" for details of se driver and scf driver.

# /opt/FJSVhwr/sbin/drcstat -device | grep /pci@80,4000/scsi@2 <Return>
00-PCI#0B "/pci@80,4000/scsi@2" 2 "glm"
00-PCI#0B "/pci@80,4000/scsi@2,1" 3 "glm"
00-PCI#0B "/pci@80,4000/scsi@2/sd@1,0" 1 "sd"
00-PCI#0B "/pci@80,4000/scsi@2/sd@3,0" 20 "sd"
# /opt/FJSVhwr/sbin/drcstat -device sb00 <Return>
00-PCI#0B "/pci@80,4000/scsi@2" 2 "glm"
00-PCI#0B "/pci@80,4000/scsi@2,1" 3 "glm"
00-PCI#0B "/pci@80,4000/scsi@2/sd@1,0" 1 "sd"
00-PCI#0B "/pci@80,4000/scsi@2/sd@3,0" 20 "sd"
00-ONBOARD "/pci@83,4000/ebus@1/FJSV,scfc@14,200000" 0 "FJSVscf2"
00-ONBOARD "/pci@83,4000/ebus@1/FJSV,se@14,400000" 0 "se"
00-ONBOARD "/pci@83,4000/network@1,1" 0 "hme"

In the following, the detach procedure is explained by taking sd20("/pci@80,4000/scsi@2/sd@3,0" 20 "sd") as an example.

Determine the logical path under /dev/dsk corresponding to the /devices side path of sd20.
The following example shows that "c2t3d0" is the logical path corresponding to sd20.

# ls -l /dev/dsk | grep /pci@80,4000/scsi@2/sd@3,0 <Return>
lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t3d0s0
-> ../../devices/pci@80,4000/scsi@2/sd@3,0:a
lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t3d0s1
-> ../../devices/pci@80,4000/scsi@2/sd@3,0:b
lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t3d0s2
-> ../../devices/pci@80,4000/scsi@2/sd@3,0:c
lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t3d0s3
-> ../../devices/pci@80,4000/scsi@2/sd@3,0:d
lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t3d0s4
-> ../../devices/pci@80,4000/scsi@2/sd@3,0:e
lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t3d0s5
-> ../../devices/pci@80,4000/scsi@2/sd@3,0:f
lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t3d0s6
-> ../../devices/pci@80,4000/scsi@2/sd@3,0:g
lrwxrwxrwx 1 root root 41 Sep 20 22:53 c2t3d0s7
-> ../../devices/pci@80,4000/scsi@2/sd@3,0:h

Stopping the high-level products (services)

Using the steps below, stop all the high-level products (services) that are using these PCI cards on the system boards equipped with the faulty PCI cards and the connected devices.
When connecting network devices that are using as step 2.b, refer to "Chapter 4 Network Devices" about how to stop them.

Please stop vold(1M).

# sh /etc/init.d/volmgt stop <Return>

Stop the use of the devices located in step 2.

[Disk devices (in file system operation)]

1) Determine the mount point of the target disk by using the logical path name (c2t3d0) in step 2.c.

# mount | grep c2t3d0 <Return>
/export/home on /dev/dsk/c2t3d0s3 setuid/read/write/largefiles on Mon Sep 30 01:00:51 2002
/develop/firm on /dev/dsk/c2t3d0s0 setuid/read/write/largefiles on Mon Sep 30 01:00:51 2002
/develop/drv on /dev/dsk/c2t3d0s1 setuid/read/write/largefiles on Mon Sep 30 01:00:51 2002
/pub on /dev/dsk/c2t3d0s6 setuid/read/write/largefiles on Mon Sep 30 01:00:50 2002

2) Stop access to the target disk. To examine the process that uses the target file system, use the fuser(1M) command as follows.

# fuser -c /export/home <Return>
/export/home: 14967c 14571c 14493ctm 14020c 13828tm 13803c 13575c 13133c 13125tm 13107c 12682ctm 12066tm 12048c 11971ctm 11952ctm 11937c 11867c 11846c 349m

3) Unmount the target disk.

# umount /export/home <Return>
# umount /develop/firm <Return>
# umount /develop/drv <Return>
# umount /pub <Return>

[Disk devices (in raw access operation) / Tape devices]

1) Check the access status of the target disk (sd20) in step 2.b up to now. This step is same for tape devices.

# iostat -xc <Return>
extended device statistics cpu
device r/s w/s kr/s kw/s wait actv svc_t %w %b us sy wt id
sd 59.7 7.5 474.5 45.0 0.0 3.9 58.6 0 41 3 7 23 67
sd1 0.1 0.3 1.0 2.5 0.0 0.0 16.0 0 0
sd20 0.0 0.1 0.3 0.7 0.0 0.0 14.7 0 0
st82 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
nfs1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0

2) Stop access to the target disk from high-level products.
The access status to the target disk can be checked in the following way, the access status for one minute is checked.
This step is same for tape devices.

# sar -d 60 1 <Return>

SunOS machine0 5.8 Generic_108528-05 sun4u 10/02/02

17:56:00 device %busy avque r+w/s blks/s avwait avserv
17:57:00 nfs1 0 0.0 0 0 0.0 0.0
sd0 2 0.3 2 37 0.0 145.5
sd0,a 1 0.1 0 5 0.0 301.4
sd0,b 0 0.0 0 9 0.0 31.5
sd0,c 0 0.0 0 0 0.0 0.0
sd0,d 1 0.0 0 4 0.0 126.8
sd0,e 0 0.0 0 0 0.0 0.0
sd0,f 1 0.1 1 6 0.0 120.9
sd0,g 1 0.1 1 14 0.0 111.7
:
sd20 0 0.0 0 0 0.0 0.0
sd20,a 0 0.0 0 0 0.0 0.0
sd20,c 0 0.0 0 0 0.0 0.0
sd20,g 0 0.0 0 0 0.0 0.0
:
st82 0 0.0 0 0 0.0 0.0
:

3) Stop (Detach) high-level products.
For details, refer to the manual for each product.

[Disk devices (swap device)]

1) Display the swap device list and check if the target disk of the logical path name (c2t3d0) in step 2.c is a swap device.

# swap -l <Return>
swapfile dev swaplo blocks free
/dev/dsk/c2t3d0s4 32,164 16 788384 683680

2) Delete the swap device.

# swap -d /dev/dsk/c2t3d0s4 <Return>

Detaching the system boards

Detach the system boards equipped with the faulty PCI cards by using the drc(1M) command, and check if the status of the system boards is changed to "Unconfigured".

When the high-level products do not stop completely, the drc(1M) command terminates abnormally. So go back to step 3 again and stop the high-level products.

# /opt/FJSVhwr/sbin/drc -disconnect sb00 -keep <Return>
# /opt/FJSVhwr/sbin/drcstat -board sb00 <Return>
SB BN Status (Detail) PID Next_PID Board_Type CPU(MHz)
-- -- --------------- --- -------- ---------- --------
00 0 Unconfigured 00 00 10 300

Replacing the PCI cards
Pull out the system boards detached in step 4 and replace the faulty PCI cards to replacement parts, then mount back the system boards and connect cable to devices. A certified service engineer takes charge of this work.When replacing the fibre channel PCI cards, the steps below are required.

[for PCI Fibre Channel(PW008FC3U/PW008FC2U/ GP7B8FC1U)]:

When using any of SAN management function of Systemwalker/StorageMGR, Softek SANView for ETERNUS (except for Vixel) and SP5000 SRM Facility

No procedure is necessary. Go to step 6.

When not using SAN management function of the above products

In case of replacing of the fibre channel cards of the below environment setting, it is necessary to change the setting of the fibre channel switch SN200 series and the disk array device ETERNUS3000/ETERNUS6000/GR700/800 series.
- Setting up the zoning by WWPN(World Wide Port Name) on SN200 series
- Using the host zoning (Host Affinity) function of ETERNUS3000/ETERNUS6000/GR700/800 series
For details, refer to the manual "Zoning User's Guide" and "ETERNUSmgr User Guide/GRmgr User Guide". If you use the Fibre Channel switch or disk array device other than described above, see the document of each product.

When changing the above setting, WWPNs (Figures of sixteen digits in all) of the replacement parts are needed. The WWPN of the replacement fibre channel card is determined by using eight-digit number printed on the seal of the card on front plate.
This figure of eight digit represents the lower eight digit of WWPN by the hexadecimal number. The upper eight digit of WWPN is 10000000 by a fixed hexadecimal number.
For instance, when the following seal puts on the replacement fibre channel card, the WWPN of the replacement card is 100000000e244061.

0e24
4061

[ for Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*)]:

To replace PCI cards with the following configurations, Fibre Channel switch and disk array device need to be reconfigured individually.
- If zone configuration by WWPN (World Wide Port Name) is done on Fibre Channel switch.
- If the Host Zoning function of disk array device is used.
For details, see the document of each product.

To perform the above reconfiguration, the WWPN(a 16-digit number) of the replacement card is needed. The WWPN of the PCI card can be known from the twelve characters shown on a label on the back of the card. These characters represent the bottom twelve digits of the WWPN in hexadecimal form. The top four digits are fixed to 1000 in hexadecimal form.

For example, if the following label is shown on the back of the card, the WWPN of the replacement card is 10000000c9366037.

IEEE:0000c9366037

Note

When the zoning setting is changed in SN200 series or other Fibre Channel switch, executing I/O requests to other devices sometimes terminate abnormally with temporary errors by the setting changes.
I/O requests to disk array devices and the like are recovered normally by retry processes, but backup processes of fibre channel tape devices sometimes terminate abnormally.
Please execute the change of the zoning setting after stopping backup processes.

Attaching the system boards

Using the steps below, attach the detached system board.

Attach the system boards equipped with the replaced PCI cards by using the drc(1M) command, and check if the status of the system boards is changed to "Configured".

# /opt/FJSVhwr/sbin/drc -connect sb00 <Return>
# /opt/FJSVhwr/sbin/drcstat -board sb00 <Return>
SB BN Status (Detail) PID Next_PID Board_Type CPU(MHz)
-- -- --------------- --- -------- ---------- --------
00 0 Configured 00 00 10 300

When replacing the fibre channel PCI cards, the steps below are required.

[for PCI Fibre Channel(PW008FC3U/PW008FC2U/ GP7B8FC1U)]:

When using any of SAN management function of Systemwalker/StorageMGR, Softek SANView for ETERNUS (except for Vixel) and SP5000 SRM Facility

1) Refresh current condition on management window.
SAN (Storage Area Network) management function searchs and indicates replaced PCI card.
Use "Inherit of Access Path" function on management window.
This function will adjust switch zoning and host affinity function of disk array device ETERNUS3000/ETERNUS6000/GR700/800 series with new WWPN of replaced PCI card.
See "Operation guide" or "User's Manual" of each product for details.

Note

When the zoning setting is changed in SN200 series, executing I/O requests to other devices sometimes terminate abnormally with temporary errors by the setting changes.
I/O requests to disk array devices and the like are recovered normally by retry processes, but backup processes of fibre channel tape devices sometimes terminate abnormally.
Please execute the change of the zoning setting after stopping backup processes.

2) Reattach the system boards equipped with the replaced PCI cards with changed zonig setting by using the following commands.

# /opt/FJSVhwr/sbin/drc -disconnect sb00 -keep <Return>
# /opt/FJSVhwr/sbin/drc -connect sb00 <Return>
# /opt/FJSVhwr/sbin/drcstat -board sb00 <Return>
SB BN Status (Detail) PID Next_PID Board_Type CPU(MHz)
-- -- --------------- --- -------- ---------- --------
00 0 Configured 00 00 10 300

If you don't use SAN management function of the above products

No procedure is necessary.

[ for Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*)]:

No procedure is necessary.

Restarting the high-level products (services)

Using the steps below, restart the stopped high-level products.
When stopping network devices in step 3, refer to "Chapter 4 Network Devices" about how to restart them.

Please start vold(1M).

# sh /etc/init.d/volmgt start <Return>

Restart the use of the devices stopped in step 3.b.

[Disk devices (in file system operation)]

Mount the unmounted file systems and restart the use.

# mount -F ufs /dev/dsk/c2t3d0s3 /export/home <Return>
# mount -F ufs /dev/dsk/c2t3d0s0 /develop/firm <Return>
# mount -F ufs /dev/dsk/c2t3d0s1 /develop/drv <Return>
# mount -F ufs /dev/dsk/c2t3d0s6 /pub <Return>

[Disk devices (in raw access operation) / Tape devices]

Reactivate high-level products and restart.
For details, refer to the manual for each product.

[Disk devices (swap device)]

Add the swap devices and restart the use.

# /sbin/swapadd -2 <Return>

Start the daemons
If you use the Fibre Channel Card (PW028FC3*/PW028FC4*/PW028FC5*), execute the following.

# /etc/rc2.d/S99ElxRMSrv start <Return>
# /etc/rc2.d/S99ElxDiscSrv start <Return>

Contents