Tuesday, June 16, 2009

Why the WWN Node numbers are different on the physically same card?

Official answer from both Emulex and Qlogic is:

Only targets (Arrays/VTLs/Filers) follow the "same Node WWN on all public FC
ports", initiators (HBA cards) don't follow this convention/rule.

So having different Node WWNs on the physically same HBA is "ok".

Monday, June 15, 2009

How many nics available on OS and link caps

#kstat -c net

In Solaris 10 you can show nics by used following command
#dladm show-dev

How to Setup root Solstice DiskSuite / Solaris Volume Manager

Common to both options below:
1. Create identical partitioning on the second disk
dd if=/dev/rdsk/c0t0d0s2 of=/dev/rdsk/c0t1d0s2 count=16
Or
prtvtoc /dev/rdsk/c0t0d0s2|fmthard -s - /dev/rdsk/c0t1d0s2
2. Setup the state databases with 3 backups per slice on each disk(create s7 before create new metadevice state database on s7)
metadb -a -f -c 3 c0t0d0s7 c0t1d0s7

For all slices except root
--------------------------
Description Mirror Name Device Name
1st mirror d4 c0t0d0s4
2nd mirror d5 c0t1d0s4
Metamirror d3 /dev/md/dsk/d3

1. Create submirrors for the mirrors
a) metainit -f d4 1 1 c0t0d0s4
b) metainit d3 -m d4
c) metainit d5 1 1 c0t1d0s4
2. Make a backup of the /etc/vfstab file before editing it
3. Make the following change in the /etc/vfstab file
/dev/md/dsk/d3 /dev/md/rdsk/d3 /usr ufs 1 no -
4. Reboot
5. Attach the submirror to the metamirror
metattach d3 d5 (NOTE: There will be lots of disk I/O)

For the root slice only
-----------------------
Description Mirror Name Device Name
1st mirror d1 c0t0d0s0
2nd mirror d2 c0t1d0s0
Metamirror d0 /dev/md/dsk/d0

1. Create submirrors for the mirrors
a) metainit -f d1 1 1 c0t0d0s0
b) metainit d0 -m d1
c) metainit d2 1 1 c0t1d0s0
2. Make a backup of the /etc/vfstab file
3. Let the metaroot command make the /etc/vfstab and /etc/system changes for you
metaroot d0
4. Run lockfs to prevent problems
lockfs -fa
5. Shutdown the server
/usr/sbin/shutdown -y -g0 -i0
6. From the boot prompt, run
show-disks
7. Pick your mirrored disk from the list and then setup an alias like so
nvalias mirror ^y (that is a -y to paste the device path)
8. Change your boot-device to first try the normal disk alias, then use your mirror disk
setenv boot-device disk mirror
9. Reset/reboot the server
reset-all
10. Attach the submirror to the metamirror
metattach d0 d2 (NOTE: There will be lots of disk I/O)
11. Do an ls and copy the info down as this is the alternate boot path
ls -l /dev/rdsk/c0t1d0s0


If you only have two mirrored root disks (DiskSuite 4.2.1, Solaris 8), put this setting in your /etc/system:
set md:mirrored_root_flag=1

For more information, go check out this handy tip:
http://www.sysunconfig.net/unixtips/ignore_quorum.txt

Solaris:-Check boot device and argument that OS current is running

#prtconf -pv|grep boot
bootarchive: '/ramdisk-root'
bootfs: fe7df548
bootargs: 00
bootpath: '/pci@1c,600000/SUNW,qlc@1/fp@0,0/disk@w5006048c53684c1e,1e:a'

error-reset-recovery: 'boot'
boot-command: 'boot'
boot-file:
boot-device: '/pci@1c,600000/SUNW,qlc@1/fp@0,0/disk@w5006048c53684c1e,1e'
auto-boot?: 'false'
boot-disk: '/pci@1d,700000/pci@1/scsi@2/disk@0,0:a'
boot-mirr: '/pci@3d,700000/pci@1/scsi@2/disk@0,0:a'
third-boot: '/pci@1d,700000/pci@1/scsi@2/disk@2,0:a'
name: 'bootbus-controller'
fc-boot-dev-portid: 00041000
fc-boot-dev-portwwn: 5006048c.53684c1e ---> Target Port (EMC FA wwpn)
name: 'bootbus-controller'

Check wwpn/wwnn on Solaris and select boot device for boot san

IN OBP:-
{a2} ok show-disks
a) /ramdisk-root
b) /pci@3d,600000/SUNW,qlc@1/fp@0,0/disk
c) /pci@3d,700000/pci@1/scsi@2,1/disk
d) /pci@3d,700000/pci@1/scsi@2/disk
e) /pci@3c,600000/SUNW,qlc@1/fp@0,0/disk
f) /pci@1d,600000/SUNW,qlc@1/fp@0,0/disk
g) /pci@1d,700000/pci@1/scsi@2,1/disk
h) /pci@1d,700000/pci@1/scsi@2/disk
i) /pci@1c,600000/SUNW,qlc@1/fp@0,0/disk
q) NO SELECTION
Enter Selection, q to quit: q
{a2} ok
{a2} ok cd /pci@3c,600000/SUNW,qlc@1
{a2} ok .properties
assigned-addresses 81000810 00000000 00000300 00000000 00000100
82000814 00000000 00100000 00000000 00002000
82000830 00000000 00120000 00000000 00020000
port-wwn 21 00 00 e0 8b 11 b0 94
node-wwn 20 00 00 e0 8b 11 b0 94

reg 00000800 00000000 00000000 00000000 00000000
01000810 00000000 00000000 00000000 00000100
02000814 00000000 00000000 00000000 00001000
compatible pci1077,2300.1077.106.1
pci1077,2300.1077.106
pci1077,106
pci1077,2300.1
pci1077,2300
pciclass,0c0400
pciclass,0c04
manufacturer QLGC
#size-cells 00000000
#address-cells 00000002
device_type scsi-fcp
name SUNW,qlc
version ISP2300 Host Adapter Driver: 1.14.01 12/20/02
fcode-rom-offset 00000000
66mhz-capable
fast-back-to-back
devsel-speed 00000001
class-code 000c0400
interrupts 00000001
latency-timer 00000040
cache-line-size 00000010
max-latency 00000000
min-grant 00000040
subsystem-id 00000106
subsystem-vendor-id 00001077
revision-id 00000001
device-id 00002300
vendor-id 00001077
{a2} ok
{a0} ok select /pci@1c,600000/SUNW,qlc@1
{a0} ok show-children
************************* Fabric Attached Devices ************************
Adapter portId - 41300
Device PortId 41000 DeviceId 1 WWPN 5006048c53684c1e
Lun 0 DISK EMC SYMMETRIX 5771
Lun 9 DISK EMC SYMMETRIX 5771
Lun a DISK EMC SYMMETRIX 5771
Lun b DISK EMC SYMMETRIX 5771
Lun c DISK EMC SYMMETRIX 5771
Lun d DISK EMC SYMMETRIX 5771
Lun e DISK EMC SYMMETRIX 5771
Lun f DISK EMC SYMMETRIX 5771
Lun 10 DISK EMC SYMMETRIX 5771
Lun 1b DISK EMC SYMMETRIX 5771
Lun 1d DISK EMC SYMMETRIX 5771
Lun 1e DISK EMC SYMMETRIX 5771
Lun 1f DISK EMC SYMMETRIX 5771
Lun 20 DISK EMC SYMMETRIX 5771
Lun 21 DISK EMC SYMMETRIX 5771
Lun 22 DISK EMC SYMMETRIX 5771
Lun 23 DISK EMC SYMMETRIX 5771
{a0} ok

{a0} ok boot /pci@1c,600000/SUNW,qlc@1/fp@0,0/disk@w5006048c53684c1e,1e
Notes:- disk@w5006048c53684c1e,1e
5006048c53684c1e is wwpn of EMC FA Port
1e is lun id in hex number it can translate to d30 in decimal



IN OS Level:-
-Sol9 and older make sure process picld is running. You can start it by below command.

#/etc/init.d/picld start
#ps -efgrep picld
root 28707 1 0 Jun 13 ? 0:01 /usr/lib/picl/picld

#/usr/sbin/prtpicl -v -c scsi-fcp grep wwn
:node-wwn 20 00 00 e0 8b 11 64 76
:port-wwn 21 00 00 e0 8b 11 64 76
:node-wwn 20 00 00 e0 8b 11 53 77
:port-wwn 21 00 00 e0 8b 11 53 77

-Sol10
#fcinfo hba-port
HBA Port WWN: 210000e08b111878
OS Device Name: /dev/cfg/c0
Manufacturer: QLogic Corp.
Model: 375-3102-xx
Firmware Version: 3.03.27
FCode/BIOS Version: fcode: 1.13;
Serial Number: not available
Driver Name: qlc
Driver Version: 20081115-2.29
Type: N-port
State: online
Supported Speeds: 1Gb 2Gb
Current Speed: 2Gb
Node WWN: 200000e08b111878

HBA Port WWN: 210000e08b11b094
OS Device Name: /dev/cfg/c2
Manufacturer: QLogic Corp.
Model: 375-3102-xx
Firmware Version: 3.03.27
FCode/BIOS Version: fcode: 1.13;
Serial Number: not available
Driver Name: qlc
Driver Version: 20081115-2.29
Type: N-port
State: online
Supported Speeds: 1Gb 2Gb
Current Speed: 2Gb
Node WWN: 200000e08b11b094

#ls -l /dev/cfg/c[0,2]
lrwxrwxrwx 1 root root 48 Jun 11 10:16 /dev/cfg/c0 -> ../../devices/pci@1c,600000/SUNW,qlc@1/fp@0,0:fc
lrwxrwxrwx 1 root root 48 Jun 11 10:16 /dev/cfg/c2 -> ../../devices/pci@3c,600000/SUNW,qlc@1/fp@0,0:fc


If you install EMC Solution Enabler then you can use syminq command to list the HBA information

#syminq hba -fibre
#luxadm -e port

Found path to 2 HBA ports

/devices/pci@dc,600000/SUNW,qlc@1/fp@0,0:devctl CONNECTED
/devices/pci@fc,600000/SUNW,qlc@1/fp@0,0:devctl CONNECTED
#luxadm -e dump_map /devices/pci@dc,600000/SUNW,qlc@1/fp@0,0:devctl
Pos Port_ID Hard_Addr Port WWN Node WWN Type
0 31000 0 5006048c53684c11 5006048c53684c11 0x0 (Disk device) ---> Target wwn (EMC FA)
1 31100 0 210000e08b116476 200000e08b116476 0x1f (Unknown Type,Host Bus Adapter) ---> HBA wwn
#luxadm -e dump_map /devices/pci@fc,600000/SUNW,qlc@1/fp@0,0:devctl
Pos Port_ID Hard_Addr Port WWN Node WWN Type
0 41000 0 5006048c53684c1e 5006048c53684c1e 0x0 (Disk device)
1 41100 0 210000e08b115377 200000e08b115377 0x1f (Unknown Type,Host Bus Adapter)

check recursive all links and its destination within a certainly directory

#find / -type l -ls

Or

#ls -alR grep "^"l