|
说明: 对于有两个或多个控制器的iscsi存储设备，或采用openfiler有两块网卡提供服务器的系统。需要配置多路径功能提高可用性。本文采用openfiler来实现iscsi设备的安装与测试。本文NAS采用openfiler 2.99.1 X64做为实验环境，为近期的一个备份环境部署做准备。本文出自：http://koumm.blog.51cto.com1. 并安装iSCSI启动器软件包rpm -ivh iscsi-initiator-utils-6.2.0.871-0.10.el5.i386.rpmchkconfig --level 2345 iscsi onchkconfig --level 2345 iscsid onservice iscsi startservice iscsid start2. 多路径软件安装准备# rpm -qa |grep mapperdevice-mapper-event-1.02.55-2.el5device-mapper-1.02.55-2.el5device-mapper-multipath-0.4.7-42.el5#设置成开机自启动multipathd# chkconfig --level 2345 multipathd on来检查安装是否正常# lsmod |grep dm_multipathdm_multipath 25421 0scsi_dh 12097 1 dm_multipathdm_mod 63097 11 dm_multipath,dm_raid45,dm_snapshot,dm_zero,dm_mirror,dm_log3. 发现存储设备# iscsiadm -m discovery -t st -p 192.168.100.238192.168.100.238:3260,1 iqn.2006-01.com.openfiler:tsn.f9d59aaffe9c192.168.100.239:3260,1 iqn.2006-01.com.openfiler:tsn.f9d59aaffe9c4. 连接两个节点，通过两个IP分别进行连接(1) 连接登陆第一个IP# iscsiadm -m node -T iqn.2006-01.com.openfiler:tsn.f9d59aaffe9c -p 192.168.100.238 -lLogging in to [iface: default, target: iqn.2006-01.com.openfiler:tsn.f9d59aaffe9c, portal: 192.168.100.238,3260]Login to [iface: default, target: iqn.2006-01.com.openfiler:tsn.f9d59aaffe9c, portal: 192.168.100.238,3260] successful.(2) 连接登录第二个IP# iscsiadm -m node -T iqn.2006-01.com.openfiler:tsn.f9d59aaffe9c -p 192.168.100.239 -lLogging in to [iface: default, target: iqn.2006-01.com.openfiler:tsn.f9d59aaffe9c, portal: 192.168.100.239,3260]Login to [iface: default, target: iqn.2006-01.com.openfiler:tsn.f9d59aaffe9c, portal: 192.168.100.239,3260] successful.5. 查看连接状态(1) 查看连接状态1# iscsiadm -m sessiontcp: [1] 192.168.14.180:3260,1 iqn.2006-01.com.openfiler:tsn.355d1613342dtcp: [2] 192.168.14.181:3260,1 iqn.2006-01.com.openfiler:tsn.355d1613342d(2) 查看iSCSI连接设备# ls -l /dev/disk/by-path/lrwxrwxrwx 1 root root 9 Jan 7 10:52 ip-192.168.100.238:3260-iscsi-iqn.2006-01.com.openfiler:tsn.f9d59aaffe9c-lun-0 -> ../../sdalrwxrwxrwx 1 root root 9 Jan 7 10:54 ip-192.168.100.239:3260-iscsi-iqn.2006-01.com.openfiler:tsn.f9d59aaffe9c-lun-0 -> ../../sdblrwxrwxrwx 1 root root 9 Jan 7 10:28 pci-0000:00:07.1-ide-0:0 -> ../../hdalrwxrwxrwx 1 root root 10 Jan 7 10:28 pci-0000:00:07.1-ide-0:0-part1 -> ../../hda1lrwxrwxrwx 1 root root 10 Jan 7 10:28 pci-0000:00:07.1-ide-0:0-part2 -> ../../hda2lrwxrwxrwx 1 root root 9 Jan 7 10:28 pci-0000:00:07.1-ide-1:0 -> ../../hdc(3) 查看详细连接状态# iscsiadm -m session -P 3iSCSI Transport Class version 2.0-871version 2.0-872Target: iqn.2006-01.com.openfiler:tsn.f9d59aaffe9c Current Portal: 192.168.100.238:3260,1 Persistent Portal: 192.168.100.238:3260,1 ********** Interface: ********** Iface Name: default Iface Transport: tcp Iface Initiatorname: iqn.1994-05.com.redhat:e2b7fdf48ccd Iface IPaddress: 192.168.100.175 Iface HWaddress: <empty> Iface Netdev: <empty> SID: 1 iSCSI Connection State: LOGGED IN iSCSI Session State: LOGGED_IN Internal iscsid Session State: NO CHANGE ************************ Negotiated iSCSI params: ************************ HeaderDigest: None DataDigest: None MaxRecvDataSegmentLength: 262144 MaxXmitDataSegmentLength: 131072 FirstBurstLength: 262144 MaxBurstLength: 262144 ImmediateData: No InitialR2T: Yes MaxOutstandingR2T: 1 ************************ Attached SCSI devices: ************************ Host Number: 0 State: running scsi0 Channel 00 Id 0 Lun: 0 Attached scsi disk sda State: running Current Portal: 192.168.100.239:3260,1 Persistent Portal: 192.168.100.239:3260,1 ********** Interface: ********** Iface Name: default Iface Transport: tcp Iface Initiatorname: iqn.1994-05.com.redhat:e2b7fdf48ccd Iface IPaddress: 192.168.100.175 Iface HWaddress: <empty> Iface Netdev: <empty> SID: 2 iSCSI Connection State: LOGGED IN iSCSI Session State: LOGGED_IN Internal iscsid Session State: NO CHANGE ************************ Negotiated iSCSI params: ************************ HeaderDigest: None DataDigest: None MaxRecvDataSegmentLength: 262144 MaxXmitDataSegmentLength: 131072 FirstBurstLength: 262144 MaxBurstLength: 262144 ImmediateData: No InitialR2T: Yes MaxOutstandingR2T: 1 ************************ Attached SCSI devices: ************************ Host Number: 1 State: running scsi1 Channel 00 Id 0 Lun: 0 Attached scsi disk sdb State: running两个设备sda,sdb分别由两个IP进行提供服务,也可查看状态都是LOGGED IN状态。6. 创建multipath配置文件(1) 生成multipath配置文件multipath默认配置文件/etc/multipath.conf。如果/etc/multipath.conf文件不存在，可以使用以下命令创建multipath.conf文件:mpathconf --enable(2) 生成磁盘设备的WWID号说明: 通常支持多路径的情况下会出现双份磁盘设备，依次执行以下命令。/sbin/scsi_id -g -u -s /block/sda/sbin/scsi_id -g -u -s /block/sdb# /sbin/scsi_id -g -u -s /block/sda14f504e46494c4552514a796e67642d786a4c642d42346271# /sbin/scsi_id -g -u -s /block/sdb14f504e46494c4552514a796e67642d786a4c642d42346271# iscsiadm -m sessiontcp: [1] 192.168.100.238:3260,1 iqn.2006-01.com.openfiler:tsn.f9d59aaffe9ctcp: [2] 192.168.100.239:3260,1 iqn.2006-01.com.openfiler:tsn.f9d59aaffe9c(3) 创建配置文件# mv /etc/multipath.conf /etc/multipath.conf.bak# grep -v '^#' /etc/multipath.conf.bak > /etc/multipath.conf# vi /etc/multipath.confblacklist { devnode "^hda"}defaults { user_friendly_names yes path_grouping_policy multibus failback immediate no_path_retry fail}multipaths { multipath { wwid 14f504e46494c4552514a796e67642d786a4c642d42346271 alias sda } multipath { wwid 14f504e46494c4552514a796e67642d786a4c642d42346271 alias sdb }}说明1: 默认multipath会把所有设备都加入到黑名单,HDA本地磁盘与光驱，需要排除掉。上面配置文件是因为本地磁盘是hda盘做的测试。说明2: 配置文件修改为类似下面的内容，这里禁止使用hda光驱,本地的sda磁盘。devnode_blacklist {devnode "^hda"wwid 14f504e46494c4552514a796e67642d786a4c642d48346537}(4) 启动服务# service multipathd start说明:采用多路径设备支持后,原来的/dev/sda,/dev/sdb设备名将不能再使用,改为/dev/mapper/mpathn,/dev/dm-n等名称。其中/dev/mapper/mpathn 是软件虚拟出来的多路径设备，这个可以被我们用来挂载使用。其中/dev/mapper/sda 是多路径设备别名，其中/dev/dm-n 这个是软件自身使用的，不可挂载使用。# multipath -llsda (14f504e46494c4552514a796e67642d786a4c642d42346271) dm-2 OPNFILER,VIRTUAL-DISK[size=2.0G][features=0][hwhandler=0][rw]\_ round-robin 0 [prio=2][active]\_ 0:0:0:0 sda 8:0 [active][ready]\_ 1:0:0:0 sdb 8:16 [active][ready]7. 挂载iSCSI设备(1). 分区说明: 由于采用multipath管理iscsi多路径设备后，实际的设备名称是/dev/mapper/sda, 可以再通过/dev/sda进行分区但是进行格式化时会出现问题。# fdisk /dev/sda或采用# fdisk /dev/mapper/sda进行分区n p 1 w(2). 格式化# mkfs.ext3 /dev/sda1mke2fs 1.39 (29-May-2006)/dev/sda1 is apparently in use by the system; will not make a filesystem here!1) 解决方法：出现以上提示时也可以采用如下方式解决。之后就可以进行格式化操作了。# dmsetup statusVolGroup00-LogVol01: 0 4128768 linearsdb: 0 8388608 multipath 2 0 0 0 2 1 A 0 1 0 8:16 A 0 E 0 1 0 8:32 A 0VolGroup00-LogVol00: 0 37552128 linear# dmsetup remove_all# dmsetup statusVolGroup00-LogVol01: 0 4128768 linearVolGroup00-LogVol00: 0 37552128 linear完成后需要重启multipath服务# service multipathd restart2) multipath基本操作命令# multipath -F #删除现有路径# multipath -ll #查看多路径3) 实现开机时网络挂载# vi /etc/fstab/dev/mapper/sdap1 /mnt/data ext3 _netdev 0 0# chkconfig netfs --level 35 on# service netfs start或采用如下方式进行挂载:------------------------------------------------------------查看磁盘的UUID# tune2fs -l /dev/sda1.....Filesystem UUID: c160af38-3928-42c8-9155-ac05a1d20538# vi /etc/fstabUUID=c160af38-3928-42c8-9155-ac05a1d20538 /mnt/data ext3 _netdev 0 0------------------------------------------------------------8. 故障处理(1) 通过对openfiler的双网卡进行故障测试openfiler系统上down掉eth0后# multipath -llsda: checker msg is "readsector0 checker reports path is down"sda (14f504e46494c4552514a796e67642d786a4c642d42346271) dm-2 OPNFILER,VIRTUAL-DISK[size=2.0G][features=0][hwhandler=0][rw]\_ round-robin 0 [prio=1][active]\_ 0:0:0:0 sda 8:0 [failed][faulty]\_ 1:0:0:0 sdb 8:16 [active][ready]openfiler系统上up 掉 eth0# multipath -llsda (14f504e46494c4552514a796e67642d786a4c642d42346271) dm-2 OPNFILER,VIRTUAL-DISK[size=2.0G][features=0][hwhandler=0][rw]\_ round-robin 0 [prio=2][active]\_ 0:0:0:0 sda 8:0 [active][ready]\_ 1:0:0:0 sdb 8:16 [active][ready]问题：多路径multipath在切换的时间会出现短时的无响应,multipath -ll命令会提示有一个路径当掉，多次切换会出现挂载的分区出现文件系统只读，重新挂载后正常。 [root@localhost iscsi]# cp aaa.txt ccc.txtcp: cannot create regular file `ccc.txt': Read-only file system(2) 删除失效的iscsi 连接：如果因为错误配置等原因创建了iscsi 对应，系统不会自动删除，需手动删除。例如：# iscsiadm -m node -o delete -T iqn.2006-01.com.openfiler:tsn.f9d59aaffe9c -p 192.168.100.238:3260
(以上内容不代表本站观点。) --------------------------------- |
本網站是"非商業"(non-commercial)。
