http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 1 [Check]
20 ○
30 ○
40 ○
50 ○
60 ○
70 ○
80 ○
90 ○
100 ○
110 ○
120 ○
130 ○
140 ○
150 ○
160 ○
170 ○
180 ○
190 ○
200 ○
210 ○
220 ○
230 ○
240 ○
250 ○
260 ○
270 ○
280 ○
290 ○
300 ○
310 ○
320 ○
330 ○
340 ○
350 ○
360 ○
370 ○
380 ○
390 ○
400 ○
410 ○
Ver.1.20
2016/12/17
【想定するサーバのスペックとネットワーク構成】
CPU: 2Core 以上
Memory: 2GB 以上
※　この手順書では、 2GB にて例示
DVD ドライブ: 1台
HDD 1: 8GB 以上 （1st Disk, OS 用）
※　この手順書では、 40GB にて例示
HDD 2: 1GB 以上 （swap 用）
※　この手順書では、 1GB にて例示
HDD 3: 1GB 以上 （2nd Disk, データ用） ※　この手順書では、100GB にて例示
HDD 4: 1GB 以上 （3rd Disk, データ用） ※　この手順書では、100GB にて例示
HDD 5: 1GB 以上 （4th Disk, データ用） ※　この手順書では、100GB にて例示
HDD 6: 1GB 以上 （5th Disk, データ用） ※　この手順書では、100GB にて例示
NIC 1: 1Gbps 以上、サービス用セグメント
（IBM Bluemix(SoftLayer) では
NIC 2: 1Gbps 以上、インターコネクト用セグメント（IBM Bluemix(SoftLayer) では
NIC 3: 1Gbps 以上、サービス用セグメント
（IBM Bluemix(SoftLayer) では
NIC 4: 1Gbps 以上、インターコネクト用セグメント（IBM Bluemix(SoftLayer) では
(eth1,eth3)bond1
.2
iscsitgt01a.example.com
1st Disk: OS
クラスタ ID: 1
Oracle Linux 7.3
Disk
Disk
Disk
Disk
Oracle Linux 7.3
LVM(vg0)
DRBD
LVM(vg1)
LVM(vg0)
DRBD
LVM(vg1)
iSCSI
lun0,1,2,3
Snapshot
.57
(eth0,eth2)bond0
VLAN）へ接続
VLAN）へ接続
VLAN）へ接続
VLAN）へ接続
bond1(eth1,eth3)
.3
iscsitgt01s.example.com
Swap Disk
2nd
3rd
4th
5th
Private
Public
Private
Public
1st Disk: OS
/dev/sda
Swap Disk
/dev/sdb
2nd
3rd
4th
5th
/dev/sdc
/dev/sdd
/dev/sde
/dev/sdf
iSCSI
lun0,1,2,3
Snapshot
.58
VIP
bond0(eth0,eth2)
←---.59 -------------------→
iscsitgt01.example.com
[インターコネクト・セグメント]
192.168.1.0/24
MTU: 9000
Disk
Disk
Disk
Disk
Router
.1
InitiatorName:
initiator01
initiator02
initiator03
initiator04
NTP1: 10.0.77.54
NTP2:
NTP3:
NTP4:
DNS1: 10.0.80.11
DNS2: 10.0.80.12
[サービス・セグメント]
10.110.88.0/26
MTU: 9000
※　当文書内で緑色にした部分は、環境に合わせて読み替えたり、カスタマイズ（名前を変えたり、実行するしないを選択）
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 2 420
430
440
450
460
470
480
490
500
510
520
530
540
550
560
570
580
590
600
610
620
630
640
650
660
670
○
する部分を表します。ただし、日付や注目していない UUID 等は除きます。
※　IBM Bluemix(SoftLayer) のベアメタルサーバで NIC を冗長化した場合、
「NIC 1 (eth0)」と「NIC 3 (eth2)」、「NIC 2 (eth1)」と「NIC 4 (eth3)」が LAG で束ねられています。
※　本手順書では、LAG の設定がない前提としますが、LAG 対応させるための設定方法は注記しておきます。
※　IBM Bluemix(SoftLayer) の仮想サーバは、ベアメタルサーバと比較して、主に以下の相違点があります。
・「NIC 3」と「NIC 4」を追加できません。
・MTU は 1500 までしかサポートされません。
・ローカルストレージのデバイス名が異なります。
○
【共有ストレージの構成】
○
○
○
○
○
○
○
○
○
○
○
○
/dev/sdc
/dev/sdd
/dev/sde
/dev/sdf
vg0
/dev/vg0/lv-drbd0
/dev/drbd0
vg1
/dev/vg1/lv-lun0000
/dev/vg1/lv-lun0001
/dev/vg1/lv-lun0002
/dev/vg1/lv-lun0003
LVM 物理ボリューム
LVM 物理ボリューム
LVM 物理ボリューム
LVM 物理ボリューム
LVM ボリュームグループ
LVM 論理ボリューム（DRBD 用ブロックデバイスとして使用）
DRBD リソース（LVM 物理ボリュームとして使用）
DRBD 上のボリュームグループ（「vg」+「クラスタ ID」）
DRBD 上の LVM 論理ボリューム（LUN としてエクスポート）
DRBD 上の LVM 論理ボリューム（LUN としてエクスポート）
DRBD 上の LVM 論理ボリューム（LUN としてエクスポート）
DRBD 上の LVM 論理ボリューム（LUN としてエクスポート）
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 3 680
690
700
710
720
730
740
750
760
770
780
790
800
810
820
830
840
850
860
870
880
890
900
910
920
930
940
950
960
970
980
990
1000
1010
1020
1030
1040
1050
1060
1070
1080
○
【OS のインストールと初期設定】
○
インストーラを DVD ドライブにセットし、サーバを起動します。
a,s
○
a,s
○
a,s
a,s
a,s
# V834394-01.iso (Oracle Linux 7.3)
インストーラの起動メニューが表示されたら60秒以内に「Tab」キーを押下します。
# Tab
起動オプションを以下のように編集し、「Enter」キーを押下します。
# vmlinuz … rd.live.check quiet
# ↓
# vmlinuz … net.ifnames=0 biosdevname=0 selinux=0 vconsole.keymap=jp106
※　英語キーボードとして認識されている状態なので、「=」を入力するには「^」を押下します。
○
a,s
○
a,s
○
a,s
anaconda の「Welcome」画面が出てきたら「Ctrl + Alt + F3」キーを押下し、シェルに移行します。
# Ctrl + Alt + F3
[anaconda root@localhost /]#
HDD の情報を確認します。
fdisk -l | grep ^Disk | sort
Disk /dev/mapper/live-base: 2147 MB, 2147483648 bytes, 4194304 sectors
Disk /dev/mapper/live-rw: 2147 MB, 2147483648 bytes, 4194304 sectors
Disk /dev/sda: 42.9 GB, 17179869184 bytes, 33554432 sectors
Disk /dev/sdb: 1073 MB, 1073741824 bytes, 2097152 sectors
Disk /dev/sdc: 107.4 GB, 107374182400 bytes, 209715200 sectors
Disk /dev/sdd: 107.4 GB, 107374182400 bytes, 209715200 sectors
Disk /dev/sde: 107.4 GB, 107374182400 bytes, 209715200 sectors
Disk /dev/sdf: 107.4 GB, 107374182400 bytes, 209715200 sectors
パーティションを作成します。
fdisk -H 64 -S 32 /dev/sda
Welcome to fdisk (util-linux 2.23.2).
Changes will remain in memory only, until you decide to write them.
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 4 1090
1100
1110
1120
1130
1140
1150
1160
1170
1180
1190
1200
1210
1220
1230
1240
1250
1260
1270
1280
1290
1300
1310
1320
1330
1340
1350
1360
1370
1380
1390
1400
1410
1420
1430
1440
1450
1460
1470
1480
1490
Be careful before using the write command.
Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0x2a058c02.
a,s
Command (m for help): o
Building a new DOS disklabel with disk identifier 0xc9c2368a.
a,s
Command (m for help): n
Partition type:
p primary (0 primary, 0 extended, 4 free)
e extended
Select (default p): [Enter]
Using default response p
Partition number (1-4, default 1): [Enter]
First sector (2048-83886079, default 2048): [Enter]
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-83886079, default 83886079): +500M
Partition 1 of type Linux and of size 500 MiB is set
a,s
a,s
a,s
a,s
a,s
Command (m for help): a
Selected partition 1
a,s
Command (m for help): n
Partition type:
p primary (1 primary, 0 extended, 3 free)
e extended
Select (default p): [Enter]
Using default response p
Partition number (2-4, default 2): [Enter]
First sector (1026048-83886079, default 1026048): [Enter]
Using default value 1026048
Last sector, +sectors or +size{K,M,G} (1026048-83886079, default 83886079): [Enter]
Using default value 83886079
Partition 2 of type Linux and of size 39.5 GiB is set
a,s
a,s
a,s
a,s
a,s
Command (m for help): p
Disk /dev/sda: 42.9 GB, 42949672960 bytes, 83886000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 5 1500
1510
1520
1530
1540
1550
1560
1570
1580
1590
1600
1610
1620
1630
1640
1650
1660
1670
1680
1690
1700
1710
1720
1730
1740
1750
1760
1770
1780
1790
1800
1810
1820
1830
1840
1850
1860
1870
1880
1890
1900
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0xc9c2368a
Device Boot
/dev/sda1
*
/dev/sda2
a,s
Start
2048
1026048
End
1026047
83886079
Blocks
512000
41430016
Id System
83 Linux
83 Linux
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
a,s
fdisk -H 64 -S 32 /dev/sdb
Welcome to fdisk (util-linux 2.23.2).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0x2a058c02.
a,s
Command (m for help): o
Building a new DOS disklabel with disk identifier 0xb3afd860.
a,s
Command (m for help): n
Partition type:
p primary (0 primary, 0 extended, 4 free)
e extended
Select (default p): [Enter]
Using default response p
Partition number (1-4, default 1): [Enter]
First sector (2048-2097151, default 2048): [Enter]
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-2097151, default 2097151): [Enter]
Using default value 2097151
Partition 1 of type Linux and of size 1023 MiB is set
a,s
a,s
a,s
a,s
a,s
Command (m for help): t
Selected partition 1
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 6 1910
1920
1930
1940
1950
1960
1970
1980
1990
2000
2010
2020
2030
2040
2050
2060
2070
2080
2090
2100
2110
2120
2130
2140
2150
2160
2170
2180
2190
2200
2210
2220
2230
2240
2250
2260
2270
2280
2290
2300
2310
a,s
Hex code (type L to list all codes): 82
Changed type of partition 'Linux' to 'Linux swap / Solaris'
a,s
Command (m for help): p
Disk /dev/sdb: 1073 MB, 1073741824 bytes, 2097152 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0xb3afd860
Device Boot
/dev/sdb1
a,s
Start
2048
End
2097151
Blocks
1047552
Id System
82 Linux swap / Solaris
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
a,s
○
a,s
○
a,s
fdisk -l | grep /dev/ | sort
/dev/sda1
*
2048
1026047
512000
83 Linux
/dev/sda2
1026048
83886079
41430016
83 Linux
/dev/sdb1
2048
2097151
1047552
82 Linux swap / Solaris
Disk /dev/mapper/live-base: 2147 MB, 2147483648 bytes, 4194304 sectors
Disk /dev/mapper/live-rw: 2147 MB, 2147483648 bytes, 4194304 sectors
Disk /dev/sda: 42.9 GB, 42949672960 bytes, 83886080 sectors
Disk /dev/sdb: 1073 MB, 1073741824 bytes, 2097152 sectors
Disk /dev/sdc: 107.4 GB, 107374182400 bytes, 209715200 sectors
Disk /dev/sdd: 107.4 GB, 107374182400 bytes, 209715200 sectors
Disk /dev/sde: 107.4 GB, 107374182400 bytes, 209715200 sectors
Disk /dev/sdf: 107.4 GB, 107374182400 bytes, 209715200 sectors
「Ctrl + Alt + F6」キーを押下し、anaconda の「welcome」画面に戻ります。
# Ctrl + Alt + F6
「English (United States)」が選択されていることを確認し、「Continue」を選択します。
# Continue
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 7 2320
2330
2340
2350
2360
2370
2380
2390
2400
2410
2420
2430
2440
2450
2460
2470
2480
2490
2500
2510
2520
2530
2540
2550
2560
2570
2580
2590
2600
2610
2620
2630
2640
2650
2660
2670
2680
2690
2700
2710
2720
○
a,s
○
a,s
○
a,s
a,s
a,s
○
a,s
○
a,s
○
a,s
○
a,s
○
「DATE & TIME」を選択し、「Asia / Tokyo」を選択します。
# DATE & TIME: Asia / Tokyo
「KEYBOARD」を選択し、「Japanese (OADG 109A)」のみ選ばれているように選択します。
# KEYBOARD: Japanese (OADG 109A)
「INSTALLATION DESTINATION」を選択し、以下のように設定します。
デバイス名 FS
MountPoint ラベル
# /dev/sda1 xfs /boot
/boot
# /dev/sda2 xfs /
/
# /dev/sdb1 swap
swap
「Begin Install」を選択します。
# Begin Install
「ROOT PASSWORD」を選択し、パスワードを設定します。
# ROOT PASSWORD: ********
「Reboot」ボタンが表示されるのを待ち、「Reboot」を選択します。
# Reboot
再起動処理中に Eject されたインストーラをDVDドライブから取り外します。
# Eject DVD
再起動完了後、コンソールにてログインします。
Oracle Linux Server 7.3
Kernel 4.1.12-61.1.18.el7uek.x86_64 on an x86_64
a,s
a,s
localhost login: root
Password: ********
[root@localhost ~] #
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 8 2730 ○
2740
2750 a,s
2760
2770
2780
2790
2800
2810
2820
2830
2840
2850
2860
2870
2880
2890
2900
2910 ○
2920
2930
2940
2950 ○
2960
2970
a
2980
2990
s
3000
3010
3020
3030 ○
3040
3050
3060
3070 ○
3080
3090
a
3100
3110
3120
a
3130
MAC アドレスを確認します。
ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP
link/ether 00:0c:29:08:b8:5f brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP
link/ether 00:0c:29:08:b8:41 brd ff:ff:ff:ff:ff:ff
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP
link/ether 00:0c:29:08:b8:4b brd ff:ff:ff:ff:ff:ff
5: eth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP
link/ether 00:0c:29:08:b8:55 brd ff:ff:ff:ff:ff:ff
qlen 1000
qlen 1000
qlen 1000
qlen 1000
MAC アドレスをもとに、 LAN ケーブルの結線(組み合わせ)を設計通りに修正します。
※　この後の手順で、NIC デバイス名の方を入れ替えても構いません。
IP アドレスを一時的に設定します。
ip addr add 10.110.88.57/26 dev eth0
ip addr add 10.110.88.58/26 dev eth0
※　デバイス名が意図するものとずれている場合、デバイス名は適宜変更する必要があります。
必要に応じて、デフォルトゲートウェイを一時的に設定します。
ip route add default via 10.110.88.1
root にて、ssh でログインします。
ssh root@10.110.88.57
The authenticity of host '10.110.88.57 (10.110.88.57)' can't be established.
ECDSA key fingerprint is 95:bc:49:71:b2:a3:dd:ab:63:ad:35:e4:fe:4d:fc:82.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '10.110.88.57' (ECDSA) to the list of known hosts.
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 9 3140
a
3150
3160
3170
s
3180
3190
3200
s
3210
3220
s
3230
3240
3250 ○
3260
3270 a,s
3280
3290
3300
3310
3320
3330
3340
3350
3360
3370
3380 a,s
3390
3400
3410
3420
3430 a,s
3440
3450
3460
3470
3480
3490
3500
3510
3520
3530
3540
root@10.110.88.57's password: ********
Last login: Sat Oct 29 18:33:24 2016
ssh root@10.110.88.58
The authenticity of host '10.110.88.58 (10.110.88.58)' can't be established.
ECDSA key fingerprint is 8f:f6:81:0f:44:e1:83:d5:0a:9d:3f:90:7c:3e:93:73.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '10.110.88.58' (ECDSA) to the list of known hosts.
root@10.110.88.58's password: ********
Last login: Sat Oct 29 18:33:24 2016
ストレージの情報を確認します。
fdisk -l | grep /dev/ | sort
/dev/sda1
*
2048
1026047
512000
83 Linux
/dev/sda2
1026048
83886079
41430016
83 Linux
/dev/sdb1
2048
2097151
1047552
82 Linux swap / Solaris
Disk /dev/sda: 42.9 GB, 42949672960 bytes, 83886080 sectors
Disk /dev/sdb: 1073 MB, 1073741824 bytes, 2097152 sectors
Disk /dev/sdc: 107.4 GB, 107374182400 bytes, 209715200 sectors
Disk /dev/sdd: 107.4 GB, 107374182400 bytes, 209715200 sectors
Disk /dev/sde: 107.4 GB, 107374182400 bytes, 209715200 sectors
Disk /dev/sdf: 107.4 GB, 107374182400 bytes, 209715200 sectors
blkid
/dev/sda2: LABEL="/" UUID="6fa3bde3-dc77-461d-8ae4-5a6ea5efba4c" TYPE="xfs"
/dev/sda1: LABEL="/boot" UUID="11b16718-fa37-4aed-baff-2b643304e705" TYPE="xfs"
/dev/sdb1: LABEL="swap" UUID="d561d285-585b-4790-9690-1b55598de94b" TYPE="swap"
cat /etc/fstab
#
# /etc/fstab
# Created by anaconda on Fri Nov 25 11:55:06 2016
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=6fa3bde3-dc77-461d-8ae4-5a6ea5efba4c /
xfs
defaults
UUID=11b16718-fa37-4aed-baff-2b643304e705 /boot
xfs
defaults
UUID=d561d285-585b-4790-9690-1b55598de94b swap
swap
defaults
0 0
0 0
0 0
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 10 3550
3560
3570
3580
3590
3600
3610
3620
3630
3640
3650
3660
3670
3680
3690
3700
3710
3720
3730
3740
3750
3760
3770
3780
3790
3800
3810
3820
3830
3840
3850
3860
3870
3880
3890
3900
3910
3920
3930
3940
3950
○
キーボード、ロケールの情報を確認します。
a,s
cat /etc/vconsole.conf
KEYMAP="jp-OADG109A"
FONT="latarcyrheb-sun16"
a,s
cat /etc/locale.conf
LANG="en_US.UTF-8"
a,s
localectl status
System Locale:
VC Keymap:
X11 Layout:
X11 Variant:
○
LANG=en_US.UTF-8
jp-OADG109A
jp
OADG109A
タイムゾーンの情報を確認します。
a,s
cat /etc/adjtime
0.0 0 0.0
0
UTC
a,s
hwclock --debug
hwclock from util-linux 2.23.2
Using /dev interface to clock.
Last drift adjustment done at 0 seconds after 1969
Last calibration done at 0 seconds after 1969
Hardware clock is on UTC time
Assuming hardware clock is kept in UTC time.
Waiting for clock tick...
...got clock tick
Time read from Hardware Clock: 2016/11/25 04:31:00
Hw clock time : 2016/11/25 04:31:00 = 1480048260 seconds since 1969
Fri 25 Nov 2016 01:31:00 PM JST -0.239477 seconds
a,s
ls -l /etc/localtime
lrwxrwxrwx 1 root root 32 Nov 25 11:58 /etc/localtime -> ../usr/share/zoneinfo/ Asia/Tokyo
a,s
timedatectl status
Local time: Fri 2016-11-25 13:31:44 JST
Universal time: Fri 2016-11-25 04:31:44 UTC
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 11 3960
3970
3980
3990
4000
4010
4020
4030
4040
4050
4060
4070
4080
4090
4100
4110
4120
4130
4140
4150
4160
4170
4180
4190
4200
4210
4220
4230
4240
4250
4260
4270
4280
4290
4300
4310
4320
4330
4340
4350
4360
RTC time:
Time zone:
NTP enabled:
NTP synchronized:
RTC in local TZ:
DST active:
○
Fri 2016-11-25 04:31:43
Asia/Tokyo (JST, +0900)
n/a
no
no
n/a
kdump の設定を確認します。
a,s
systemctl is-enabled kdump.service
enabled
a,s
kdumpctl status
Kdump is operational
○
a,s
SELinux を無効化します。
sed -i -e 's/^SELINUX=.*$/SELINUX=disabled/' /etc/sysconfig/selinux
※　カーネルパラメータで無効化していますが、運用上紛らわしいので設定ファイルも変更します。
○
SELinux の設定を確認します。
a,s
grep -v ^# /etc/sysconfig/selinux
SELINUX=disabled
SELINUXTYPE=targeted
a,s
getenforce
Disabled
○
管理者用一般ユーザを作成します。
a,s
sed -i -e 's/^CREATE_MAIL_SPOOL=.*$/CREATE_MAIL_SPOOL=no/' /etc/default/useradd
a,s
a,s
a,s
groupadd -g 1000 admin
useradd -g admin -G wheel -u 1000 admin
echo 'password' | passwd --stdin admin
Changing password for user admin.
passwd: all authentication tokens updated successfully.
a,s
id admin
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 12 4370
4380
4390
4400
4410
4420
4430
4440
4450
4460
4470
4480
4490
4500
4510
4520
4530
4540
4550
4560
4570
4580
4590
4600
4610
4620
4630
4640
4650
4660
4670
4680
4690
4700
4710
4720
4730
4740
4750
4760
4770
uid=1000(admin) gid=1000(admin) groups=1000(admin),10(wheel)
○
a,s
○
wheel グループのユーザがパスワードなしで sudo コマンドを使えるように設定します。
echo '%wheel ALL=(ALL) NOPASSWD: ALL' > /etc/sudoers.d/wheel
管理者用一般ユーザにて、ssh でログインします。
a
a
ssh admin@10.110.88.57
admin@10.110.88.57's password: ********
s
s
ssh admin@10.110.88.58
admin@10.110.88.58's password: ********
○
a,s
a,s
○
a,s
a,s
○
wheel グループのユーザのみが su コマンドを使えるように設定します。
sudo sed -i -e '/^#auth.*required.*pam_wheel.so use_uid$/ s/#//' /etc/pam.d/su
echo "SU_WHEEL_ONLY yes" | sudo tee -a /etc/login.defs
root アカウントでのパスワード認証による ssh 接続を禁止します。
sudo sed -i -e 's/^#PermitRootLogin .*$/PermitRootLogin without-password/' /etc/ssh/sshd_config
sudo systemctl restart sshd
参照・監視用一般ユーザを作成します。
a,s
a,s
a,s
sudo groupadd -g 1001 monitor
sudo useradd -g monitor -u 1001 monitor
echo 'password' | sudo passwd --stdin monitor
Changing password for user monitor.
passwd: all authentication tokens updated successfully.
a,s
id monitor
uid=1001(monitor) gid=1001(monitor) groups=1001(monitor)
○
a,s
a,s
a,s
a,s
NIC のデバイス名をバス情報に基づいて固定します。
sudo cp /dev/null /etc/udev/rules.d/70-persistent-net.rules
NUM=0
while :
do
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 13 4780
4790
4800
4810
4820
4830
4840
4850
4860
4870
4880
4890
4900
4910
4920
4930
4940
4950
4960
4970
4980
4990
5000
5010
5020
5030
5040
5050
5060
5070
5080
5090
5100
5110
5120
5130
5140
5150
5160
5170
5180
a,s
a,s
a,s
a,s
a,s
a,s
a,s
ip addr show eth$NUM > /dev/null 2>&1 || break
BUS=$(ethtool -i eth$NUM | grep bus-info | awk '{print $2}')
cat << EOF | sudo tee -a /etc/udev/rules.d/70-persistent-net.rules
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", KERNELS=="$BUS", ATTR{type}=="1", NAME="eth$NUM"
EOF
NUM=$((NUM+1))
done
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", KERNELS=="0000:04:00.0" , ATTR{type}=="1", NAME="eth0"
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", KERNELS=="0000:0b:00.0" , ATTR{type}=="1", NAME="eth1"
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", KERNELS=="0000:13:00.0" , ATTR{type}=="1", NAME="eth2"
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", KERNELS=="0000:1b:00.0" , ATTR{type}=="1", NAME="eth3"
※　このファイルを編集して、NIC デバイス名を入れ替えても構いません。
※　KERNELS=="<バス情報>" を ATTR{address}=="<MAC アドレス>" に入れ替えても構いません。
○
a,s
a,s
a,s
a,s
a,s
OS 起動時のカーネルパラメータを変更します。
sudo sed -i -e '/^GRUB_CMDLINE_LINUX=/ s/ *biosdevname=[^ "]*//' /etc/default/grub
sudo sed -i -e '/^GRUB_CMDLINE_LINUX=/ s/ *net\.ifnames=[^ "]*//' /etc/default/grub
sudo sed -i -e '/^GRUB_CMDLINE_LINUX=/ s/rhgb quiet/ipv6.disable=1 sysrq_always_enabled=1 log_buf_len=16777216/' /etc/default/grub
echo 'kernel.sysrq = 1' | sudo tee -a /etc/sysctl.d/99-sysctl.conf
sudo grub2-mkconfig -o /boot/grub2/grub.cfg
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-4.1.12-61.1.18.el7uek.x86_64
Found initrd image: /boot/initramfs-4.1.12-61.1.18.el7uek.x86_64.img
Found linux image: /boot/vmlinuz-3.10.0-514.el7.x86_64
Found initrd image: /boot/initramfs-3.10.0-514.el7.x86_64.img
Found linux image: /boot/vmlinuz-0-rescue-06dccd866dbd479b8a41b818455151b2
Found initrd image: /boot/initramfs-0-rescue-06dccd866dbd479b8a41b818455151b2.img
done
※　「net.ifnames=0」「biosdevname=0」があると、前項の udev 設定が機能しません。
※　IPv6 を無効化しています。
○
a,s
a,s
a,s
a,s
a,s
NIC を設定します。
BOND0_BONDING_OPTS="resend_igmp=1 updelay=0 use_carrier=1 miimon=100 downdelay=0 xmit_hash_policy=0"
BOND0_BONDING_OPTS="$BOND0_BONDING_OPTS primary_reselect=0 fail_over_mac=0 arp_validate=0"
BOND0_BONDING_OPTS="$BOND0_BONDING_OPTS mode= active-backup primary=eth0 "
※　LAG(LACP) の場合は「mode=802.3ad」
BOND0_BONDING_OPTS="$BOND0_BONDING_OPTS lacp_rate=0 arp_interval=0 ad_select=0"
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 14 5190
5200
5210
5220
5230
5240
5250
5260
5270
5280
5290
5300
5310
5320
5330
5340
5350
5360
5370
5380
5390
5400
5410
5420
5430
5440
5450
5460
5470
5480
5490
5500
5510
5520
5530
5540
5550
5560
5570
5580
5590
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
BOND1_BONDING_OPTS="resend_igmp=1 updelay=0 use_carrier=1 miimon=100 downdelay=0 xmit_hash_policy=0"
BOND1_BONDING_OPTS="$BOND1_BONDING_OPTS primary_reselect=0 fail_over_mac=0 arp_validate=0"
BOND1_BONDING_OPTS="$BOND1_BONDING_OPTS mode= active-backup primary=eth1 "
※　LAG(LACP) の場合は「mode=802.3ad」
BOND1_BONDING_OPTS="$BOND1_BONDING_OPTS lacp_rate=0 arp_interval=0 ad_select=0"
cat << EOF | sudo tee /etc/sysconfig/network-scripts/ifcfg-bond0
DEVICE=bond0
NAME=bond0
TYPE=Bond
UUID=$(uuidgen)
BONDING_OPTS="$BOND0_BONDING_OPTS"
BONDING_MASTER=yes
ONBOOT=yes
BOOTPROTO=none
DEFROUTE=yes
PEERDNS=no
PEERROUTES=no
IPV4_FAILURE_FATAL=yes
IPV6INIT=no
IPV6_AUTOCONF=no
IPV6_DEFROUTE=no
IPV6_PEERDNS=no
IPV6_PEERROUTES=no
IPV6_FAILURE_FATAL=no
EOF
cat << EOF | sudo tee /etc/sysconfig/network-scripts/ifcfg-bond1
DEVICE=bond1
NAME=bond1
TYPE=Bond
UUID=$(uuidgen)
BONDING_OPTS="$BOND1_BONDING_OPTS"
BONDING_MASTER=yes
ONBOOT=yes
BOOTPROTO=none
DEFROUTE=no
PEERDNS=no
PEERROUTES=no
IPV4_FAILURE_FATAL=yes
IPV6INIT=no
IPV6_AUTOCONF=no
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 15 5600
5610
5620
5630
5640
5650
5660
5670
5680
5690
5700
5710
5720
5730
5740
5750
5760
5770
5780
5790
5800
5810
5820
5830
5840
5850
5860
5870
5880
5890
5900
5910
5920
5930
5940
5950
5960
5970
5980
5990
6000
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
IPV6_DEFROUTE=no
IPV6_PEERDNS=no
IPV6_PEERROUTES=no
IPV6_FAILURE_FATAL=no
EOF
cat << EOF | sudo tee /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
NAME=eth0
TYPE=Ethernet
UUID=$(uuidgen)
MASTER=bond0
SLAVE=yes
ONBOOT=yes
MTU=9000
EOF
cat << EOF | sudo tee /etc/sysconfig/network-scripts/ifcfg-eth2
DEVICE=eth2
NAME=eth2
TYPE=Ethernet
UUID=$(uuidgen)
MASTER=bond0
SLAVE=yes
ONBOOT=yes
MTU=9000
EOF
cat << EOF | sudo tee /etc/sysconfig/network-scripts/ifcfg-eth1
DEVICE=eth1
NAME=eth1
TYPE=Ethernet
UUID=$(uuidgen)
MASTER=bond1
SLAVE=yes
ONBOOT=yes
MTU=9000
EOF
cat << EOF | sudo tee /etc/sysconfig/network-scripts/ifcfg-eth3
DEVICE=eth3
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 16 6010
6020
6030
6040
6050
6060
6070
6080
6090
6100
6110
6120
6130
6140
6150
6160
6170
6180
6190
6200
6210
6220
6230
6240
6250
6260
6270
6280
6290
6300
6310
6320
6330
6340
6350
6360
6370
6380
6390
6400
6410
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
a
s
s
s
s
s
s
s
s
s
s
s
s
s
s
s
NAME=eth3
TYPE=Ethernet
UUID=$(uuidgen)
MASTER=bond1
SLAVE=yes
ONBOOT=yes
MTU=9000
EOF
# for Active
cat << EOF | sudo tee -a /etc/sysconfig/network-scripts/ifcfg-bond0
IPADDR=10.110.88.57
PREFIX=26
GATEWAY=10.110.88.1
DNS1=10.0.80.11
DNS2=10.0.80.12
DOMAIN=example.com
MTU=9000
EOF
cat << EOF | sudo tee -a /etc/sysconfig/network-scripts/ifcfg-bond1
IPADDR=192.168.1.2
PREFIX=24
MTU=9000
EOF
# for Stand-by
cat << EOF | sudo tee -a /etc/sysconfig/network-scripts/ifcfg-bond0
IPADDR=10.110.88.58
PREFIX=26
GATEWAY=10.110.88.1
DNS1=10.0.80.11
DNS2=10.0.80.12
DOMAIN=example.com
MTU=9000
EOF
cat << EOF | sudo tee -a /etc/sysconfig/network-scripts/ifcfg-bond1
IPADDR=192.168.1.3
PREFIX=24
MTU=9000
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 17 6420
6430
6440
6450
6460
6470
6480
6490
6500
6510
6520
6530
6540
6550
6560
6570
6580
6590
6600
6610
6620
6630
6640
6650
6660
6670
6680
6690
6700
6710
6720
6730
6740
6750
6760
6770
6780
6790
6800
6810
6820
s
○
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
EOF
NIC オフロード機能を無効化します。
cat << 'EOF' | sudo tee /etc/NetworkManager/dispatcher.d/00-ethertool
#!/bin/sh
if [ "$2" == "up" ]; then
if [ "${1:0:3}" == "eth" ]; then
ethtool -K $1 \
rx off \
tx off \
sg off \
tso off \
ufo off \
gso off \
gro off \
lro off \
rxvlan off \
txvlan off \
ntuple off \
rxhash off \
highdma off \
rx-vlan-filter off \
tx-gso-robust off \
tx-fcoe-segmentation off \
fcoe-mtu off \
tx-nocache-copy off \
loopback off \
rx-fcs off \
rx-all off
#ethtool -K $1 vlan-challenged off tx-lockless off netns-local off
ethtool -G $1 rx 4096 tx 4096 rx-jumbo 2048
fi
fi
EOF
sudo chmod 755 /etc/NetworkManager/dispatcher.d/00-ethertool
※　「VMXNET 3」「e1000e」「igb」「ixgbe」でのみ動作確認しました。
※　NIC のリンク速度、duplex モードを設定したい場合はこのスクリプトに組み込みます。
○
hosts を設定します。
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 18 6830
6840
6850
6860
6870
6880
6890
6900
6910
6920
6930
6940
6950
6960
6970
6980
6990
7000
7010
7020
7030
7040
7050
7060
7070
7080
7090
7100
7110
7120
7130
7140
7150
7160
7170
7180
7190
7200
7210
7220
7230
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
○
cat << 'EOF' | sudo tee /etc/hosts
127.0.0.1
localhost localhost.localdomain localhost4 localhost4.localdomain4
::1
localhost localhost.localdomain localhost6 localhost6.localdomain6
10.110.88.57
iscsitgt01a.example.com iscsitgt01a
10.110.88.58
iscsitgt01s.example.com iscsitgt01s
10.110.88.59
iscsitgt01.example.com iscsitgt01
192.168.1.2
iscsitgt01a-ic.example.com iscsitgt01a-ic
192.168.1.3
iscsitgt01s-ic.example.com iscsitgt01s-ic
EOF
hostname を設定します。
a
sudo hostnamectl set-hostname iscsitgt01a.example.com
s
sudo hostnamectl set-hostname iscsitgt01s.example.com
○
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
○
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
不要なログ出力を抑止します。
cat << 'EOF' | sudo tee /etc/rsyslog.d/ignore-systemd-session-slice.conf
if $programname == "systemd" and ($msg contains "Created slice user-" or $msg contains "Removed slice user-" or $msg contains "Starting user-" or $msg contains "Started Session " or $msg contains "Starting Session " or $msg contains "Stopping user-") then stop
EOF
cat << 'EOF' | sudo tee /etc/rsyslog.d/ignore-chronyd-selected-source.conf
if $programname == "chronyd" and $msg contains "Selected source " then stop
EOF
sudo systemctl restart rsyslog
yum リポジトリを設定します。
cat << 'EOF' | sudo tee /etc/yum.repos.d/media.repo
[media]
name=media
baseurl=file:///mnt
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
gpgcheck=1
enabled=0
[media-mysql]
name=media-mysql
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 19 7240
7250
7260
7270
7280
7290
7300
7310
7320
7330
7340
7350
7360
7370
7380
7390
7400
7410
7420
7430
7440
7450
7460
7470
7480
7490
7500
7510
7520
7530
7540
7550
7560
7570
7580
7590
7600
7610
7620
7630
7640
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
baseurl=file:///mnt/addons/Mysql
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
gpgcheck=1
enabled=0
[media-ha]
name=media-ha
baseurl=file:///mnt/addons/HighAvailability
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
gpgcheck=1
enabled=0
[media-rs]
name=media-rs
baseurl=file:///mnt/addons/ResilientStorage
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
gpgcheck=1
enabled=0
EOF
※　インストール・メディアを利用可能にします。
a,s
sudo sed -i -e 's/^/#/' /etc/yum.repos.d/public-yum-ol7.repo
※　インターネット接続していないとエラーとなるリポジトリを無効化します。
○
a,s
○
a,s
○
a,s
a,s
a,s
a,s
以下のインストーラを DVD ドライブにセットします。
# V834394-01.iso (Oracle Linux 7.3)
インストーラをマウントします。
sudo mount /dev/cdrom /mnt
mount: /dev/sr0 is write-protected, mounting read-only
どのような環境でも共通して導入しておいた方がよいと思われる標準パッケージをインストールします。
sudo yum -y --disablerepo=\* --enablerepo=media,media-mysql install \
@development \
@base \
OpenIPMI \
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 20 7650
7660
7670
7680
7690
7700
7710
7720
7730
7740
7750
7760
7770
7780
7790
7800
7810
7820
7830
7840
7850
7860
7870
7880
7890
7900
7910
7920
7930
7940
7950
7960
7970
7980
7990
8000
8010
8020
8030
8040
8050
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
aide \
crash \
dos2unix \
dropwatch \
dstat \
expect \
filebench \
freeipmi-bmc-watchdog \
freeipmi-ipmidetectd \
ftp \
fuse \
fuse-devel \
haproxy \
hdparm \
iotop \
ipmitool \
iptables-services \
iptraf-ng \
iptstate \
ipvsadm \
iscsi-initiator-utils \
keepalived \
kernel-uek-devel \
kernel-uek-doc \
latrace \
lftp \
libuuid-devel \
lm_sensors \
logwatch \
lrzsz \
ltrace \
net-snmp-utils \
nmap \
openssl-devel \
oprofile \
pax \
perf \
prelink \
screen \
sg3_utils \
snapper \
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 21 8060
8070
8080
8090
8100
8110
8120
8130
8140
8150
8160
8170
8180
8190
8200
8210
8220
8230
8240
8250
8260
8270
8280
8290
8300
8310
8320
8330
8340
8350
8360
8370
8380
8390
8400
8410
8420
8430
8440
8450
8460
a,s
a,s
a,s
a,s
a,s
○
o
o
○
telnet \
tmpwatch \
trace-cmd \
tree \
x86info
インターネットと接続可能な端末で以下のコマンドを実行する等して、必要なパッケージを収集します。
curl -O https://oss.oracle.com/ol7/debuginfo/kernel-uek-debuginfo- 4.1.12-61.1.18.el7uek .x86_64.rpm
curl -O https://oss.oracle.com/ol7/debuginfo/kernel-uek-debuginfo-common- 4.1.12-61.1.18.el7uek .x86_64.rpm
収集したパッケージをホームディレクトリにコピーし、確認します。
a,s
a,s
scp xxxx@yyy:kernel-uek-debuginfo- 4.1.12-61.1.18.el7uek .x86_64.rpm .
scp xxxx@yyy:kernel-uek-debuginfo-common- 4.1.12-61.1.18.el7uek .x86_64.rpm .
a,s
ls -l *.rpm
-rw-rw-r-- 1 admin admin 382756108 Dec 15 12:06 kernel-uek-debuginfo- 4.1.12-61.1.18.el7uek .x86_64.rpm
-rw-rw-r-- 1 admin admin 52360420 Dec 15 12:08 kernel-uek-debuginfo-common- 4.1.12-61.1.18.el7uek .x86_64.rpm
a,s
file *.rpm
kernel-uek-debuginfo-4.1.12-61.1.18.el7uek.x86_64.rpm:
RPM v3.0 bin i386/x86_64 kernel-uek-debuginfo-4.1.12-61.1.18.el7uek
kernel-uek-debuginfo-common-4.1.12-61.1.18.el7uek.x86_64.rpm: RPM v3.0 bin i386/x86_64 kernel-uek-debuginfo-common-4.1.12-61.1.18.el7uek
○
a,s
a,s
a,s
○
a,s
○
a,s
○
a,s
カーネルのデバッグ情報をインストールします。
sudo yum -y --disablerepo=\* --enablerepo=media localinstall kernel-uek-debuginfo-*.rpm
sudo mkdir /opt/packages/
sudo mv kernel-uek-debuginfo-*.rpm /opt/packages/
インストーラをアンマウントします。
sudo umount /mnt
インストーラをDVDドライブから取り外します。
# Eject DVD
NTP を設定します。
cat << 'EOF' | sudo tee /etc/chrony.conf
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 22 8470
8480
8490
8500
8510
8520
8530
8540
8550
8560
8570
8580
8590
8600
8610
8620
8630
8640
8650
8660
8670
8680
8690
8700
8710
8720
8730
8740
8750
8760
8770
8780
8790
8800
8810
8820
8830
8840
8850
8860
8870
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
server 10.0.77.54 iburst
# server ***.***.***.*** iburst
# server ***.***.***.*** iburst
# server ***.***.***.*** iburst
# Use public servers from the pool.ntp.org project.
# Please consider joining the pool (http://www.pool.ntp.org/join.html).
# Ignore stratum in source selection.
stratumweight 0
# Record the rate at which the system clock gains/losses time.
driftfile /var/lib/chrony/drift
# Enable kernel RTC synchronization.
rtcsync
# In first three updates step the system clock instead of slew
# if the adjustment is larger than 10 seconds.
makestep 10 3
# Allow NTP client access from local network.
#allow 192.168/16
# Listen for commands only on localhost.
bindcmdaddress 127.0.0.1
#bindcmdaddress ::1
# Serve time even if not synchronized to any NTP server.
#local stratum 10
keyfile /etc/chrony.keys
# Specify the key used as password for chronyc.
commandkey 1
# Generate command key if missing.
generatecommandkey
# Disable logging of client accesses.
noclientlog
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 23 8880
8890
8900
8910
8920
8930
8940
8950
8960
8970
8980
8990
9000
9010
9020
9030
9040
9050
9060
9070
9080
9090
9100
9110
9120
9130
9140
9150
9160
9170
9180
9190
9200
9210
9220
9230
9240
9250
9260
9270
9280
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
○
a,s
a,s
a,s
a,s
# Send a message to syslog if a clock adjustment is larger than 0.5 seconds.
logchange 0.5
logdir /var/log/chrony
#log measurements statistics tracking
EOF
cat << 'EOF' | sudo tee /etc/sysconfig/chronyd
OPTIONS="-4"
EOF
不要なサービスを無効化します。
sudo
sudo
sudo
sudo
systemctl
systemctl
systemctl
systemctl
disable
disable
disable
disable
dmraid-activation.service
firewalld.service
mdmonitor.service
postfix.service
※　仮想環境の場合は、「lm_sensors.service 」も無効化します。
※　仮想環境の場合は、「smartd.service 」も無効化します。RAID コントローラが対応していない場合も無効化します。
○
a,s
○
必要なサービスを有効化します。
sudo systemctl enable psacct.service
iSCSI イニシエータ関連サービスの自動起動を無効化します。
a,s
sudo systemctl disable iscsi.service
Removed symlink /etc/systemd/system/sysinit.target.wants/iscsi.service.
a,s
sudo systemctl disable iscsid.socket
Removed symlink /etc/systemd/system/sockets.target.wants/iscsid.socket.
a,s
sudo systemctl disable iscsiuio.socket
Removed symlink /etc/systemd/system/sockets.target.wants/iscsiuio.socket.
○
a,s
a,s
IPv6 無効化に伴う不具合を解消するための設定変更を行います。
sudo sed -i -e 's/^#AddressFamily .*$/AddressFamily inet/' /etc/ssh/sshd_config
sudo sed -i -e 's/^inet_interfaces .*$/inet_interfaces = 127.0.0.1/' /etc/postfix/main.cf
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 24 9290
9300
9310
9320
9330
9340
9350
9360
9370
9380
9390
9400
9410
9420
9430
9440
9450
9460
9470
9480
9490
9500
9510
9520
9530
9540
9550
9560
9570
9580
9590
9600
9610
9620
9630
9640
9650
9660
9670
9680
9690
a,s
sudo sed -i -e 's/^inet_protocols .*$/inet_protocols = ipv4/' /etc/postfix/main.cf
a,s
sudo sed -i -e 's/^udp6/#udp6/' -e 's/^tcp6/#tcp6/' /etc/netconfig
○
a,s
○
再起動します。
sudo reboot
管理者用一般ユーザにて、ssh でログインします。
a
a
ssh admin@10.110.88.57
admin@10.110.88.57's password: ********
s
s
ssh admin@10.110.88.58
admin@10.110.88.58's password: ********
○
a,s
カーネル起動パラメータを確認します。
cat /proc/cmdline
BOOT_IMAGE=/vmlinuz-4.1.12-61.1.18.el7uek.x86_64 root=UUID=657f59aa-f627-4096-9970-9238b234ef00 ro crashkernel=auto selinux=0 ipv6.disable=1
※　「crashkernel」の値は、搭載メモリサイズに応じて自動的に固定値へ変更される場合があります。
○
a,s
ネットワーク設定を確認します。
ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
2: eth0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 9000 qdisc mq master
link/ether 00:0c:29:08:b8:5f brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 9000 qdisc mq master
link/ether 00:0c:29:08:b8:41 brd ff:ff:ff:ff:ff:ff
4: eth2: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 9000 qdisc mq master
link/ether 00:0c:29:08:b8:5f brd ff:ff:ff:ff:ff:ff
5: eth3: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 9000 qdisc mq master
link/ether 00:0c:29:08:b8:41 brd ff:ff:ff:ff:ff:ff
6: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 9000 qdisc noqueue
link/ether 00:0c:29:08:b8:5f brd ff:ff:ff:ff:ff:ff
inet 10.110.88.57/26 brd 10.110.88.63 scope global bond0
7: bond1: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 9000 qdisc noqueue
bond0 state UP qlen 1000
bond1 state UP qlen 1000
bond0 state UP qlen 1000
bond1 state UP qlen 1000
state UNKNOWN
state UNKNOWN
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 25 9700
9710
9720
9730
9740
9750 a,s
9760
9770
9780
9790
9800
9810
9820
9830
9840
9850
9860
9870
9880
9890
9900
9910
9920
9930
9940
9950
9960
9970
9980
9990
10000
10010
10020 a,s
10030
10040
10050
10060
10070
10080
10090
10100
link/ether 00:0c:29:08:b8:41 brd ff:ff:ff:ff:ff:ff
inet 192.168.1.2/24 brd 192.168.0.255 scope global bond1
※　IPv6 のリンクローカルアドレスが存在しないことも確認します。
cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)
Bonding Mode: fault-tolerance (active-backup)
Primary Slave: eth0 (primary_reselect always)
Currently Active Slave: eth0
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
Slave Interface: eth0
MII Status: up
Speed: 10000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:0c:29:08:b8:5f
Slave queue ID: 0
Slave Interface: eth2
MII Status: up
Speed: 10000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:0c:29:08:b8:4b
Slave queue ID: 0
cat /proc/net/bonding/bond1
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)
Bonding Mode: fault-tolerance (active-backup)
Primary Slave: eth1 (primary_reselect always)
Currently Active Slave: eth1
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 26 10110
10120
10130
10140
10150
10160
10170
10180
10190
10200
10210
10220
10230
10240
10250
10260
10270
10280
10290
10300
10310
10320
10330
10340
10350
10360
10370
10380
10390
10400
10410
10420
10430
10440
10450
10460
10470
10480
10490
10500
10510
Down Delay (ms): 0
Slave Interface: eth1
MII Status: up
Speed: 10000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:0c:29:08:b8:41
Slave queue ID: 0
Slave Interface: eth3
MII Status: up
Speed: 10000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:0c:29:08:b8:55
Slave queue ID: 0
※　ボンディング設定時は、このコマンドで個々の NIC の MAC アドレスを確認できます。
※　LAG(LACP) の場合、以下のように表示されます。
cat /proc/net/bonding/bond1
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)
Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer2 (0)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
802.3ad info
LACP rate: slow
Min links: 0
Aggregator selection policy (ad_select): stable
Active Aggregator Info:
Aggregator ID: 1
Number of ports: 1
Actor Key: 13
Partner Key: 1
Partner Mac Address: 00:00:00:00:00:00
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 27 10520
10530
10540
10550
10560
10570
10580
10590
10600
10610
10620
10630
10640
10650
10660
10670
10680
10690
10700
10710
10720
10730
10740
10750
10760
10770
10780
10790
10800
10810
10820
10830
10840
10850
10860
10870
10880
10890
10900
10910
10920
Slave Interface: eth1
MII Status: up
Speed: 10000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:0c:29:8b:ff:e8
Slave queue ID: 0
Aggregator ID: 1
Actor Churn State: none
Partner Churn State: churned
Actor Churned Count: 0
Partner Churned Count: 1
details actor lacp pdu:
system priority: 0
port key: 13
port priority: 255
port number: 1
port state: 205
details partner lacp pdu:
system priority: 65535
oper key: 1
port priority: 255
port number: 1
port state: 3
Slave Interface: eth3
MII Status: up
Speed: 10000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:0c:29:8b:ff:fc
Slave queue ID: 0
Aggregator ID: 2
Actor Churn State: churned
Partner Churn State: churned
Actor Churned Count: 1
Partner Churned Count: 1
details actor lacp pdu:
system priority: 0
port key: 13
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 28 10930
10940
10950
10960
10970
10980
10990
11000
11010
11020
11030
a
11040
11050
11060
11070
11080
s
11090
11100
11110
11120
11130 a,s
11140
11150
11160
11170
11180
11190 ○
11200
11210
a
11220
11230
11240
11250
11260
11270
11280
11290
11300
11310
11320
11330
s
port priority: 255
port number: 2
port state: 197
details partner lacp pdu:
system priority: 65535
oper key: 1
port priority: 255
port number: 1
port state: 3
ip route show
default via 10.110.88.1 dev bond0 proto static metric 300
10.110.88.0/26 dev bond0 proto kernel scope link src 10.110.88.57 metric 300
192.168.1.0/24 dev bond1 proto kernel scope link src 192.168.0.2 metric 300
ip route show
default via 10.110.88.1 dev bond0 proto static metric 300
10.110.88.0/26 dev bond0 proto kernel scope link src 10.110.88.58 metric 300
192.168.1.0/24 dev bond1 proto kernel scope link src 192.168.0.3 metric 300
cat /etc/resolv.conf
# Generated by NetworkManager
search example.com
nameserver 10.0.80.11
nameserver 10.0.80.12
hostname 設定を確認します。
hostnamectl status
Static hostname:
Icon name:
Chassis:
Machine ID:
Boot ID:
Virtualization:
Operating System:
CPE OS Name:
Kernel:
Architecture:
iscsitgt01a.example.com
computer-vm
vm
d7806eba789047baa165a57149c83843
b5b36a3403dd403aad4656d2f7f9e7aa
vmware
Oracle Linux Server 7.3
cpe:/o:oracle:linux:7:2:server
Linux 4.1.12-61.1.18.el7uek.x86_64
x86-64
hostnamectl status
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 29 11340
11350
11360
11370
11380
11390
11400
11410
11420
11430
11440
11450 ○
11460
11470 a,s
11480
11490
11500
11510
11520
11530
11540
11550
11560
11570
11580
11590
11600
11610
11620
11630
11640
11650
11660
11670
11680
11690
11700
11710
11720
11730
11740
Static hostname:
Icon name:
Chassis:
Machine ID:
Boot ID:
Virtualization:
Operating System:
CPE OS Name:
Kernel:
Architecture:
iscsitgt01s.example.com
computer-vm
vm
b325c1c5d682439a91a65f7cfc317b20
a419d4d1ef00452f93da10a227365aca
vmware
Oracle Linux Server 7.3
cpe:/o:oracle:linux:7:2:server
Linux 4.1.12-61.1.18.el7uek.x86_64
x86-64
NIC のオフロード設定を確認します。
ethtool -k eth0
Features for eth0:
rx-checksumming: off
tx-checksumming: off
tx-checksum-ipv4: off [fixed]
tx-checksum-ip-generic: off
tx-checksum-ipv6: off [fixed]
tx-checksum-fcoe-crc: off [fixed]
tx-checksum-sctp: off [fixed]
scatter-gather: off
tx-scatter-gather: off
tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: off
tx-tcp-segmentation: off
tx-tcp-ecn-segmentation: off [fixed]
tx-tcp6-segmentation: off
udp-fragmentation-offload: off [fixed]
generic-segmentation-offload: off
generic-receive-offload: off
large-receive-offload: off
rx-vlan-offload: off
tx-vlan-offload: off
ntuple-filters: off [fixed]
receive-hashing: off
highdma: off
rx-vlan-filter: on [fixed]
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 30 11750
11760
11770
11780
11790
11800
11810
11820
11830
11840
11850
11860
11870
11880
11890
11900
11910
11920
11930
11940
11950
11960
11970
11980
11990
12000
12010
12020
12030
12040
12050
12060
12070
12080
12090
12100
12110
12120
12130
12140
12150
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: off [fixed]
tx-ipip-segmentation: off [fixed]
tx-sit-segmentation: off [fixed]
tx-udp_tnl-segmentation: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off [fixed]
rx-all: off [fixed]
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
l2-fwd-offload: off [fixed]
busy-poll: off [fixed]
hw-switch-offload: off [fixed]
a,s
a,s
a,s
ethtool -k eth1
ethtool -k eth2
ethtool -k eth3
a,s
ethtool -g eth0
Ring parameters for eth0:
Pre-set maximums:
RX:
4096
RX Mini:
0
RX Jumbo:
2048
TX:
4096
Current hardware settings:
RX:
4032
RX Mini:
0
RX Jumbo:
2048
TX:
4096
※　搭載メモリサイズに応じて結果が異なります。
a,s
a,s
a,s
ethtool -g eth1
ethtool -g eth2
ethtool -g eth3
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 31 12160
12170
12180
12190
12200
12210
12220
12230
12240
12250
12260
12270
12280
12290
12300
12310
12320
12330
12340
12350
12360
12370
12380
12390
12400
12410
12420
12430
12440
12450
12460
12470
12480
12490
12500
12510
12520
12530
12540
12550
12560
○
a,s
NTP の状態を確認します。
systemctl status chronyd.service -l
● chronyd.service - NTP client/server
Loaded: loaded (/usr/lib/systemd/system/chronyd.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2016-11-25 15:23:28 JST; 17min ago
Process: 601 ExecStartPost=/usr/libexec/chrony-helper update-daemon (code=exited, status=0/SUCCESS)
Process: 576 ExecStart=/usr/sbin/chronyd $OPTIONS (code=exited, status=0/SUCCESS)
Main PID: 583 (chronyd)
CGroup: /system.slice/chronyd.service
┗583 /usr/sbin/chronyd -4
Nov
Nov
Nov
Nov
Nov
25
25
25
25
25
15:23:28
15:23:28
15:23:28
15:23:28
15:23:35
iscsitgt01a.example.com
iscsitgt01a.example.com
iscsitgt01a.example.com
iscsitgt01a.example.com
iscsitgt01a.example.com
systemd[1]: Starting NTP client/server...
chronyd[584]: chronyd version 2.1.1 starting (+CMDMON +NTP +REFCLOCK +RTC +PRIVDROP +DEBUG +ASYNCDNS +IPV6 +SECHASH)
chronyd[584]: Generated key 1
systemd[1]: Started NTP client/server.
chronyd[584]: Selected source 10.0.77.54
a,s
chronyc sources
210 Number of sources = 1
MS Name/IP address
Stratum Poll Reach LastRx Last sample
===============================================================================
^* 10.0.77.54
1 10
377
217
-177us[ -161us] +/- 4360us
a,s
timedatectl status
Local time: Fri 2016-11-25 15:43:54 JST
Universal time: Fri 2016-11-25 06:43:54 UTC
RTC time: Fri 2016-11-25 06:43:54
Time zone: Asia/Tokyo (JST, +0900)
NTP enabled: yes
NTP synchronized: yes
RTC in local TZ: no
DST active: n/a
○
a,s
自動起動するサービスを確認します。
systemctl list-unit-files | grep enabled | LANG=C sort
NetworkManager-dispatcher.service
enabled
NetworkManager.service
enabled
abrt-ccpp.service
enabled
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 32 12570
12580
12590
12600
12610
12620
12630
12640
12650
12660
12670
12680
12690
12700
12710
12720
12730
12740
12750
12760
12770
12780
12790
12800
12810
12820
12830
12840
12850
12860
12870
12880
12890
12900
12910
12920
12930
12940
12950
12960
12970
abrt-oops.service
abrt-vmcore.service
abrt-xorg.service
abrtd.service
atd.service
auditd.service
autovt@.service
chronyd.service
crond.service
dbus-org.freedesktop.NetworkManager.service
dbus-org.freedesktop.nm-dispatcher.service
default.target
dm-event.socket
getty@.service
irqbalance.service
kdump.service
libstoragemgmt.service
lm_sensors.service
lvm2-lvmetad.socket
lvm2-lvmpolld.socket
lvm2-monitor.service
microcode.service
multi-user.target
psacct.service
remote-fs.target
rngd.service
rpcbind.socket
rsyslog.service
runlevel2.target
runlevel3.target
runlevel4.target
smartd.service
sshd.service
sysstat.service
systemd-readahead-collect.service
systemd-readahead-drop.service
systemd-readahead-replay.service
tuned.service
vmtoolsd.service
○
enabled
enabled
enabled
enabled
enabled
enabled
enabled
enabled
enabled
enabled
enabled
enabled
enabled
enabled
enabled
enabled
enabled
enabled
enabled
enabled
enabled
enabled
enabled
enabled
enabled
enabled
enabled
enabled
enabled
enabled
enabled
enabled
enabled
enabled
enabled
enabled
enabled
enabled
enabled
※　仮想環境の場合、不要
※　LVM を利用しない場合、lvmetad を利用しない場合、不要
※　LVM を利用しない場合、lvmpolld を利用しない場合、不要
※　LVM を利用しない場合、不要
※　nfs を利用しない場合、不要
※　仮想環境、RAID コントローラ未対応の場合、不要
※　ESXi で動かす場合のみ必要
設定ファイルをバックアップします。
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 33 12980
12990 a,s
13000
13010
sudo cp -a /etc{,~}
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 34 13020
13030
13040
13050
13060
13070
13080
13090
13100
13110
13120
13130
13140
13150
13160
13170
13180
13190
13200
13210
13220
13230
13240
13250
13260
13270
13280
13290
13300
13310
13320
13330
13340
13350
13360
13370
13380
13390
13400
13410
13420
○
【iSCSI Target クラスタのインストールと初期設定】
○
以下のインストーラを DVD ドライブにセットします。
a,s
○
a,s
○
a,s
a,s
a,s
a,s
a,s
a,s
○
o
○
# V834394-01.iso (Oracle Linux 7.3)
インストーラをマウントします。
sudo mount /dev/cdrom /mnt
mount: /dev/sr0 is write-protected, mounting read-only
当該 OS で必要となる標準パッケージをインストールします。
sudo yum -y --disablerepo=\* --enablerepo=media,media-mysql,media-ha install \
fence-agents-ipmilan \
omping \
pcs \
rubygem-abrt \
targetcli
インターネットと接続可能な端末で以下のコマンドを実行する等して、必要なパッケージを収集します。
curl -O http://elrepo.org/linux/elrepo/el7/x86_64/RPMS/drbd84-utils- 8.9.6-1.el7.elrepo .x86_64.rpm
収集したパッケージをホームディレクトリにコピーし、確認します。
a,s
scp xxxx@yyy:drbd84-utils- 8.9.6-1.el7.elrepo .x86_64.rpm .
a,s
ls -l *.rpm
-rw-rw-r-- 1 admin admin 410308 Nov 25 16:10 drbd84-utils-8.9.6-1.el7.elrepo .x86_64.rpm
a,s
file *.rpm
drbd84-utils-8.9.6-1.el7.elrepo.x86_64.rpm:
○
a,s
a,s
a,s
○
RPM v3.0 bin i386/x86_64 drbd84-utils-8.9.6-1.el7.elrepo
drbd 管理ツールをインストールします。Oracle 社サポート外のパッケージです。
sudo yum -y --disablerepo=\* --enablerepo=media localinstall drbd84-utils-*.rpm
sudo mv drbd84-utils-*.rpm /opt/packages/
sudo chmod 644 /usr/lib/systemd/system/drbd.service
インストーラをアンマウントします。
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 35 13430
13440
13450
13460
13470
13480
13490
13500
13510
13520
13530
13540
13550
13560
13570
13580
13590
13600
13610
13620
13630
13640
13650
13660
13670
13680
13690
13700
13710
13720
13730
13740
13750
13760
13770
13780
13790
13800
13810
13820
13830
a,s
○
a,s
○
sudo umount /mnt
インストーラをDVDドライブから外します。
# Eject DVD
追加インストールしたパッケージの設定をバックアップします。
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
○
sudo
sudo
sudo
sudo
sudo
sudo
sudo
sudo
sudo
sudo
sudo
sudo
sudo
sudo
sudo
sudo
sudo
sudo
sudo
sudo
sudo
sudo
sudo
sudo
sudo
sudo
cp
cp
cp
cp
cp
cp
cp
cp
cp
cp
cp
cp
cp
cp
cp
cp
cp
cp
cp
cp
cp
cp
cp
cp
cp
cp
-a
-a
-a
-a
-a
-a
-a
-a
-a
-a
-a
-a
-a
-a
-a
-a
-a
-a
-a
-a
-a
-a
-a
-a
-a
-a
/etc{,~}/bash_completion.d/drbdadm
/etc{,~}/corosync
/etc{,~}/dbus-1/system.d/corosync-signals.conf
/etc{,~}/drbd.conf
/etc{,~}/drbd.d
/etc{,~}/ha.d
/etc{,~}/libreport/events.d/ruby_event.conf
/etc{,~}/logrotate.d/corosync
/etc{,~}/logrotate.d/pacemaker
/etc{,~}/logrotate.d/pcsd
/etc{,~}/pam.d/pcsd
/etc{,~}/sysconfig/corosync
/etc{,~}/sysconfig/corosync-notifyd
/etc{,~}/sysconfig/crm_mon
/etc{,~}/sysconfig/pacemaker
/etc{,~}/sysconfig/pcsd
/etc{,~}/target
/etc{,~}/xen
/etc/passwd
/etc~/passwd_$(date +%Y%m%d_%H%M%S)
/etc/passwd- /etc~/passwd-_$(date +%Y%m%d_%H%M%S)
/etc/shadow
/etc~/shadow_$(date +%Y%m%d_%H%M%S)
/etc/shadow- /etc~/shadow-_$(date +%Y%m%d_%H%M%S)
/etc/group
/etc~/group_$(date +%Y%m%d_%H%M%S)
/etc/group/etc~/group-_$(date +%Y%m%d_%H%M%S)
/etc/gshadow /etc~/gshadow_$(date +%Y%m%d_%H%M%S)
/etc/gshadow- /etc~/gshadow-_$(date +%Y%m%d_%H%M%S)
a,s
a,s
a,s
cat << 'EOF' | sudo tee /etc/sysctl.d/tgt.conf
net.core.netdev_max_backlog = 250000
net.core.optmem_max = 16777216
カーネルパラメータを設定します。
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 36 13840
13850
13860
13870
13880
13890
13900
13910
13920
13930
13940
13950
13960
13970
13980
13990
14000
14010
14020
14030
14040
14050
14060
14070
14080
14090
14100
14110
14120
14130
14140
14150
14160
14170
14180
14190
14200
14210
14220
14230
14240
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
○
a,s
○
net.core.rmem_default = 16777216
net.core.rmem_max = 16777216
net.core.wmem_default = 16777216
net.core.wmem_max = 16777216
net.ipv4.tcp_mem = 39363 209944 314904
net.ipv4.tcp_rmem = 8192 87380 16777216
net.ipv4.tcp_wmem = 8192 65536 16777216
net.ipv4.tcp_no_metrics_save = 1
net.ipv4.tcp_sack = 0
net.ipv4.tcp_timestamps = 0
EOF
再起動します。
sudo reboot
管理者用一般ユーザにて、ssh でログインします。
a
a
ssh admin@10.110.88.57
admin@10.110.88.57's password: ********
s
s
ssh admin@10.110.88.58
admin@10.110.88.58's password: ********
○
a,s
カーネルパラメータを確認します。
sysctl -a 2> /dev/null | egrep 'net\.core\..*mem|net\.core\.netdev_m|net\.ipv4\.tcp_.*mem|net\.ipv4\.tcp_no_|net\.ipv4\.tcp_sa|net\.ipv4\.tcp_ti'
net.core.netdev_max_backlog = 250000
net.core.optmem_max = 16777216
net.core.rmem_default = 16777216
net.core.rmem_max = 16777216
net.core.wmem_default = 16777216
net.core.wmem_max = 16777216
net.ipv4.tcp_mem = 39363
209944
net.ipv4.tcp_no_metrics_save = 1
net.ipv4.tcp_rmem = 8192
87380
net.ipv4.tcp_sack = 0
net.ipv4.tcp_timestamps = 0
net.ipv4.tcp_wmem = 8192
65536
○
314904
16777216
16777216
LVM の設定を変更します。
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 37 14250
14260
14270
14280
14290
14300
14310
14320
14330
14340
14350
14360
14370
14380
14390
14400
14410
14420
14430
14440
14450
14460
14470
14480
14490
14500
14510
14520
14530
14540
14550
14560
14570
14580
14590
14600
14610
14620
14630
14640
14650
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
○
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
○
sudo sed -i -e 's/obtain_device_list_from_udev =.*$/obtain_device_list_from_udev = 0/' /etc/lvm/lvm.conf
sudo sed -i -e 's/use_blkid_wiping =.*$/use_blkid_wiping = 0/' /etc/lvm/lvm.conf
sudo sed -i -e 's/use_lvmetad =.*$/use_lvmetad = 0/' /etc/lvm/lvm.conf
sudo sed -i -e 's/use_lvmpolld =.*$/use_lvmpolld = 0/' /etc/lvm/lvm.conf
sudo sed -i -e 's/write_cache_state =.*$/write_cache_state = 0/' /etc/lvm/lvm.conf
sudo sed -i -e 's/readahead =.*$/readahead = "none"/' /etc/lvm/lvm.conf
sudo patch --ignore-whitespace /etc/lvm/lvm.conf << 'EOF'
diff -upr /etc/lvm/lvm.conf /etc/lvm/lvm.conf.new
--- /etc/lvm/lvm.conf
2015-11-21 12:01:29.000000000 +0900
+++ /etc/lvm/lvm.conf.new
2016-09-21 07:52:14.164259555 +0900
@@ -139,6 +139,7 @@ devices {
#
# This configuration option has an automatic default value.
# filter = [ "a|.*/|" ]
+filter = ["r| vg.*|", "a|sd.*|", "a|drbd.*|", "r|.*|"]
# Configuration option devices/global_filter.
# Limit the block devices that are used by LVM system components.
EOF
sudo
sudo
sudo
sudo
sudo
sudo
systemctl stop lvm2-lvmetad.socket
systemctl stop lvm2-lvmpolld.socket
systemctl disable lvm2-lvmetad.socket
systemctl disable lvm2-lvmpolld.socket
rm -f /etc/lvm/cache/.cache
cp -a /etc/lvm/lvm.conf /etc~/lvm/lvm.conf_$(date +%Y%m%d_%H%M%S)
LVM の設定変更を初期化 RAM ディスクに反映します。
for i in /boot/initramfs-*
do
KVER=$(echo $i | sed -n 's%/boot/initramfs-\(.*\).img%\1%p')
if echo $KVER | grep -q -v rescue; then
if echo $KVER | grep -q -v kdump; then
sudo dracut --force /boot/initramfs-$KVER.img $KVER;
fi
fi
done
LVM 物理ボリュームを作成します。
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 38 14660
14670
14680
14690
14700
14710
14720
14730
14740
14750
14760
14770
14780
14790
14800
14810
14820
14830
14840
14850
14860
14870
14880
14890
14900
14910
14920
14930
14940
14950
14960
14970
14980
14990
15000
15010
15020
15030
15040
15050
15060
a,s
sudo pvcreate /dev/sdc
Physical volume "/dev/sdc" successfully created
a,s
sudo pvcreate /dev/sdd
Physical volume "/dev/sdd" successfully created
a,s
sudo pvcreate /dev/sde
Physical volume "/dev/sde" successfully created
a,s
sudo pvcreate /dev/sdf
Physical volume "/dev/sdf" successfully created
○
a,s
○
a,s
○
a,s
LVM ボリュームグループを作成します。
sudo vgcreate -s 4M vg0 /dev/sdc /dev/sdd /dev/sde /dev/sdf
Volume group "vg0" successfully created
LVM 論理ボリュームを作成します。
sudo lvcreate --name lv-drbd0 --extents 90%FREE vg0
Logical volume "lv-drbd0" created.
LVM の状態を確認します。
sudo pvs
PV
/dev/sdc
/dev/sdd
/dev/sde
/dev/sdf
VG
vg0
vg0
vg0
vg0
Fmt
lvm2
lvm2
lvm2
lvm2
Attr
a-a-a-a--
PSize
PFree
100.00g
0
100.00g
0
100.00g
0
100.00g 40.00g
a,s
sudo vgs
VG #PV #LV #SN Attr
VSize
VFree
vg0
4
1
0 wz--n- 399.98g 40.00g
a,s
sudo lvs
LV
VG
lv-drbd0 vg0
○
Attr
LSize
Pool Origin Data%
-wi-a----- 359.98g
Meta%
Move Log Cpy%Sync Convert
DRBD の設定ファイルを作成します。
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 39 15070
15080
15090
15100
15110
15120
15130
15140
15150
15160
15170
15180
15190
15200
15210
15220
15230
15240
15250
15260
15270
15280
15290
15300
15310
15320
15330
15340
15350
15360
15370
15380
15390
15400
15410
15420
15430
15440
15450
15460
15470
a,s
cat /etc/drbd.conf
# You can find an example in
/usr/share/doc/drbd.../drbd.conf.example
include "drbd.d/global_common.conf";
include "drbd.d/*.res";
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
cat << 'EOF' | sudo tee /etc/drbd.d/global_common.conf
global {
usage-count no;
}
common {
handlers {
pri-on-incon-degr "/usr/lib/drbd/notify-pri-on-incon-degr.sh; /usr/lib/drbd/notify-emergency-reboot.sh; echo 1 > /proc/sys/kernel/sysrq; echo b > /proc/sysrq-trigger; reboot -f";
local-io-error "/usr/lib/drbd/notify-io-error.sh; /usr/lib/drbd/notify-emergency-shutdown.sh; echo 1 > /proc/sys/kernel/sysrq; echo o > /proc/sysrq-trigger; halt -f";
fence-peer "/usr/lib/drbd/crm-fence-peer.sh";
before-resync-target "/usr/lib/drbd/snapshot-resync-target-lvm.sh -p 4";
after-resync-target "/usr/lib/drbd/unsnapshot-resync-target-lvm.sh; /usr/lib/drbd/crm-unfence-peer.sh";
}
startup {
#wfc# wfc-timeout 10;
#wfc# degr-wfc-timeout 10;
#wfc# outdated-wfc-timeout 10;
}
disk {
on-io-error detach;
fencing resource-only;
al-extents 6433;
c-plan-ahead 20;
c-delay-target 100;
c-fill-target 0;
c-max-rate 100M;
c-min-rate 1M;
}
net {
protocol C;
max-buffers 128k;
sndbuf-size 0;
rcvbuf-size 0;
cram-hmac-alg sha1;
shared-secret "password";
congestion-fill 100M;
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 40 15480
15490
15500
15510
15520
15530
15540
15550
15560
15570
15580
15590
15600
15610
15620
15630
15640
15650
15660
15670
15680
15690
15700
15710
15720
15730
15740
15750
15760
15770
15780
15790
15800
15810
15820
15830
15840
15850
15860
15870
15880
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
○
a,s
○
a,s
○
congestion-extents 2000;
csums-alg md5;
verify-alg md5;
use-rle yes;
}
}
EOF
sudo cp -a /etc/drbd.d/global_common.conf /etc~/drbd.d/global_common.conf_$(date +%Y%m%d_%H%M%S)
cat << 'EOF' | sudo tee /etc/drbd.d/r0.res
resource r0 {
volume 0 {
device /dev/drbd0;
disk /dev/vg0/lv-drbd0;
meta-disk internal;
}
on iscsitgt01a.example.com {
address 192.168.1.2:7788;
}
on iscsitgt01s.example.com {
address 192.168.1.3:7788;
}
}
EOF
DRBD リソースを初期化します。
sudo drbdadm create-md r0
initializing activity log
NOT initializing bitmap
Writing meta data...
New drbd meta data block successfully created.
targetcli から exit する際に自動的に設定を save する挙動を無効化します。
sudo targetcli set global auto_save_on_exit=false
Warning: Could not load preferences file /root/.targetcli/prefs.bin.
Parameter auto_save_on_exit is now 'false'.
targetcli から target を追加する際に自動的に portal が作成される挙動を無効化します。
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 41 15890 a,s
15900
15910
15920 ○
15930
15940 a,s
15950
15960
15970
15980
15990
16000
16010
16020
16030
16040
16050
16060
16070
16080
16090
16100
16110
16120
16130
16140
16150
16160
16170
16180
16190
16200
16210
16220
16230
16240
16250
16260
16270
16280
16290
sudo targetcli set global auto_add_default_portal=false
Parameter auto_add_default_portal is now 'false'.
targetcli コマンドのデフォルト設定を確認します。
sudo targetcli get global
GLOBAL CONFIG GROUP
===================
auto_add_default_portal=false
---------------------------If true, adds a portal listening on all IPs to new targets.
auto_add_mapped_luns=true
------------------------If true, automatically create node ACLs mapped LUNs after creating a new target LUN or a new node ACL
auto_cd_after_create=false
-------------------------If true, changes current path to newly created objects.
auto_enable_tpgt=true
--------------------If true, automatically enables TPGTs upon creation.
auto_save_on_exit=false
---------------------If true, saves configuration on exit.
color_command=cyan
-----------------Color to use for command completions.
color_default=none
-----------------Default text display color.
color_keyword=cyan
-----------------Color to use for keyword completions.
color_mode=true
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 42 16300
16310
16320
16330
16340
16350
16360
16370
16380
16390
16400
16410
16420
16430
16440
16450
16460
16470
16480
16490
16500
16510
16520
16530
16540
16550
16560
16570
16580
16590
16600
16610
16620
16630
16640
16650
16660
16670
16680
16690
16700
--------------Console color display mode.
color_parameter=magenta
----------------------Color to use for parameter completions.
color_path=magenta
-----------------Color to use for path completions
export_backstore_name_as_model=true
----------------------------------If true, the backstore name is used for the scsi inquiry model name.
logfile=/root/.targetcli/log.txt
-------------------------------Logfile to use.
loglevel_console=info
--------------------Log level for messages going to the console.
loglevel_file=debug
------------------Log level for messages going to the log file.
prompt_length=30
---------------Max length of the shell prompt path, 0 for infinite.
tree_max_depth=0
---------------Maximum depth of displayed node tree.
tree_round_nodes=true
--------------------Tree node display style.
tree_show_root=true
------------------Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 43 16710
16720
16730
16740
16750
16760
16770
16780
16790
16800
16810
16820
16830
16840
16850
16860
16870
16880
16890
16900
16910
16920
16930
16940
16950
16960
16970
16980
16990
17000
17010
17020
17030
17040
17050
17060
17070
17080
17090
17100
17110
Whether or not to display tree root.
tree_status_mode=true
--------------------Whether or not to display status in tree.
○
a,s
a,s
a,s
a,s
○
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
不要なログ出力を抑止します。
cat << 'EOF' | sudo tee /etc/rsyslog.d/ignore-lio.conf
if $programname == "kernel" and $msg contains "MODE SENSE: unimplemented page/subpage:" then stop
EOF
sudo systemctl restart rsyslog
LIO のリソース・エージェントを作成します。
cat << 'EOF_LIO' | sudo tee /usr/lib/ocf/resource.d/heartbeat/LIO
#!/bin/bash
#
#
LIO OCF RA. manages iSCSI target LIO.
#
# (c) 2009-2010 Florian Haas, Dejan Muhamedagic,
#
and Linux-HA contributors
#
#
modified by Katsuaki Hamada (hamada@pc-office.net), 10 Dec 2016
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of version 2 of the GNU General Public License as
# published by the Free Software Foundation.
#
# This program is distributed in the hope that it would be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
#
# Further, this software is distributed without any warranty that it is
# free of the rightful claim of any third person regarding infringement
# or the like. Any license provided herein, whether implied or
# otherwise, applies only to this software file. Patent licenses, if
# any, provided herein do not apply to combinations of this program with
# other software, or any other product whatsoever.
#
# You should have received a copy of the GNU General Public License
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 44 17120
17130
17140
17150
17160
17170
17180
17190
17200
17210
17220
17230
17240
17250
17260
17270
17280
17290
17300
17310
17320
17330
17340
17350
17360
17370
17380
17390
17400
17410
17420
17430
17440
17450
17460
17470
17480
17490
17500
17510
17520
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
# along with this program; if not, write the Free Software Foundation,
# Inc., 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
#
#######################################################################
# Initialization:
: ${OCF_FUNCTIONS_DIR=${OCF_ROOT}/lib/heartbeat}
. ${OCF_FUNCTIONS_DIR}/ocf-shellfuncs
# Lockfile, used for selecting a target ID
LOCKFILE=${HA_RSCTMP}/target.lock
#######################################################################
meta_data() {
cat <<END
<?xml version="1.0"?>
<!DOCTYPE resource-agent SYSTEM "ra-api-1.dtd">
<resource-agent name="LIO" version="1.0">
<version>0.9</version>
<longdesc lang="en">
Manages iSCSI target LIO. An iSCSI target is a collection of SCSI Logical
Units (LUs) exported via a daemon that speaks the iSCSI protocol.
</longdesc>
<shortdesc lang="en">iSCSI target export agent</shortdesc>
<parameters>
<parameter name="iqn" required="0" unique="1">
<longdesc lang="en">
The target iSCSI Qualified Name (IQN). Should follow the conventional
iqn.yyyy-mm.&lt;reversed domain name&gt;[:identifier] syntax.
</longdesc>
<shortdesc lang="en">iSCSI target IQN</shortdesc>
<content type="string" />
</parameter>
</parameters>
<actions>
<action name="start" timeout="10" />
<action name="stop" timeout="10" />
<action name="status" timeout="10" interval="10" depth="0" />
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 45 17530
17540
17550
17560
17570
17580
17590
17600
17610
17620
17630
17640
17650
17660
17670
17680
17690
17700
17710
17720
17730
17740
17750
17760
17770
17780
17790
17800
17810
17820
17830
17840
17850
17860
17870
17880
17890
17900
17910
17920
17930
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
<action name="monitor" timeout="10" interval="10" depth="0" />
<action name="meta-data" timeout="5" />
<action name="validate-all" timeout="10" />
</actions>
</resource-agent>
END
}
#######################################################################
LIO_usage() {
cat <<END
usage: $0 {start|stop|status|monitor|validate-all|meta-data}
Expects to have a fully populated OCF RA-compliant environment set.
END
}
LIO_start() {
LIO_monitor
[ $? = $OCF_SUCCESS ] && return $OCF_SUCCESS
/usr/bin/timeout 3 /usr/bin/targetctl restore
LIO_monitor
}
LIO_stop() {
LIO_monitor
[ $? -eq $OCF_NOT_RUNNING ] && return $OCF_SUCCESS
/usr/bin/timeout 3 /usr/bin/targetctl clear
rc=$?
[ $rc -eq 0 ] && return $OCF_SUCCESS
[ -e /etc/ha.d/noreboot ] && return $rc
echo 1 > /proc/sys/kernel/sysrq
echo s > /proc/sysrq-trigger
echo u > /proc/sysrq-trigger
echo c > /proc/sysrq-trigger
/sbin/reboot -f
return $rc
}
LIO_monitor() {
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 46 17940
17950
17960
17970
17980
17990
18000
18010
18020
18030
18040
18050
18060
18070
18080
18090
18100
18110
18120
18130
18140
18150
18160
18170
18180
18190
18200
18210
18220
18230
18240
18250
18260
18270
18280
18290
18300
18310
18320
18330
18340
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
○
a,s
a,s
for i in /sys/kernel/config/target/iscsi/iqn.*
do
[ -d $i ] && [ $(cat $i/tpgt_1/enable) -eq 1 ] && return $OCF_SUCCESS
done
return $OCF_NOT_RUNNING
}
LIO_validate() {
if ! ocf_is_probe; then
# Do we have all required binaries?
check_binary targetctl
fi
return $OCF_SUCCESS
}
case $1 in
meta-data) meta_data; exit $OCF_SUCCESS;;
usage|help) LIO_usage; exit $OCF_SUCCESS;;
esac
# Everything except usage and meta-data must pass the validate test
LIO_validate
case $__OCF_ACTION in
start)
LIO_start;;
stop)
LIO_stop;;
monitor|status) LIO_monitor;;
reload)
ocf_log err "Reloading..."; LIO_start;;
validate-all)
;;
*)
LIO_usage; exit $OCF_ERR_UNIMPLEMENTED;;
esac
rc=$?
ocf_log debug "${OCF_RESOURCE_INSTANCE} $__OCF_ACTION : $rc"
exit $rc
EOF_LIO
sudo chmod 755 /usr/lib/ocf/resource.d/heartbeat/LIO
VIP に関するリソース・エージェント (IPaddr2) の名前を変更します。
sed -e 's/IPaddr2/VIP/g' /usr/lib/ocf/resource.d/heartbeat/IPaddr2 | sudo tee /usr/lib/ocf/resource.d/heartbeat/VIP > /dev/null
sudo chmod 755 /usr/lib/ocf/resource.d/heartbeat/VIP
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 47 18350
18360
18370
18380
18390
18400
18410
18420
18430
18440
18450
18460
18470
18480
18490
18500
18510
18520
18530
18540
18550
18560
18570
18580
18590
18600
18610
18620
18630
18640
18650
18660
18670
18680
18690
18700
18710
18720
18730
18740
18750
※　LVM, LIO, VIP リソース・エージェント名文字数を統一し、「sudo pcs status」等の実行結果を見やすくします。
※　サポート問い合わせ時は、IPaddr2 を上記のコマンドで変更している点を伝えないと話が通じないものと思われます。
○
pcs の利用環境を整えます。
a,s
echo 'password' | sudo passwd --stdin hacluster
Changing password for user hacluster.
passwd: all authentication tokens updated successfully.
a,s
a,s
sudo cp -a /etc/shadow
sudo cp -a /etc/shadow-
a,s
sudo usermod -a -G haclient admin
a,s
id admin
uid=1000(admin) gid=1000(admin) groups=1000(admin),10(wheel),189(haclient)
a,s
sudo usermod -a -G haclient monitor
a,s
id monitor
uid=1001(monitor) gid=1001(monitor) groups=1001(monitor),189(haclient)
a,s
a,s
sudo cp -a /etc/group
sudo cp -a /etc/group-
a,s
a,s
sudo systemctl start pcsd
sudo systemctl enable pcsd
Created symlink from /etc/systemd/system/multi-user.target.wants/pcsd.service to /usr/lib/systemd/system/pcsd.service.
○
a,s
a,s
a,s
a,s
/etc~/shadow_$(date +%Y%m%d_%H%M%S)
/etc~/shadow-_$(date +%Y%m%d_%H%M%S)
/etc~/group_$(date +%Y%m%d_%H%M%S)
/etc~/group-_$(date +%Y%m%d_%H%M%S)
Corosync のサービス設定を変更します。
sed -e 's/^#Restart=on-failure.*$/Restart=on-failure/' \
-e 's/^#RestartSec=.*$/RestartSec=70/' \
-e 's%^#ExecStartPre=/sbin/modprobe softdog soft_margin=.*$%ExecStartPre=/sbin/modprobe softdog soft_margin=6%' \
/usr/lib/systemd/system/corosync.service | sudo tee /etc/systemd/system/corosync.service
[Unit]
Description=Corosync Cluster Engine
ConditionKernelCommandLine=!nocluster
Requires=network-online.target
After=network-online.target
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 48 18760
18770
18780
18790
18800
18810
18820
18830
18840
18850
18860
18870
18880
18890
18900
18910
18920
18930
18940
18950
18960
18970
18980
18990
19000 a,s
19010
19020
19030
19040
19050
19060
19070
19080
19090
19100
19110
19120 ○
19130
19140 a,s
19150 a,s
19160
[Service]
ExecStart=/usr/share/corosync/corosync start
ExecStop=/usr/share/corosync/corosync stop
Type=forking
# The following config is for corosync with enabled watchdog service.
#
# When corosync watchdog service is being enabled and using with
# pacemaker.service, and if you want to exert the watchdog when a
# corosync process is terminated abnormally,
# uncomment the line of the following Restart= and RestartSec=.
Restart=on-failure
# Specify a period longer than soft_margin as RestartSec.
RestartSec=70
# rewrite according to environment.
ExecStartPre=/sbin/modprobe softdog soft_margin=6
[Install]
WantedBy=multi-user.target
※　カーネル内のソフトウェア watchdog 機能を有効化します。
※　Corosync プロセス障害検知時間を6秒以内とします。
cat /etc/sysconfig/corosync
# Corosync init script configuration file
# COROSYNC_INIT_TIMEOUT specifies number of seconds to wait for corosync
# initialization (default is one minute).
COROSYNC_INIT_TIMEOUT=60
# COROSYNC_OPTIONS specifies options passed to corosync command
# (default is no options).
# See "man corosync" for detailed descriptions of the options.
COROSYNC_OPTIONS=""
Pacemaker のサービス設定を変更します。
sed -e "s%^# ExecStopPost=/bin/sh -c 'pidof crmd || killall -TERM corosync'$%ExecStopPost=/bin/sh -c 'pidof crmd || killall -TERM corosync'%" \
/usr/lib/systemd/system/pacemaker.service | sudo tee /etc/systemd/system/pacemaker.service
[Unit]
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 49 19170
19180
19190
19200
19210
19220
19230
19240
19250
19260
19270
19280
19290
19300
19310
19320
19330
19340
19350
19360
19370
19380
19390
19400
19410
19420
19430
19440
19450
19460
19470
19480
19490
19500
19510
19520
19530
19540
19550
19560
19570
Description=Pacemaker High Availability Cluster Manager
After=dbus.service
After=basic.target
After=syslog.service
After=network.target
After=corosync.service
Requires=dbus.service
Requires=basic.target
Requires=corosync.service
# if you use crm_mon, uncomment the line below.
# Wants=crm_mon.service
[Install]
WantedBy=multi-user.target
[Service]
Type=simple
KillMode=process
NotifyAccess=main
EnvironmentFile=-/etc/sysconfig/pacemaker
EnvironmentFile=-/etc/sysconfig/sbd
SuccessExitStatus=100
ExecStart=/usr/sbin/pacemakerd -f
# If pacemakerd doesn't stop, its probably waiting on a cluster
# resource. Sending -KILL will just get the node fenced
SendSIGKILL=no
#
#
#
#
#
#
#
#
If we ever hit the StartLimitInterval/StartLimitBurst limit and the
admin wants to stop the cluster while pacemakerd is not running, it
might be a good idea to enable the ExecStopPost directive below.
Although the node will likely end up being fenced as a result so its
not on by default
ExecStopPost=/usr/bin/killall -TERM crmd attrd fenced cib pengine lrmd
# If you want Corosync to stop whenever Pacemaker is stopped,
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 50 19580
19590
19600
19610
19620
19630
19640
19650
19660
19670
19680
19690
19700
19710
19720
19730
19740
19750
19760
19770
19780
19790
19800
19810
19820
19830 a,s
19840 a,s
19850 a,s
19860
19870
19880
19890
19900
19910
19920
19930
19940
19950
19960
19970
19980
# uncomment the next line too:
#
ExecStopPost=/bin/sh -c 'pidof crmd || killall -TERM corosync'
# Uncomment this for older versions of systemd that didn't support
# TimeoutStopSec
# TimeoutSec=30min
# Pacemaker can only exit after all managed services have shut down
# A HA database could conceivably take even longer than this
TimeoutStopSec=30min
TimeoutStartSec=60s
# Restart options include: no, on-success, on-failure, on-abort or always
Restart=on-failure
# crm_perror() writes directly to stderr, so ignore it here
# to avoid double-logging with the wrong format
StandardError=null
# if you use crm_mon, uncomment the line below.
# ExecStopPost=/bin/sh -c 'systemctl status crm_mon >/dev/null && systemctl stop crm_mon'
※　Pacemaker サービス停止時に Corosync サービスを停止します。
sudo sed -i -e 's/^# PCMK_fail_fast=.*$/PCMK_fail_fast=yes/' /etc/sysconfig/pacemaker
sudo cp -a /etc/sysconfig/pacemaker /etc~/sysconfig/pacemaker_$(date +%Y%m%d_%H%M%S)
cat /etc/sysconfig/pacemaker
# For non-systemd based systems, prefix export to each enabled line
# Turn on special handling for CMAN clusters in the init script
# Without this, fenced (and by inference, cman) cannot reliably be made to shut down
# PCMK_STACK=cman
#==#==# Variables that control logging
#
#
#
#
Enable debug logging globally or per-subsystem
Multiple subsystems may me listed separated by commas
eg. PCMK_debug=crmd,pengine
PCMK_debug=yes|no|crmd|pengine|cib|stonith-ng|attrd|pacemakerd
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 51 19990
20000
20010
20020
20030
20040
20050
20060
20070
20080
20090
20100
20110
20120
20130
20140
20150
20160
20170
20180
20190
20200
20210
20220
20230
20240
20250
20260
20270
20280
20290
20300
20310
20320
20330
20340
20350
20360
20370
20380
20390
#
#
#
#
Send INFO (and higher) messages to the named log file
Additional messages may also appear here depending on any configured debug and trace settings
By default Pacemaker will inherit the logfile specified in corosync.conf
PCMK_logfile=/var/log/pacemaker.log
#
#
#
#
Specify an alternate syslog target for NOTICE (and higher) messages
Use 'none' to disable - not recommended
The default value is 'daemon'
PCMK_logfacility=none|daemon|user|local0|local1|local2|local3|local4|local5|local6|local7
#
#
#
#
Send all messages up-to-and-including the configured priority to syslog
A value of 'info' will be far too verbose for most installations and 'debug' is almost certain to send you blind
The default value is 'notice'
PCMK_logpriority=emerg|alert|crit|error|warning|notice|info|debug
# Log all messages from a comma-separated list of functions
# PCMK_trace_functions=function1,function2,function3
# Log all messages from a comma-separated list of files (no path)
# Supports wildcards eg. PCMK_trace_files=prefix*.c
# PCMK_trace_files=file.c,other.h
# Log all messages matching comma-separated list of formats
# PCMK_trace_formats="Sent delete %d"
# Log all messages from a comma-separated list of tags
# PCMK_trace_tags=tag1,tag2
# Dump the blackbox whenever the message at function and line is printed
# eg. PCMK_trace_blackbox=te_graph_trigger:223,unpack_clone:81
# PCMK_trace_blackbox=fn:line,fn2:line2,...
#
#
#
#
#
#
#
#
#
Enable blackbox logging globally or per-subsystem
The blackbox contains a rolling buffer of all logs (including info+debug+trace)
and is written after a crash, assertion failure and/or when SIGTRAP is received
The blackbox recorder can also be enabled for Pacemaker daemons at runtime by
sending SIGUSR1 (or SIGTRAP), and disabled by sending SIGUSR2
Multiple subsystems may me listed separated by commas
eg. PCMK_blackbox=crmd,pengine
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 52 20400
20410
20420
20430
20440
20450
20460
20470
20480
20490
20500
20510
20520
20530
20540
20550
20560
20570
20580
20590
20600
20610
20620
20630
20640
20650
20660
20670
20680
20690
20700
20710
20720
20730
20740
20750
20760
20770
20780
20790
20800
# PCMK_blackbox=yes|no|crmd|pengine|cib|stonith-ng|attrd|pacemakerd
#==#==# Advanced use only
# Enable this for compatibility with older corosync (prior to 2.0)
# based clusters which used the nodes uname as its uuid also
# PCMK_uname_is_uuid=no
# Specify an alternate location for RNG schemas and XSL transforms
# Mostly only useful for developer testing
# PCMK_schema_directory=/some/path
# Enable this for rebooting this machine at the time of process (subsystem) failure
PCMK_fail_fast=yes
#==#==# Pacemaker Remote
# Use a custom directory for finding the authkey.
# PCMK_authkey_location=/etc/pacemaker/authkey
#
# Specify a custom port for Pacemaker Remote connections
# PCMK_remote_port=3121
#==#==# IPC
# Force use of a particular class of IPC connection
# PCMK_ipc_type=shared-mem|socket|posix|sysv
# Specify an IPC buffer size in bytes
# Useful when connecting to really big clusters that exceed the default 20k buffer
# PCMK_ipc_buffer=20480
#==#==# Profiling and memory leak testing
#
#
#
#
#
#
#
#
Variables for running child daemons under valgrind and/or checking for memory problems
G_SLICE=always-malloc
MALLOC_PERTURB_=221 # or 0
MALLOC_CHECK_=3
# or 0,1,2
PCMK_valgrind_enabled=yes
PCMK_valgrind_enabled=cib,crmd
PCMK_callgrind_enabled=yes
PCMK_callgrind_enabled=cib,crmd
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 53 20810
20820
20830
20840
20850
20860
20870
20880
20890
20900
20910
20920
20930
20940
20950
20960
20970
20980
20990
21000
21010
21020
21030
21040
21050
21060
21070
21080
21090
21100
21110
21120
21130
21140
21150
21160
21170
21180
21190
21200
21210
# VALGRIND_OPTS="--leak-check=full --trace-children=no --num-callers=25 --log-file=/var/lib/pacemaker/valgrind-%p
--suppressions=/usr/share/pacemaker/tests/valgrind-pcmk.suppressions --gen-suppressions=all"
※　Pacemaker の内部プロセス障害をノード障害として扱うようにします。
○
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
Pacemaker のリソース設定スクリプトを作成します。
cat << 'EOF'
#!/bin/bash
pcs property
pcs property
pcs property
pcs property
pcs property
pcs property
pcs property
pcs property
pcs property
pcs property
pcs property
pcs property
pcs property
pcs property
pcs property
pcs property
pcs property
pcs property
pcs property
pcs property
pcs property
pcs property
pcs property
pcs property
pcs property
pcs property
pcs property
pcs property
pcs property
pcs property
pcs property
pcs property
| sudo tee /etc/ha.d/crm.sh
set
set
set
set
set
set
set
set
set
set
set
set
set
set
set
set
set
set
set
set
set
set
set
set
set
set
set
set
set
set
set
set
batch-limit=30
cluster-delay=60
cluster-recheck-interval=15min
crmd-finalization-timeout=30min
crmd-integration-timeout=3min
crmd-transition-delay=0s
dc-deadtime=20s
default-action-timeout=20
election-timeout=2min
enable-acl=true --force
enable-startup-probes=true
is-managed-default=true
load-threshold=80%
maintenance-mode=false
migration-limit=-1
no-quorum-policy=ignore
node-action-limit=0
node-health-green=0
node-health-red=-INFINITY
node-health-strategy=none
node-health-yellow=0
notification-agent=/dev/null
pe-error-series-max=100
pe-input-series-max=100
pe-warn-series-max=100
placement-strategy=default
remove-after-stop=false
shutdown-escalation=20min
start-failure-is-fatal=true
startup-fencing=true
stonith-action=reboot
stonith-enabled=false
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 54 21220
21230
21240
21250
21260
21270
21280
21290
21300
21310
21320
21330
21340
21350
21360
21370
21380
21390
21400
21410
21420
21430
21440
21450
21460
21470
21480
21490
21500
21510
21520
21530
21540
21550
21560
21570
21580
21590
21600
21610
21620
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
pcs
pcs
pcs
pcs
pcs
property
property
property
property
property
set
set
set
set
set
stonith-timeout=60
stop-all-resources=false
stop-orphan-actions=true
stop-orphan-resources=true
symmetric-cluster=true
pcs resource defaults resource-stickiness=200 migration-threshold=2
pcs acl role create write-access description="Full access" write xpath /cib
pcs acl role create read-only description="Read access to cluster" read xpath /cib
pcs acl user create admin write-access
pcs acl user create monitor read-only
pcs resource create p_drbd_r0 ocf:linbit:drbd \
params drbd_resource=r0 \
op start
timeout=240 \
op monitor interval=10 role=Master timeout=20 \
op monitor interval=20 role=Slave timeout=20 \
op notify
timeout=90 \
op stop
timeout=100 \
op promote
timeout=90 \
op demote
timeout=90
pcs resource master ms_drbd_r0 p_drbd_r0 \
meta master-max=1 master-node-max=1 clone-max=2 \
clone-node-max=1 notify=true target-role=Started \
is-managed=true
pcs resource create p_lvm ocf:heartbeat:LVM \
params volgrpname=vg1 \
op start
timeout=30 \
op monitor interval=5 timeout=10 \
op stop
timeout=30
pcs resource create p_lio ocf:heartbeat:LIO \
op start
timeout=10 \
op monitor interval=5 timeout=5 \
op stop
timeout=10
pcs resource create p_vip ocf:heartbeat:VIP \
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 55 21630
21640
21650
21660
21670
21680
21690
21700
21710
21720
21730
21740
21750
21760
21770
21780
21790
21800
21810
21820
21830
21840
21850
21860
21870
21880
21890
21900
21910
21920
21930
21940
21950
21960
21970
21980
21990
22000
22010
22020
22030
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
params ip=10.110.88.59 cidr_netmask=26 nic=bond0 iflabel=1 arp_interval=200 arp_count=5 \
op start
timeout=20 \
op monitor interval=5 timeout=10 \
op stop
timeout=20
pcs resource group add g_tgt p_lvm p_lio p_vip
pcs constraint location add lc_tgt g_tgt iscsitgt01a.example.com 100
pcs constraint colocation add g_tgt \
ms_drbd_r0 INFINITY with-rsc-role=Master
pcs constraint order promote ms_drbd_r0 then start p_lvm
EOF
sudo chmod 755 /etc/ha.d/crm.sh
sudo cp -a /etc{,~}/ha.d/crm.sh
※　ここからの作業は、Active 機と Stand-by 機が連動して動作していく前提の操作となります。
○
a,s
Active 機と Stand-by 機の間の疎通を確認します。
ping -c 1 -M do -s 8972 10.110.88.57 || echo Error
PING 10.110.88.57 (10.110.88.57) 8972(9000) bytes of data.
8980 bytes from 10.110.88.57: icmp_seq=1 ttl=64 time=0.136 ms
--- 10.110.88.57 ping statistics --1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.136/0.136/0.136/0.000 ms
a,s
traceroute -F 10.110.88.57 8972
traceroute to 10.110.88.57 (10.110.88.57), 30 hops max, 8972 byte packets
1 iscsitgt01a.example.com (10.110.88.57) 0.303 ms 0.265 ms 0.256 ms
a,s
a,s
ping -c 1 -M do -s 8972 10.110.88.58 || echo Error
traceroute -F 10.110.88.58 8972
a,s
a,s
ping -c 1 -M do -s 8972 192.168.1.2 || echo Error
traceroute -F 192.168.1.2 8972
a,s
a,s
ping -c 1 -M do -s 8972 192.168.1.3 || echo Error
traceroute -F 192.168.1.3 8972
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 56 22040
22050 ○
22060
22070
a
22080
a
22090
a
22100
22110
22120
a
22130
22140
a
22150
22160
22170
22180
22190 ○
22200
22210 a,s
22220 a,s
22230
22240 ○
22250
22260 a,s
22270
22280 ○
22290
22300
s
22310
22320
22330
22340
22350
22360
22370
22380 ○
22390
22400
a
22410
22420 ○
22430
22440
Active 機で ssh 鍵を作成し、Stand-by 機にコピーします。
ssh-keygen -q -f ~/.ssh/id_rsa -N ""
mv -f ~/.ssh/id_rsa.pub ~/.ssh/authorized_keys
scp -pr .ssh/ iscsitgt01s:
The authenticity of host 'iscsitgt01s (10.110.88.58)' can't be established.
ECDSA key fingerprint is cf:3a:39:91:fc:c9:ac:5c:4e:16:38:72:97:88:28:b2.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'iscsitgt01s,10.110.88.58' (ECDSA) to the list of known hosts.
admin@iscsitgt01s's password: ********
id_rsa
100% 1679
1.6KB/s
00:00
authorized_keys
100% 411
0.4KB/s
00:00
known_hosts
100% 186
0.2KB/s
00:00
Active 機と Stand-by 機で、ssh 鍵を root アカウント用にコピーします。
sudo cp -a .ssh/ /root/
sudo chown -R root:root /root/.ssh
Active 機と Stand-by 機でほぼ同時に DRBD サービスを起動します。
sudo systemctl start drbd.service
Stand-by 機で DRBD の状態をワッチします。
watch cat /proc/drbd
Every 2.0s: cat /proc/drbd
Fri Nov 25 16:35:43 2016
version: 8.4.5 (api:1/proto:86-101)
srcversion: 1AEFF755B8BD61B81A0AF27
0: cs:Connected ro:Secondary/Secondary ds: Inconsistent/Inconsistent C r----ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:377459420
Active 機で DRBD の初期同期を開始します。
sudo drbdadm primary --force all
Stand-by 機で DRBD の状態を確認します。
Every 2.0s: cat /proc/drbd
Fri Nov 25 16:36:21 2016
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 57 22450
22460
22470
22480
22490
22500
22510
22520
22530
22540
22550
22560
22570
22580
22590
22600
22610
22620
22630
22640
22650
22660
22670
22680
22690
22700
22710
22720
22730
22740
22750
22760
22770
22780
22790
22800
22810
22820
22830
22840
22850
version: 8.4.5 (api:1/proto:86-101)
srcversion: 1AEFF755B8BD61B81A0AF27
0: cs:SyncTarget ro:Secondary/Primary ds:Inconsistent/UpToDate C r----ns:0 nr:0 dw:0 dr:355856 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:377103564
[>....................] sync'ed: 0.1% (368264/368612)M
finish: 0:52:57 speed: 118,616 (118,616) want: 102,400 K/sec
※　この状態でも、Active 機側で作業を続行できます。今回は、初期同期の完了を待つことにします。
Every 2.0s: cat /proc/drbd
Fri Nov 25 17:44:37 2016
version: 8.4.5 (api:1/proto:86-101)
srcversion: 1AEFF755B8BD61B81A0AF27
0: cs:Connected ro:Secondary/Primary ds: UpToDate/UpToDate C r----ns:0 nr:0 dw:0 dr:377459420 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
※　「自機/対向機」がともに「UpToDate/UpToDate 」となっているのが正常な状態です。
○
Active 機で DRBD デバイス上に LVM を構成します。
a
sudo pvcreate /dev/drbd0
Physical volume "/dev/drbd0" successfully created
a
sudo vgcreate -s 4M vg1 /dev/drbd0
Volume group "vg1" successfully created
a
sudo lvcreate --name lv-lun0000 --extents 90%VG vg1
Logical volume "lv-lun0000" created.
a
sudo lvcreate --name lv-lun0001 --extents 2%VG vg1
Logical volume "lv-lun0001" created.
a
sudo lvcreate --name lv-lun0002 --extents 2%VG vg1
Logical volume "lv-lun0002" created.
a
sudo lvcreate --name lv-lun0003 --extents 2%VG vg1
Logical volume "lv-lun0003" created.
a
sudo pvs
PV
VG
Fmt
Attr PSize
PFree
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 58 22860
22870
22880
22890
22900
22910
22920
22930
22940
22950
22960
22970
22980
22990
23000
23010
23020
23030
23040
23050
23060
23070
23080
23090
23100
23110
23120
23130
23140
23150
23160
23170
23180
23190
23200
23210
23220
23230
23240
23250
23260
/dev/drbd0
/dev/sdc1
/dev/sdd1
/dev/sde1
/dev/sdf1
vg1
vg0
vg0
vg0
vg0
lvm2
lvm2
lvm2
lvm2
lvm2
a-a-a-a-a--
359.97g 14.40g
100.00g
0
100.00g
0
100.00g
0
100.00g 40.00g
a
sudo vgs
VG
#PV #LV #SN Attr
VSize
VFree
vg0
4
1
0 wz--n- 399.98g 40.00g
vg1
1
4
0 wz--n- 359.97g 14.40g
a
sudo lvs
LV
lv-drbd0
lv-lun0000
lv-lun0001
lv-lun0002
lv-lun0003
○
a
○
VG
vg0
vg1
vg1
vg1
vg1
Attr
LSize
Pool Origin Data%
-wi-ao---- 359.98g
-wi-a----- 323.97g
-wi-a----7.20g
-wi-a----7.20g
-wi-a----7.20g
Meta%
Move Log Cpy%Sync Convert
Active 機で、targetcli から状態を確認します。
sudo targetcli ls /
o- / ..................................................................... [...]
o- backstores .......................................................... [...]
| o- block .............................................. [Storage Objects: 0]
| o- fileio ............................................. [Storage Objects: 0]
| o- pscsi .............................................. [Storage Objects: 0]
| o- ramdisk ............................................ [Storage Objects: 0]
o- iscsi ........................................................ [Targets: 0]
o- loopback ..................................................... [Targets: 0]
Active 機で、IQN を定義します。
a
sudo targetcli /iscsi create iqn.2016-09.com.example:iscsitgt01-0000
Created target iqn.2016-09.com.example:iscsitgt01-0000 .
Created TPG 1.
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0000/tpg1 set attribute default_cmdsn_depth = 128
Parameter default_cmdsn_depth is now '128'.
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0000/tpg1 set parameter MaxConnections = 1
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 59 23270
23280
23290
23300
23310
23320
23330
23340
23350
23360
23370
23380
23390
23400
23410
23420
23430
23440
23450
23460
23470
23480
23490
23500
23510
23520
23530
23540
23550
23560
23570
23580
23590
23600
23610
23620
23630
23640
23650
23660
23670
Parameter MaxConnections is now '1'.
a
sudo targetcli /iscsi create iqn.2016-09.com.example:iscsitgt01-0001
Created target iqn.2016-09.com.example:iscsitgt01-0001 .
Created TPG 1.
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0001/tpg1 set attribute default_cmdsn_depth = 128
Parameter default_cmdsn_depth is now '128'.
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0001/tpg1 set parameter MaxConnections = 1
Parameter MaxConnections is now '1'.
a
sudo targetcli /iscsi create iqn.2016-09.com.example:iscsitgt01-0002
Created target iqn.2016-09.com.example:iscsitgt01-0002 .
Created TPG 1.
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0002/tpg1 set attribute default_cmdsn_depth = 128
Parameter default_cmdsn_depth is now '128'.
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0002/tpg1 set parameter MaxConnections = 1
Parameter MaxConnections is now '1'.
a
sudo targetcli /iscsi create iqn.2016-09.com.example:iscsitgt01-0003
Created target iqn.2016-09.com.example:iscsitgt01-0003 .
Created TPG 1.
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0003/tpg1 set attribute default_cmdsn_depth = 128
Parameter default_cmdsn_depth is now '128'.
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0003/tpg1 set parameter MaxConnections = 1
Parameter MaxConnections is now '1'.
a
sudo targetcli ls /
o- / ..................................................................... [...]
o- backstores .......................................................... [...]
| o- block .............................................. [Storage Objects: 0]
| o- fileio ............................................. [Storage Objects: 0]
| o- pscsi .............................................. [Storage Objects: 0]
| o- ramdisk ............................................ [Storage Objects: 0]
o- iscsi ........................................................ [Targets: 4]
| o- iqn.2016-09.com.example:iscsitgt01-0000 ....................... [TPGs: 1]
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 60 23680
23690
23700
23710
23720
23730
23740
23750
23760
23770
23780
23790
23800
23810
23820
23830
23840
23850
23860
23870
23880
23890
23900
23910
23920
23930
23940
23950
23960
23970
23980
23990
24000
24010
24020
24030
24040
24050
24060
24070
24080
| | o- tpg1 ........................................... [no-gen-acls, no-auth]
| |
o- acls ...................................................... [ACLs: 0]
| |
o- luns ...................................................... [LUNs: 0]
| |
o- portals ................................................ [Portals: 0]
| o- iqn.2016-09.com.example:iscsitgt01-0001 ....................... [TPGs: 1]
| | o- tpg1 ........................................... [no-gen-acls, no-auth]
| |
o- acls ...................................................... [ACLs: 0]
| |
o- luns ...................................................... [LUNs: 0]
| |
o- portals ................................................ [Portals: 0]
| o- iqn.2016-09.com.example:iscsitgt01-0002 ....................... [TPGs: 1]
| | o- tpg1 ........................................... [no-gen-acls, no-auth]
| |
o- acls ...................................................... [ACLs: 0]
| |
o- luns ...................................................... [LUNs: 0]
| |
o- portals ................................................ [Portals: 0]
| o- iqn.2016-09.com.example:iscsitgt01-0003 ....................... [TPGs: 1]
| o- tpg1 ........................................... [no-gen-acls, no-auth]
|
o- acls ...................................................... [ACLs: 0]
|
o- luns ...................................................... [LUNs: 0]
|
o- portals ................................................ [Portals: 0]
o- loopback ..................................................... [Targets: 0]
○
a
a
Active 機で、ACL（アクセス許可リスト）にイニシエータ名を登録します。必要に応じて CHAP 認証情報も紐付けします。
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0000/tpg1/acls create iqn.2016-09.com.example:initiator01
Created Node ACL for iqn.2016-09.com.example:initiator01
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0000/tpg1/acls/iqn.2016-09.com.example:initiator01 set auth userid=iscsiuser01
Parameter userid is now 'iscsiuser01'.
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0000/tpg1/acls/iqn.2016-09.com.example:initiator01 set auth password='password-user01'
Parameter password is now 'password-user01 '.
a
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0001/tpg1/acls create iqn.2016-09.com.example:initiator01
Created Node ACL for iqn.2016-09.com.example:initiator01
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0001/tpg1/acls/iqn.2016-09.com.example:initiator01 set auth userid=iscsiuser01
Parameter userid is now 'iscsiuser01'.
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0001/tpg1/acls/iqn.2016-09.com.example:initiator01 set auth password='password-user01'
Parameter password is now 'password-user01 '.
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 61 24090
24100
24110
24120
24130
24140
24150
24160
24170
24180
24190
24200
24210
24220
24230
24240
24250
24260
24270
24280
24290
24300
24310
24320
24330
24340
24350
24360
24370
24380
24390
24400
24410
24420
24430
24440
24450
24460
24470
24480
24490
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0002/tpg1/acls create iqn.2016-09.com.example:initiator01
Created Node ACL for iqn.2016-09.com.example:initiator01
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0002/tpg1/acls/iqn.2016-09.com.example:initiator01 set auth userid=iscsiuser01
Parameter userid is now 'iscsiuser01'.
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0002/tpg1/acls/iqn.2016-09.com.example:initiator01 set auth password='password-user01'
Parameter password is now 'password-user01 '.
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0003/tpg1/acls create iqn.2016-09.com.example:initiator01
Created Node ACL for iqn.2016-09.com.example:initiator01
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0003/tpg1/acls/iqn.2016-09.com.example:initiator01 set auth userid=iscsiuser01
Parameter userid is now 'iscsiuser01'.
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0003/tpg1/acls/iqn.2016-09.com.example:initiator01 set auth password='password-user01'
Parameter password is now 'password-user01 '.
a
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0000/tpg1/acls create iqn.2016-09.com.example:initiator02
Created Node ACL for iqn.2016-09.com.example:initiator02
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0000/tpg1/acls/iqn.2016-09.com.example:initiator02 set auth userid=iscsiuser02
Parameter userid is now 'iscsiuser02'.
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0000/tpg1/acls/iqn.2016-09.com.example:initiator02 set auth password='password-user02'
Parameter password is now 'password-user02 '.
a
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0001/tpg1/acls create iqn.2016-09.com.example:initiator02
Created Node ACL for iqn.2016-09.com.example:initiator02
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0001/tpg1/acls/iqn.2016-09.com.example:initiator02 set auth userid=iscsiuser02
Parameter userid is now 'iscsiuser02'.
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0001/tpg1/acls/iqn.2016-09.com.example:initiator02 set auth password='password-user02'
Parameter password is now 'password-user02 '.
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0002/tpg1/acls create iqn.2016-09.com.example:initiator02
Created Node ACL for iqn.2016-09.com.example:initiator02
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0002/tpg1/acls/iqn.2016-09.com.example:initiator02 set auth userid=iscsiuser02
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 62 24500
24510
24520
24530
24540
24550
24560
24570
24580
24590
24600
24610
24620
24630
24640
24650
24660
24670
24680
24690
24700
24710
24720
24730
24740
24750
24760
24770
24780
24790
24800
24810
24820
24830
24840
24850
24860
24870
24880
24890
24900
Parameter userid is now 'iscsiuser02'.
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0002/tpg1/acls/iqn.2016-09.com.example:initiator02 set auth password='password-user02'
Parameter password is now 'password-user02 '.
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0003/tpg1/acls create iqn.2016-09.com.example:initiator02
Created Node ACL for iqn.2016-09.com.example:initiator02
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0003/tpg1/acls/iqn.2016-09.com.example:initiator02 set auth userid=iscsiuser02
Parameter userid is now 'iscsiuser02'.
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0003/tpg1/acls/iqn.2016-09.com.example:initiator02 set auth password='password-user02'
Parameter password is now 'password-user02 '.
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0000/tpg1/acls create iqn.2016-09.com.example:initiator03
Created Node ACL for iqn.2016-09.com.example:initiator03
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0000/tpg1/acls/iqn.2016-09.com.example:initiator03 set auth userid=iscsiuser03
Parameter userid is now 'iscsiuser03'.
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0000/tpg1/acls/iqn.2016-09.com.example:initiator03 set auth password='password-user03'
Parameter password is now 'password-user03 '.
a
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0001/tpg1/acls create iqn.2016-09.com.example:initiator03
Created Node ACL for iqn.2016-09.com.example:initiator03
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0001/tpg1/acls/iqn.2016-09.com.example:initiator03 set auth userid=iscsiuser03
Parameter userid is now 'iscsiuser03'.
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0001/tpg1/acls/iqn.2016-09.com.example:initiator03 set auth password='password-user03'
Parameter password is now 'password-user03 '.
a
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0002/tpg1/acls create iqn.2016-09.com.example:initiator03
Created Node ACL for iqn.2016-09.com.example:initiator03
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0002/tpg1/acls/iqn.2016-09.com.example:initiator03 set auth userid=iscsiuser03
Parameter userid is now 'iscsiuser03'.
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0002/tpg1/acls/iqn.2016-09.com.example:initiator03 set auth password='password-user03'
Parameter password is now 'password-user03 '.
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 63 24910
24920
24930
24940
24950
24960
24970
24980
24990
25000
25010
25020
25030
25040
25050
25060
25070
25080
25090
25100
25110
25120
25130
25140
25150
25160
25170
25180
25190
25200
25210
25220
25230
25240
25250
25260
25270
25280
25290
25300
25310
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0003/tpg1/acls create iqn.2016-09.com.example:initiator03
Created Node ACL for iqn.2016-09.com.example:initiator03
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0003/tpg1/acls/iqn.2016-09.com.example:initiator03 set auth userid=iscsiuser03
Parameter userid is now 'iscsiuser03'.
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0003/tpg1/acls/iqn.2016-09.com.example:initiator03 set auth password='password-user03'
Parameter password is now 'password-user03 '.
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0000/tpg1/acls create iqn.2016-09.com.example:initiator04
Created Node ACL for iqn.2016-09.com.example:initiator04
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0000/tpg1/acls/iqn.2016-09.com.example:initiator04 set auth userid=iscsiuser04
Parameter userid is now 'iscsiuser04'.
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0000/tpg1/acls/iqn.2016-09.com.example:initiator04 set auth password='password-user04'
Parameter password is now 'password-user04 '.
a
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0001/tpg1/acls create iqn.2016-09.com.example:initiator04
Created Node ACL for iqn.2016-09.com.example:initiator04
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0001/tpg1/acls/iqn.2016-09.com.example:initiator04 set auth userid=iscsiuser04
Parameter userid is now 'iscsiuser04'.
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0001/tpg1/acls/iqn.2016-09.com.example:initiator04 set auth password='password-user04'
Parameter password is now 'password-user04 '.
a
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0002/tpg1/acls create iqn.2016-09.com.example:initiator04
Created Node ACL for iqn.2016-09.com.example:initiator04
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0002/tpg1/acls/iqn.2016-09.com.example:initiator04 set auth userid=iscsiuser04
Parameter userid is now 'iscsiuser04'.
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0002/tpg1/acls/iqn.2016-09.com.example:initiator04 set auth password='password-user04'
Parameter password is now 'password-user04 '.
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0003/tpg1/acls create iqn.2016-09.com.example:initiator04
Created Node ACL for iqn.2016-09.com.example:initiator04
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 64 25320
25330
25340
25350
25360
25370
25380
25390
25400
25410
25420
25430
25440
25450
25460
25470
25480
25490
25500
25510
25520
25530
25540
25550
25560
25570
25580
25590
25600
25610
25620
25630
25640
25650
25660
25670
25680
25690
25700
25710
25720
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0003/tpg1/acls/iqn.2016-09.com.example:initiator04 set auth userid=iscsiuser04
Parameter userid is now 'iscsiuser04'.
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0003/tpg1/acls/iqn.2016-09.com.example:initiator04 set auth password='password-user04'
Parameter password is now 'password-user04 '.
a
sudo targetcli ls /
o- / ..................................................................... [...]
o- backstores .......................................................... [...]
| o- block .............................................. [Storage Objects: 0]
| o- fileio ............................................. [Storage Objects: 0]
| o- pscsi .............................................. [Storage Objects: 0]
| o- ramdisk ............................................ [Storage Objects: 0]
o- iscsi ........................................................ [Targets: 4]
| o- iqn.2016-09.com.example:iscsitgt01-0000 ....................... [TPGs: 1]
| | o- tpg1 ........................................... [no-gen-acls, no-auth]
| |
o- acls ...................................................... [ACLs: 4]
| |
| o- iqn.2016-09.com.example:initiator01 .............. [Mapped LUNs: 0]
| |
| o- iqn.2016-09.com.example:initiator02 .............. [Mapped LUNs: 0]
| |
| o- iqn.2016-09.com.example:initiator03 .............. [Mapped LUNs: 0]
| |
| o- iqn.2016-09.com.example:initiator04 .............. [Mapped LUNs: 0]
| |
o- luns ...................................................... [LUNs: 0]
| |
o- portals ................................................ [Portals: 0]
| o- iqn.2016-09.com.example:iscsitgt01-0001 ....................... [TPGs: 1]
| | o- tpg1 ........................................... [no-gen-acls, no-auth]
| |
o- acls ...................................................... [ACLs: 4]
| |
| o- iqn.2016-09.com.example:initiator01 .............. [Mapped LUNs: 0]
| |
| o- iqn.2016-09.com.example:initiator02 .............. [Mapped LUNs: 0]
| |
| o- iqn.2016-09.com.example:initiator03 .............. [Mapped LUNs: 0]
| |
| o- iqn.2016-09.com.example:initiator04 .............. [Mapped LUNs: 0]
| |
o- luns ...................................................... [LUNs: 0]
| |
o- portals ................................................ [Portals: 0]
| o- iqn.2016-09.com.example:iscsitgt01-0002 ....................... [TPGs: 1]
| | o- tpg1 ........................................... [no-gen-acls, no-auth]
| |
o- acls ...................................................... [ACLs: 4]
| |
| o- iqn.2016-09.com.example:initiator01 .............. [Mapped LUNs: 0]
| |
| o- iqn.2016-09.com.example:initiator02 .............. [Mapped LUNs: 0]
| |
| o- iqn.2016-09.com.example:initiator03 .............. [Mapped LUNs: 0]
| |
| o- iqn.2016-09.com.example:initiator04 .............. [Mapped LUNs: 0]
| |
o- luns ...................................................... [LUNs: 0]
| |
o- portals ................................................ [Portals: 0]
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 65 25730
25740
25750
25760
25770
25780
25790
25800
25810
25820
25830
25840
25850
25860
25870
25880
25890
25900
25910
25920
25930
25940
25950
25960
25970
25980
25990
26000
26010
26020
26030
26040
26050
26060
26070
26080
26090
26100
26110
26120
26130
| o- iqn.2016-09.com.example:iscsitgt01-0003 ....................... [TPGs: 1]
| o- tpg1 ........................................... [no-gen-acls, no-auth]
|
o- acls ...................................................... [ACLs: 4]
|
| o- iqn.2016-09.com.example:initiator01 .............. [Mapped LUNs: 0]
|
| o- iqn.2016-09.com.example:initiator02 .............. [Mapped LUNs: 0]
|
| o- iqn.2016-09.com.example:initiator03 .............. [Mapped LUNs: 0]
|
| o- iqn.2016-09.com.example:initiator04 .............. [Mapped LUNs: 0]
|
o- luns ...................................................... [LUNs: 0]
|
o- portals ................................................ [Portals: 0]
o- loopback ..................................................... [Targets: 0]
○
Active 機で、バックエンド・デバイスを指定し、IQN に紐付けます。
a
sudo targetcli /backstores/block create name=lun0000 dev=/dev/vg1/lv-lun0000
Created block storage object lun0000 using /dev/ vg1/lv-lun0000.
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0000/tpg1/luns create /backstores/block/lun0000
Created LUN 0.
Created LUN 0->0 mapping in node ACL iqn.2016-09.com.example:initiator04
Created LUN 0->0 mapping in node ACL iqn.2016-09.com.example:initiator03
Created LUN 0->0 mapping in node ACL iqn.2016-09.com.example:initiator02
Created LUN 0->0 mapping in node ACL iqn.2016-09.com.example:initiator01
a
sudo targetcli /backstores/block create name=lun0001 dev=/dev/vg1/lv-lun0001
Created block storage object lun0001 using /dev/ vg1/lv-lun0001.
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0001/tpg1/luns create /backstores/block/lun0001
Created LUN 0.
Created LUN 0->0 mapping in node ACL iqn.2016-09.com.example:initiator04
Created LUN 0->0 mapping in node ACL iqn.2016-09.com.example:initiator03
Created LUN 0->0 mapping in node ACL iqn.2016-09.com.example:initiator02
Created LUN 0->0 mapping in node ACL iqn.2016-09.com.example:initiator01
a
sudo targetcli /backstores/block create name=lun0002 dev=/dev/vg1/lv-lun0002
Created block storage object lun0002 using /dev/ vg1/lv-lun0002.
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0002/tpg1/luns create /backstores/block/lun0002
Created LUN 0.
Created LUN 0->0 mapping in node ACL iqn.2016-09.com.example:initiator04
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 66 26140
26150
26160
26170
26180
26190
26200
26210
26220
26230
26240
26250
26260
26270
26280
26290
26300
26310
26320
26330
26340
26350
26360
26370
26380
26390
26400
26410
26420
26430
26440
26450
26460
26470
26480
26490
26500
26510
26520
26530
26540
Created LUN 0->0 mapping in node ACL iqn.2016-09.com.example:initiator03
Created LUN 0->0 mapping in node ACL iqn.2016-09.com.example:initiator02
Created LUN 0->0 mapping in node ACL iqn.2016-09.com.example:initiator01
a
sudo targetcli /backstores/block create name=lun0003 dev=/dev/vg1/lv-lun0003
Created block storage object lun0003 using /dev/ vg1/lv-lun0003.
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0003/tpg1/luns create /backstores/block/lun0003
Created LUN 0.
Created LUN 0->0 mapping in node ACL iqn.2016-09.com.example:initiator04
Created LUN 0->0 mapping in node ACL iqn.2016-09.com.example:initiator03
Created LUN 0->0 mapping in node ACL iqn.2016-09.com.example:initiator02
Created LUN 0->0 mapping in node ACL iqn.2016-09.com.example:initiator01
a
sudo targetcli ls /
o- / ..................................................................... [...]
o- backstores .......................................................... [...]
| o- block .............................................. [Storage Objects: 4]
| | o- lun0000 ......... [/dev/vg1/lv-lun0000 (324.0GiB) write-thru activated]
| | o- lun0001 ........... [/dev/vg1/lv-lun0001 (7.2GiB) write-thru activated]
| | o- lun0002 ........... [/dev/vg1/lv-lun0002 (7.2GiB) write-thru activated]
| | o- lun0003 ........... [/dev/vg1/lv-lun0003 (7.2GiB) write-thru activated]
| o- fileio ............................................. [Storage Objects: 0]
| o- pscsi .............................................. [Storage Objects: 0]
| o- ramdisk ............................................ [Storage Objects: 0]
o- iscsi ........................................................ [Targets: 4]
| o- iqn.2016-09.com.example:iscsitgt01-0000 ....................... [TPGs: 1]
| | o- tpg1 ........................................... [no-gen-acls, no-auth]
| |
o- acls ...................................................... [ACLs: 4]
| |
| o- iqn.2016-09.com.example:initiator01 .............. [Mapped LUNs: 1]
| |
| | o- mapped_lun0 ........................... [lun0 block/lun0000 (rw)]
| |
| o- iqn.2016-09.com.example:initiator02 .............. [Mapped LUNs: 1]
| |
| | o- mapped_lun0 ........................... [lun0 block/lun0000 (rw)]
| |
| o- iqn.2016-09.com.example:initiator03 .............. [Mapped LUNs: 1]
| |
| | o- mapped_lun0 ........................... [lun0 block/lun0000 (rw)]
| |
| o- iqn.2016-09.com.example:initiator04 .............. [Mapped LUNs: 1]
| |
|
o- mapped_lun0 ........................... [lun0 block/lun0000 (rw)]
| |
o- luns ...................................................... [LUNs: 4]
| |
| o- lun0 ........................ [block/lun0000 (/dev/vg1/lv-lun0000)]
| o- iqn.2016-09.com.example:iscsitgt01-0001 ....................... [TPGs: 1]
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 67 26550
26560
26570
26580
26590
26600
26610
26620
26630
26640
26650
26660
26670
26680
26690
26700
26710
26720
26730
26740
26750
26760
26770
26780
26790
26800
26810
26820
26830
26840
26850
26860
26870
26880
26890
26900
26910
26920
26930
26940
26950
| | o- tpg1 ........................................... [no-gen-acls, no-auth]
| |
o- acls ...................................................... [ACLs: 4]
| |
| o- iqn.2016-09.com.example:initiator01 .............. [Mapped LUNs: 1]
| |
| | o- mapped_lun0 ........................... [lun0 block/lun0001 (rw)]
| |
| o- iqn.2016-09.com.example:initiator02 .............. [Mapped LUNs: 1]
| |
| | o- mapped_lun0 ........................... [lun0 block/lun0001 (rw)]
| |
| o- iqn.2016-09.com.example:initiator03 .............. [Mapped LUNs: 1]
| |
| | o- mapped_lun0 ........................... [lun0 block/lun0001 (rw)]
| |
| o- iqn.2016-09.com.example:initiator04 .............. [Mapped LUNs: 1]
| |
|
o- mapped_lun0 ........................... [lun0 block/lun0001 (rw)]
| |
o- luns ...................................................... [LUNs: 1]
| |
| o- lun0 ........................ [block/lun0001 (/dev/vg1/lv-lun0001)]
| o- iqn.2016-09.com.example:iscsitgt01-0002 ....................... [TPGs: 1]
| | o- tpg1 ........................................... [no-gen-acls, no-auth]
| |
o- acls ...................................................... [ACLs: 4]
| |
| o- iqn.2016-09.com.example:initiator01 .............. [Mapped LUNs: 1]
| |
| | o- mapped_lun0 ........................... [lun0 block/lun0002 (rw)]
| |
| o- iqn.2016-09.com.example:initiator02 .............. [Mapped LUNs: 1]
| |
| | o- mapped_lun0 ........................... [lun0 block/lun0002 (rw)]
| |
| o- iqn.2016-09.com.example:initiator03 .............. [Mapped LUNs: 1]
| |
| | o- mapped_lun0 ........................... [lun0 block/lun0002 (rw)]
| |
| o- iqn.2016-09.com.example:initiator04 .............. [Mapped LUNs: 1]
| |
|
o- mapped_lun0 ........................... [lun0 block/lun0002 (rw)]
| |
o- luns ...................................................... [LUNs: 1]
| |
| o- lun0 ........................ [block/lun0002 (/dev/vg1/lv-lun0002)]
| o- iqn.2016-09.com.example:iscsitgt01-0003 ....................... [TPGs: 1]
| o- tpg1 ........................................... [no-gen-acls, no-auth]
|
o- acls ...................................................... [ACLs: 4]
|
| o- iqn.2016-09.com.example:initiator01 .............. [Mapped LUNs: 1]
|
| | o- mapped_lun0 ........................... [lun0 block/lun0003 (rw)]
|
| o- iqn.2016-09.com.example:initiator02 .............. [Mapped LUNs: 1]
|
| | o- mapped_lun0 ........................... [lun0 block/lun0003 (rw)]
|
| o- iqn.2016-09.com.example:initiator03 .............. [Mapped LUNs: 1]
|
| | o- mapped_lun0 ........................... [lun0 block/lun0003 (rw)]
|
| o- iqn.2016-09.com.example:initiator04 .............. [Mapped LUNs: 1]
|
|
o- mapped_lun0 ........................... [lun0 block/lun0003 (rw)]
|
o- luns ...................................................... [LUNs: 1]
|
| o- lun0 ........................ [block/lun0003 (/dev/vg1/lv-lun0003)]
|
o- portals ................................................ [Portals: 0]
o- loopback ..................................................... [Targets: 0]
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 68 26960
26970
26980
26990
27000
27010
27020
27030
27040
27050
27060
27070
27080
27090
27100
27110
27120
27130
27140
27150
27160
27170
27180
27190
27200
27210
27220
27230
27240
27250
27260
27270
27280
27290
27300
27310
27320
27330
27340
27350
27360
○
Active 機で、IQN に portal を作成します。
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0000/tpg1/portals create 10.110.88.59 3260
Using default IP port 3260
Created network portal 10.110.88.59:3260.
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0001/tpg1/portals create 10.110.88.59 3260
Using default IP port 3260
Created network portal 10.110.88.59:3260.
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0002/tpg1/portals create 10.110.88.59 3260
Using default IP port 3260
Created network portal 10.110.88.59:3260.
a
sudo targetcli /iscsi/iqn.2016-09.com.example:iscsitgt01-0003/tpg1/portals create 10.110.88.59 3260
Using default IP port 3260
Created network portal 10.110.88.59:3260.
a
sudo targetcli ls /
o- / ..................................................................... [...]
o- backstores .......................................................... [...]
| o- block .............................................. [Storage Objects: 4]
| | o- lun0000 ......... [/dev/vg1/lv-lun0000 (324.0GiB) write-thru activated]
| | o- lun0001 ........... [/dev/vg1/lv-lun0001 (7.2GiB) write-thru activated]
| | o- lun0002 ........... [/dev/vg1/lv-lun0002 (7.2GiB) write-thru activated]
| | o- lun0003 ........... [/dev/vg1/lv-lun0003 (7.2GiB) write-thru activated]
| o- fileio ............................................. [Storage Objects: 0]
| o- pscsi .............................................. [Storage Objects: 0]
| o- ramdisk ............................................ [Storage Objects: 0]
o- iscsi ........................................................ [Targets: 4]
| o- iqn.2016-09.com.example:iscsitgt01-0000 ....................... [TPGs: 1]
| o- tpg1 ........................................... [no-gen-acls, no-auth]
|
o- acls ...................................................... [ACLs: 4]
|
| o- iqn.2016-09.com.example:initiator01 .............. [Mapped LUNs: 4]
| |
| | o- mapped_lun0 ........................... [lun0 block/lun0000 (rw)]
| |
| o- iqn.2016-09.com.example:initiator02 .............. [Mapped LUNs: 1]
| |
| | o- mapped_lun0 ........................... [lun0 block/lun0000 (rw)]
| |
| o- iqn.2016-09.com.example:initiator03 .............. [Mapped LUNs: 1]
| |
| | o- mapped_lun0 ........................... [lun0 block/lun0000 (rw)]
| |
| o- iqn.2016-09.com.example:initiator04 .............. [Mapped LUNs: 1]
| |
|
o- mapped_lun0 ........................... [lun0 block/lun0000 (rw)]
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 69 27370
27380
27390
27400
27410
27420
27430
27440
27450
27460
27470
27480
27490
27500
27510
27520
27530
27540
27550
27560
27570
27580
27590
27600
27610
27620
27630
27640
27650
27660
27670
27680
27690
27700
27710
27720
27730
27740
27750
27760
27770
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
o- luns ...................................................... [LUNs: 1]
|
| o- lun0 ........................ [block/lun0000 (/dev/vg1/lv-lun0000)]
|
o- portals ................................................ [Portals: 1]
|
o- 10.110.88.59:3260 ............................................ [OK]
o- iqn.2016-09.com.example:iscsitgt01-0001 ....................... [TPGs: 1]
| o- tpg1 ........................................... [no-gen-acls, no-auth]
|
o- acls ...................................................... [ACLs: 4]
|
| o- iqn.2016-09.com.example:initiator01 .............. [Mapped LUNs: 1]
|
| | o- mapped_lun0 ........................... [lun0 block/lun0001 (rw)]
|
| o- iqn.2016-09.com.example:initiator02 .............. [Mapped LUNs: 1]
|
| | o- mapped_lun0 ........................... [lun0 block/lun0001 (rw)]
|
| o- iqn.2016-09.com.example:initiator03 .............. [Mapped LUNs: 1]
|
| | o- mapped_lun0 ........................... [lun0 block/lun0001 (rw)]
|
| o- iqn.2016-09.com.example:initiator04 .............. [Mapped LUNs: 1]
|
|
o- mapped_lun0 ........................... [lun0 block/lun0001 (rw)]
|
o- luns ...................................................... [LUNs: 1]
|
| o- lun0 ........................ [block/lun0001 (/dev/vg1/lv-lun0001)]
|
o- portals ................................................ [Portals: 1]
|
o- 10.110.88.59:3260 ............................................ [OK]
o- iqn.2016-09.com.example:iscsitgt01-0002 ....................... [TPGs: 1]
| o- tpg1 ........................................... [no-gen-acls, no-auth]
|
o- acls ...................................................... [ACLs: 4]
|
| o- iqn.2016-09.com.example:initiator01 .............. [Mapped LUNs: 1]
|
| | o- mapped_lun0 ........................... [lun0 block/lun0002 (rw)]
|
| o- iqn.2016-09.com.example:initiator02 .............. [Mapped LUNs: 1]
|
| | o- mapped_lun0 ........................... [lun0 block/lun0002 (rw)]
|
| o- iqn.2016-09.com.example:initiator03 .............. [Mapped LUNs: 1]
|
| | o- mapped_lun0 ........................... [lun0 block/lun0002 (rw)]
|
| o- iqn.2016-09.com.example:initiator04 .............. [Mapped LUNs: 1]
|
|
o- mapped_lun0 ........................... [lun0 block/lun0002 (rw)]
|
o- luns ...................................................... [LUNs: 1]
|
| o- lun0 ........................ [block/lun0002 (/dev/vg1/lv-lun0002)]
|
o- portals ................................................ [Portals: 1]
|
o- 10.110.88.59:3260 ............................................ [OK]
o- iqn.2016-09.com.example:iscsitgt01-0003 ....................... [TPGs: 1]
o- tpg1 ........................................... [no-gen-acls, no-auth]
o- acls ...................................................... [ACLs: 4]
| o- iqn.2016-09.com.example:initiator01 .............. [Mapped LUNs: 1]
| | o- mapped_lun0 ........................... [lun0 block/lun0003 (rw)]
| o- iqn.2016-09.com.example:initiator02 .............. [Mapped LUNs: 1]
| | o- mapped_lun0 ........................... [lun0 block/lun0003 (rw)]
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 70 27780
27790
27800
27810
27820
27830
27840
27850
27860
27870
27880
27890
27900
27910
27920
27930
27940
27950
27960
27970
27980
27990
28000
28010
28020
28030
28040
28050
28060
28070
28080
28090
28100
28110
28120
28130
28140
28150
28160
28170
28180
|
| o- iqn.2016-09.com.example:initiator03 .............. [Mapped LUNs: 1]
|
| | o- mapped_lun0 ........................... [lun0 block/lun0003 (rw)]
|
| o- iqn.2016-09.com.example:initiator04 .............. [Mapped LUNs: 1]
|
|
o- mapped_lun0 ........................... [lun0 block/lun0003 (rw)]
|
o- luns ...................................................... [LUNs: 1]
|
| o- lun0 ........................ [block/lun0003 (/dev/vg1/lv-lun0003)]
|
o- portals ................................................ [Portals: 1]
|
o- 10.110.88.59:3260 ............................................ [OK]
o- loopback ..................................................... [Targets: 0]
a
○
ss -ant | grep LISTEN.\*3260
LISTEN
0
256
10.110.88.59:3260
*:*
Active 機で、設定ファイルに設定を保存し、Stand-by 機にコピーします。
a
sudo targetcli saveconfig
Last 10 configs saved in /etc/target/backup.
Configuration saved to /etc/target/saveconfig.json
a
sudo scp -p /etc/target/saveconfig.json iscsitgt01s:/etc/target/saveconfig.json
saveconfig.json
100%
11KB 11.0KB/s
00:00
○
a
Active 機で、設定ファイルを確認します。
sudo cat /etc/target/saveconfig.json
{
"fabric_modules": [],
"storage_objects": [
{
"attributes": {
"block_size": 512,
"emulate_3pc": 1,
"emulate_caw": 1,
"emulate_dpo": 0,
"emulate_fua_read": 0,
"emulate_fua_write": 1,
"emulate_model_alias": 1,
"emulate_rest_reord": 0,
"emulate_tas": 1,
"emulate_tpu": 0,
"emulate_tpws": 0,
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 71 28190
28200
28210
28220
28230
28240
28250
28260
28270
28280
28290
28300
28310
28320
28330
28340
28350
28360
28370
28380
28390
28400
28410
28420
28430
28440
28450
28460
28470
28480
28490
28500
28510
28520
28530
28540
28550
28560
28570
28580
28590
"emulate_ua_intlck_ctrl": 0,
"emulate_write_cache": 0,
"enforce_pr_isids": 1,
"force_pr_aptpl": 0,
"is_nonrot": 0,
"max_unmap_block_desc_count": 1,
"max_unmap_lba_count": 8192,
"max_write_same_len": 65535,
"optimal_sectors": 2048,
"pi_prot_format": 0,
"pi_prot_type": 0,
"queue_depth": 128,
"unmap_granularity": 2048,
"unmap_granularity_alignment": 0
},
"dev": "/dev/vg1/lv-lun0003",
"name": "lun0003",
"plugin": "block",
"readonly": false,
"write_back": false,
"wwn": "03880f22-4ca1-48e8-b6e5-cf303af79ade "
},
{
"attributes": {
"block_size": 512,
"emulate_3pc": 1,
"emulate_caw": 1,
"emulate_dpo": 0,
"emulate_fua_read": 0,
"emulate_fua_write": 1,
"emulate_model_alias": 1,
"emulate_rest_reord": 0,
"emulate_tas": 1,
"emulate_tpu": 0,
"emulate_tpws": 0,
"emulate_ua_intlck_ctrl": 0,
"emulate_write_cache": 0,
"enforce_pr_isids": 1,
"force_pr_aptpl": 0,
"is_nonrot": 0,
"max_unmap_block_desc_count": 1,
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 72 28600
28610
28620
28630
28640
28650
28660
28670
28680
28690
28700
28710
28720
28730
28740
28750
28760
28770
28780
28790
28800
28810
28820
28830
28840
28850
28860
28870
28880
28890
28900
28910
28920
28930
28940
28950
28960
28970
28980
28990
29000
"max_unmap_lba_count": 8192,
"max_write_same_len": 65535,
"optimal_sectors": 2048,
"pi_prot_format": 0,
"pi_prot_type": 0,
"queue_depth": 128,
"unmap_granularity": 2048,
"unmap_granularity_alignment": 0
},
"dev": "/dev/vg1/lv-lun0002",
"name": "lun0002",
"plugin": "block",
"readonly": false,
"write_back": false,
"wwn": "9c5e3ced-aea8-46d5-89a8-c5944a1d4cd3 "
},
{
"attributes": {
"block_size": 512,
"emulate_3pc": 1,
"emulate_caw": 1,
"emulate_dpo": 0,
"emulate_fua_read": 0,
"emulate_fua_write": 1,
"emulate_model_alias": 1,
"emulate_rest_reord": 0,
"emulate_tas": 1,
"emulate_tpu": 0,
"emulate_tpws": 0,
"emulate_ua_intlck_ctrl": 0,
"emulate_write_cache": 0,
"enforce_pr_isids": 1,
"force_pr_aptpl": 0,
"is_nonrot": 0,
"max_unmap_block_desc_count": 1,
"max_unmap_lba_count": 8192,
"max_write_same_len": 65535,
"optimal_sectors": 2048,
"pi_prot_format": 0,
"pi_prot_type": 0,
"queue_depth": 128,
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 73 29010
29020
29030
29040
29050
29060
29070
29080
29090
29100
29110
29120
29130
29140
29150
29160
29170
29180
29190
29200
29210
29220
29230
29240
29250
29260
29270
29280
29290
29300
29310
29320
29330
29340
29350
29360
29370
29380
29390
29400
29410
"unmap_granularity": 2048,
"unmap_granularity_alignment": 0
},
"dev": "/dev/vg1/lv-lun0001",
"name": "lun0001",
"plugin": "block",
"readonly": false,
"write_back": false,
"wwn": "5a7d4ce5-206e-40f2-a22f-b143637136ab "
},
{
"attributes": {
"block_size": 512,
"emulate_3pc": 1,
"emulate_caw": 1,
"emulate_dpo": 0,
"emulate_fua_read": 0,
"emulate_fua_write": 1,
"emulate_model_alias": 1,
"emulate_rest_reord": 0,
"emulate_tas": 1,
"emulate_tpu": 0,
"emulate_tpws": 0,
"emulate_ua_intlck_ctrl": 0,
"emulate_write_cache": 0,
"enforce_pr_isids": 1,
"force_pr_aptpl": 0,
"is_nonrot": 0,
"max_unmap_block_desc_count": 1,
"max_unmap_lba_count": 8192,
"max_write_same_len": 65535,
"optimal_sectors": 2048,
"pi_prot_format": 0,
"pi_prot_type": 0,
"queue_depth": 128,
"unmap_granularity": 2048,
"unmap_granularity_alignment": 0
},
"dev": "/dev/vg1/lv-lun0000",
"name": "lun0000",
"plugin": "block",
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 74 29420
29430
29440
29450
29460
29470
29480
29490
29500
29510
29520
29530
29540
29550
29560
29570
29580
29590
29600
29610
29620
29630
29640
29650
29660
29670
29680
29690
29700
29710
29720
29730
29740
29750
29760
29770
29780
29790
29800
29810
29820
"readonly": false,
"write_back": false,
"wwn": "117473ae-68c7-44cd-b665-f7ad42bf1bd0 "
}
],
"targets": [
{
"fabric": "iscsi",
"tpgs": [
{
"attributes": {
"authentication": 0,
"cache_dynamic_acls": 0,
"default_cmdsn_depth": 128,
"default_erl": 0,
"demo_mode_discovery": 1,
"demo_mode_write_protect": 1,
"fabric_prot_type": 0,
"generate_node_acls": 0,
"login_timeout": 15,
"netif_timeout": 2,
"prod_mode_write_protect": 0,
"t10_pi": 0
},
"enable": true,
"luns": [
{
"index": 0,
"storage_object": "/backstores/block/lun0003"
}
],
"node_acls": [
{
"attributes": {
"dataout_timeout": 3,
"dataout_timeout_retries": 5,
"default_erl": 0,
"nopin_response_timeout": 30,
"nopin_timeout": 15,
"random_datain_pdu_offsets": 0,
"random_datain_seq_offsets": 0,
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 75 29830
29840
29850
29860
29870
29880
29890
29900
29910
29920
29930
29940
29950
29960
29970
29980
29990
30000
30010
30020
30030
30040
30050
30060
30070
30080
30090
30100
30110
30120
30130
30140
30150
30160
30170
30180
30190
30200
30210
30220
30230
"random_r2t_offsets": 0
},
"chap_password": "password-user04 ",
"chap_userid": "iscsiuser04",
"mapped_luns": [
{
"index": 0,
"tpg_lun": 0,
"write_protect": false
}
],
"node_wwn": "iqn.2016-09.com.example:initiator04 "
},
{
"attributes": {
"dataout_timeout": 3,
"dataout_timeout_retries": 5,
"default_erl": 0,
"nopin_response_timeout": 30,
"nopin_timeout": 15,
"random_datain_pdu_offsets": 0,
"random_datain_seq_offsets": 0,
"random_r2t_offsets": 0
},
"chap_password": "password-user03 ",
"chap_userid": "iscsiuser03",
"mapped_luns": [
{
"index": 0,
"tpg_lun": 0,
"write_protect": false
}
],
"node_wwn": "iqn.2016-09.com.example:initiator03 "
},
{
"attributes": {
"dataout_timeout": 3,
"dataout_timeout_retries": 5,
"default_erl": 0,
"nopin_response_timeout": 30,
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 76 30240
30250
30260
30270
30280
30290
30300
30310
30320
30330
30340
30350
30360
30370
30380
30390
30400
30410
30420
30430
30440
30450
30460
30470
30480
30490
30500
30510
30520
30530
30540
30550
30560
30570
30580
30590
30600
30610
30620
30630
30640
"nopin_timeout": 15,
"random_datain_pdu_offsets": 0,
"random_datain_seq_offsets": 0,
"random_r2t_offsets": 0
},
"chap_password": "password-user02 ",
"chap_userid": "iscsiuser02",
"mapped_luns": [
{
"index": 0,
"tpg_lun": 0,
"write_protect": false
}
],
"node_wwn": "iqn.2016-09.com.example:initiator02 "
},
{
"attributes": {
"dataout_timeout": 3,
"dataout_timeout_retries": 5,
"default_erl": 0,
"nopin_response_timeout": 30,
"nopin_timeout": 15,
"random_datain_pdu_offsets": 0,
"random_datain_seq_offsets": 0,
"random_r2t_offsets": 0
},
"chap_password": "password-user01 ",
"chap_userid": "iscsiuser01",
"mapped_luns": [
{
"index": 0,
"tpg_lun": 0,
"write_protect": false
}
],
"node_wwn": "iqn.2016-09.com.example:initiator01 "
}
],
"parameters": {
"AuthMethod": "CHAP,None",
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 77 30650
30660
30670
30680
30690
30700
30710
30720
30730
30740
30750
30760
30770
30780
30790
30800
30810
30820
30830
30840
30850
30860
30870
30880
30890
30900
30910
30920
30930
30940
30950
30960
30970
30980
30990
31000
31010
31020
31030
31040
31050
"DataDigest": "CRC32C,None",
"DataPDUInOrder": "Yes",
"DataSequenceInOrder": "Yes",
"DefaultTime2Retain": "20",
"DefaultTime2Wait": "2",
"ErrorRecoveryLevel": "0",
"FirstBurstLength": "65536",
"HeaderDigest": "CRC32C,None",
"IFMarkInt": "2048~65535",
"IFMarker": "No",
"ImmediateData": "Yes",
"InitialR2T": "Yes",
"MaxBurstLength": "262144",
"MaxConnections": "1",
"MaxOutstandingR2T": "1",
"MaxRecvDataSegmentLength": "8192",
"MaxXmitDataSegmentLength": "262144",
"OFMarkInt": "2048~65535",
"OFMarker": "No",
"TargetAlias": "LIO Target"
},
"portals": [
{
"ip_address": "10.110.88.59",
"iser": false,
"port": 3260
}
],
"tag": 1
}
],
"wwn": "iqn.2016-09.com.example:iscsitgt01-0003 "
},
{
"fabric": "iscsi",
"tpgs": [
{
"attributes": {
"authentication": 0,
"cache_dynamic_acls": 0,
"default_cmdsn_depth": 128,
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 78 31060
31070
31080
31090
31100
31110
31120
31130
31140
31150
31160
31170
31180
31190
31200
31210
31220
31230
31240
31250
31260
31270
31280
31290
31300
31310
31320
31330
31340
31350
31360
31370
31380
31390
31400
31410
31420
31430
31440
31450
31460
"default_erl": 0,
"demo_mode_discovery": 1,
"demo_mode_write_protect": 1,
"fabric_prot_type": 0,
"generate_node_acls": 0,
"login_timeout": 15,
"netif_timeout": 2,
"prod_mode_write_protect": 0,
"t10_pi": 0
},
"enable": true,
"luns": [
{
"index": 0,
"storage_object": "/backstores/block/lun0002"
}
],
"node_acls": [
{
"attributes": {
"dataout_timeout": 3,
"dataout_timeout_retries": 5,
"default_erl": 0,
"nopin_response_timeout": 30,
"nopin_timeout": 15,
"random_datain_pdu_offsets": 0,
"random_datain_seq_offsets": 0,
"random_r2t_offsets": 0
},
"chap_password": "password-user04 ",
"chap_userid": "iscsiuser04",
"mapped_luns": [
{
"index": 0,
"tpg_lun": 0,
"write_protect": false
}
],
"node_wwn": "iqn.2016-09.com.example:initiator04 "
},
{
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 79 31470
31480
31490
31500
31510
31520
31530
31540
31550
31560
31570
31580
31590
31600
31610
31620
31630
31640
31650
31660
31670
31680
31690
31700
31710
31720
31730
31740
31750
31760
31770
31780
31790
31800
31810
31820
31830
31840
31850
31860
31870
"attributes": {
"dataout_timeout": 3,
"dataout_timeout_retries": 5,
"default_erl": 0,
"nopin_response_timeout": 30,
"nopin_timeout": 15,
"random_datain_pdu_offsets": 0,
"random_datain_seq_offsets": 0,
"random_r2t_offsets": 0
},
"chap_password": "password-user03 ",
"chap_userid": "iscsiuser03",
"mapped_luns": [
{
"index": 0,
"tpg_lun": 0,
"write_protect": false
}
],
"node_wwn": "iqn.2016-09.com.example:initiator03 "
},
{
"attributes": {
"dataout_timeout": 3,
"dataout_timeout_retries": 5,
"default_erl": 0,
"nopin_response_timeout": 30,
"nopin_timeout": 15,
"random_datain_pdu_offsets": 0,
"random_datain_seq_offsets": 0,
"random_r2t_offsets": 0
},
"chap_password": "password-user02 ",
"chap_userid": "iscsiuser02",
"mapped_luns": [
{
"index": 0,
"tpg_lun": 0,
"write_protect": false
}
],
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 80 31880
31890
31900
31910
31920
31930
31940
31950
31960
31970
31980
31990
32000
32010
32020
32030
32040
32050
32060
32070
32080
32090
32100
32110
32120
32130
32140
32150
32160
32170
32180
32190
32200
32210
32220
32230
32240
32250
32260
32270
32280
"node_wwn": "iqn.2016-09.com.example:initiator02 "
},
{
"attributes": {
"dataout_timeout": 3,
"dataout_timeout_retries": 5,
"default_erl": 0,
"nopin_response_timeout": 30,
"nopin_timeout": 15,
"random_datain_pdu_offsets": 0,
"random_datain_seq_offsets": 0,
"random_r2t_offsets": 0
},
"chap_password": "password-user01 ",
"chap_userid": "iscsiuser01",
"mapped_luns": [
{
"index": 0,
"tpg_lun": 0,
"write_protect": false
}
],
"node_wwn": "iqn.2016-09.com.example:initiator01 "
}
],
"parameters": {
"AuthMethod": "CHAP,None",
"DataDigest": "CRC32C,None",
"DataPDUInOrder": "Yes",
"DataSequenceInOrder": "Yes",
"DefaultTime2Retain": "20",
"DefaultTime2Wait": "2",
"ErrorRecoveryLevel": "0",
"FirstBurstLength": "65536",
"HeaderDigest": "CRC32C,None",
"IFMarkInt": "2048~65535",
"IFMarker": "No",
"ImmediateData": "Yes",
"InitialR2T": "Yes",
"MaxBurstLength": "262144",
"MaxConnections": "1",
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 81 32290
32300
32310
32320
32330
32340
32350
32360
32370
32380
32390
32400
32410
32420
32430
32440
32450
32460
32470
32480
32490
32500
32510
32520
32530
32540
32550
32560
32570
32580
32590
32600
32610
32620
32630
32640
32650
32660
32670
32680
32690
"MaxOutstandingR2T": "1",
"MaxRecvDataSegmentLength": "8192",
"MaxXmitDataSegmentLength": "262144",
"OFMarkInt": "2048~65535",
"OFMarker": "No",
"TargetAlias": "LIO Target"
},
"portals": [
{
"ip_address": "10.110.88.59",
"iser": false,
"port": 3260
}
],
"tag": 1
}
],
"wwn": "iqn.2016-09.com.example:iscsitgt01-0002 "
},
{
"fabric": "iscsi",
"tpgs": [
{
"attributes": {
"authentication": 0,
"cache_dynamic_acls": 0,
"default_cmdsn_depth": 128,
"default_erl": 0,
"demo_mode_discovery": 1,
"demo_mode_write_protect": 1,
"fabric_prot_type": 0,
"generate_node_acls": 0,
"login_timeout": 15,
"netif_timeout": 2,
"prod_mode_write_protect": 0,
"t10_pi": 0
},
"enable": true,
"luns": [
{
"index": 0,
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 82 32700
32710
32720
32730
32740
32750
32760
32770
32780
32790
32800
32810
32820
32830
32840
32850
32860
32870
32880
32890
32900
32910
32920
32930
32940
32950
32960
32970
32980
32990
33000
33010
33020
33030
33040
33050
33060
33070
33080
33090
33100
"storage_object": "/backstores/block/lun0001"
}
],
"node_acls": [
{
"attributes": {
"dataout_timeout": 3,
"dataout_timeout_retries": 5,
"default_erl": 0,
"nopin_response_timeout": 30,
"nopin_timeout": 15,
"random_datain_pdu_offsets": 0,
"random_datain_seq_offsets": 0,
"random_r2t_offsets": 0
},
"chap_password": "password-user04 ",
"chap_userid": "iscsiuser04",
"mapped_luns": [
{
"index": 0,
"tpg_lun": 0,
"write_protect": false
}
],
"node_wwn": "iqn.2016-09.com.example:initiator04 "
},
{
"attributes": {
"dataout_timeout": 3,
"dataout_timeout_retries": 5,
"default_erl": 0,
"nopin_response_timeout": 30,
"nopin_timeout": 15,
"random_datain_pdu_offsets": 0,
"random_datain_seq_offsets": 0,
"random_r2t_offsets": 0
},
"chap_password": "password-user03 ",
"chap_userid": "iscsiuser03",
"mapped_luns": [
{
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 83 33110
33120
33130
33140
33150
33160
33170
33180
33190
33200
33210
33220
33230
33240
33250
33260
33270
33280
33290
33300
33310
33320
33330
33340
33350
33360
33370
33380
33390
33400
33410
33420
33430
33440
33450
33460
33470
33480
33490
33500
33510
"index": 0,
"tpg_lun": 0,
"write_protect": false
}
],
"node_wwn": "iqn.2016-09.com.example:initiator03 "
},
{
"attributes": {
"dataout_timeout": 3,
"dataout_timeout_retries": 5,
"default_erl": 0,
"nopin_response_timeout": 30,
"nopin_timeout": 15,
"random_datain_pdu_offsets": 0,
"random_datain_seq_offsets": 0,
"random_r2t_offsets": 0
},
"chap_password": "password-user02 ",
"chap_userid": "iscsiuser02",
"mapped_luns": [
{
"index": 0,
"tpg_lun": 0,
"write_protect": false
}
],
"node_wwn": "iqn.2016-09.com.example:initiator02 "
},
{
"attributes": {
"dataout_timeout": 3,
"dataout_timeout_retries": 5,
"default_erl": 0,
"nopin_response_timeout": 30,
"nopin_timeout": 15,
"random_datain_pdu_offsets": 0,
"random_datain_seq_offsets": 0,
"random_r2t_offsets": 0
},
"chap_password": "password-user01 ",
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 84 33520
33530
33540
33550
33560
33570
33580
33590
33600
33610
33620
33630
33640
33650
33660
33670
33680
33690
33700
33710
33720
33730
33740
33750
33760
33770
33780
33790
33800
33810
33820
33830
33840
33850
33860
33870
33880
33890
33900
33910
33920
"chap_userid": "iscsiuser01",
"mapped_luns": [
{
"index": 0,
"tpg_lun": 0,
"write_protect": false
}
],
"node_wwn": "iqn.2016-09.com.example:initiator01 "
}
],
"parameters": {
"AuthMethod": "CHAP,None",
"DataDigest": "CRC32C,None",
"DataPDUInOrder": "Yes",
"DataSequenceInOrder": "Yes",
"DefaultTime2Retain": "20",
"DefaultTime2Wait": "2",
"ErrorRecoveryLevel": "0",
"FirstBurstLength": "65536",
"HeaderDigest": "CRC32C,None",
"IFMarkInt": "2048~65535",
"IFMarker": "No",
"ImmediateData": "Yes",
"InitialR2T": "Yes",
"MaxBurstLength": "262144",
"MaxConnections": "1",
"MaxOutstandingR2T": "1",
"MaxRecvDataSegmentLength": "8192",
"MaxXmitDataSegmentLength": "262144",
"OFMarkInt": "2048~65535",
"OFMarker": "No",
"TargetAlias": "LIO Target"
},
"portals": [
{
"ip_address": "10.110.88.59",
"iser": false,
"port": 3260
}
],
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 85 33930
33940
33950
33960
33970
33980
33990
34000
34010
34020
34030
34040
34050
34060
34070
34080
34090
34100
34110
34120
34130
34140
34150
34160
34170
34180
34190
34200
34210
34220
34230
34240
34250
34260
34270
34280
34290
34300
34310
34320
34330
"tag": 1
}
],
"wwn": "iqn.2016-09.com.example:iscsitgt01-0001 "
},
{
"fabric": "iscsi",
"tpgs": [
{
"attributes": {
"authentication": 0,
"cache_dynamic_acls": 0,
"default_cmdsn_depth": 128,
"default_erl": 0,
"demo_mode_discovery": 1,
"demo_mode_write_protect": 1,
"fabric_prot_type": 0,
"generate_node_acls": 0,
"login_timeout": 15,
"netif_timeout": 2,
"prod_mode_write_protect": 0,
"t10_pi": 0
},
"enable": true,
"luns": [
{
"index": 0,
"storage_object": "/backstores/block/lun0000"
}
],
"node_acls": [
{
"attributes": {
"dataout_timeout": 3,
"dataout_timeout_retries": 5,
"default_erl": 0,
"nopin_response_timeout": 30,
"nopin_timeout": 15,
"random_datain_pdu_offsets": 0,
"random_datain_seq_offsets": 0,
"random_r2t_offsets": 0
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 86 34340
34350
34360
34370
34380
34390
34400
34410
34420
34430
34440
34450
34460
34470
34480
34490
34500
34510
34520
34530
34540
34550
34560
34570
34580
34590
34600
34610
34620
34630
34640
34650
34660
34670
34680
34690
34700
34710
34720
34730
34740
},
"chap_password": "password-user04 ",
"chap_userid": "iscsiuser04",
"mapped_luns": [
{
"index": 0,
"tpg_lun": 0,
"write_protect": false
}
],
"node_wwn": "iqn.2016-09.com.example:initiator04 "
},
{
"attributes": {
"dataout_timeout": 3,
"dataout_timeout_retries": 5,
"default_erl": 0,
"nopin_response_timeout": 30,
"nopin_timeout": 15,
"random_datain_pdu_offsets": 0,
"random_datain_seq_offsets": 0,
"random_r2t_offsets": 0
},
"chap_password": "password-user03 ",
"chap_userid": "iscsiuser03",
"mapped_luns": [
{
"index": 0,
"tpg_lun": 0,
"write_protect": false
}
],
"node_wwn": "iqn.2016-09.com.example:initiator03 "
},
{
"attributes": {
"dataout_timeout": 3,
"dataout_timeout_retries": 5,
"default_erl": 0,
"nopin_response_timeout": 30,
"nopin_timeout": 15,
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 87 34750
34760
34770
34780
34790
34800
34810
34820
34830
34840
34850
34860
34870
34880
34890
34900
34910
34920
34930
34940
34950
34960
34970
34980
34990
35000
35010
35020
35030
35040
35050
35060
35070
35080
35090
35100
35110
35120
35130
35140
35150
"random_datain_pdu_offsets": 0,
"random_datain_seq_offsets": 0,
"random_r2t_offsets": 0
},
"chap_password": "password-user02 ",
"chap_userid": "iscsiuser02",
"mapped_luns": [
{
"index": 0,
"tpg_lun": 0,
"write_protect": false
}
],
"node_wwn": "iqn.2016-09.com.example:initiator02 "
},
{
"attributes": {
"dataout_timeout": 3,
"dataout_timeout_retries": 5,
"default_erl": 0,
"nopin_response_timeout": 30,
"nopin_timeout": 15,
"random_datain_pdu_offsets": 0,
"random_datain_seq_offsets": 0,
"random_r2t_offsets": 0
},
"chap_password": "password-user01 ",
"chap_userid": "iscsiuser01",
"mapped_luns": [
{
"index": 0,
"tpg_lun": 0,
"write_protect": false
}
],
"node_wwn": "iqn.2016-09.com.example:initiator01 "
}
],
"parameters": {
"AuthMethod": "CHAP,None",
"DataDigest": "CRC32C,None",
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 88 35160
35170
35180
35190
35200
35210
35220
35230
35240
35250
35260
35270
35280
35290
35300
35310
35320
35330
35340
35350
35360
35370
35380
35390
35400
35410
35420
35430
35440
35450
35460
35470
35480
35490
35500
35510
35520
35530
35540
35550
35560
"DataPDUInOrder": "Yes",
"DataSequenceInOrder": "Yes",
"DefaultTime2Retain": "20",
"DefaultTime2Wait": "2",
"ErrorRecoveryLevel": "0",
"FirstBurstLength": "65536",
"HeaderDigest": "CRC32C,None",
"IFMarkInt": "2048~65535",
"IFMarker": "No",
"ImmediateData": "Yes",
"InitialR2T": "Yes",
"MaxBurstLength": "262144",
"MaxConnections": "1",
"MaxOutstandingR2T": "1",
"MaxRecvDataSegmentLength": "8192",
"MaxXmitDataSegmentLength": "262144",
"OFMarkInt": "2048~65535",
"OFMarker": "No",
"TargetAlias": "LIO Target"
},
"portals": [
{
"ip_address": "10.110.88.59",
"iser": false,
"port": 3260
}
],
"tag": 1
}
],
"wwn": "iqn.2016-09.com.example:iscsitgt01-0000 "
}
]
}
{
"fabric_modules": [],
"storage_objects": [
{
"attributes": {
"block_size": 512,
"emulate_3pc": 1,
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 89 35570
35580
35590
35600
35610
35620
35630
35640
35650
35660
35670
35680
35690
35700
35710
35720
35730
35740
35750
35760
35770
35780
35790
35800
35810
35820
35830
35840
35850
35860
35870
35880
35890
35900
35910
35920
35930
35940
35950
35960
35970
"emulate_caw": 1,
"emulate_dpo": 0,
"emulate_fua_read": 0,
"emulate_fua_write": 1,
"emulate_model_alias": 1,
"emulate_rest_reord": 0,
"emulate_tas": 1,
"emulate_tpu": 0,
"emulate_tpws": 0,
"emulate_ua_intlck_ctrl": 0,
"emulate_write_cache": 0,
"enforce_pr_isids": 1,
"force_pr_aptpl": 0,
"is_nonrot": 0,
"max_unmap_block_desc_count": 1,
"max_unmap_lba_count": 8192,
"max_write_same_len": 65535,
"optimal_sectors": 2048,
"pi_prot_format": 0,
"pi_prot_type": 0,
"queue_depth": 128,
"unmap_granularity": 2048,
"unmap_granularity_alignment": 0
},
"dev": "/dev/vg1/lv-lun0003",
"name": "lun0003",
"plugin": "block",
"readonly": false,
"write_back": false,
"wwn": "1ade2d3a-db39-4880-8e80-59d51ca87c60 "
},
{
"attributes": {
"block_size": 512,
"emulate_3pc": 1,
"emulate_caw": 1,
"emulate_dpo": 0,
"emulate_fua_read": 0,
"emulate_fua_write": 1,
"emulate_model_alias": 1,
"emulate_rest_reord": 0,
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 90 35980
35990
36000
36010
36020
36030
36040
36050
36060
36070
36080
36090
36100
36110
36120
36130
36140
36150
36160
36170
36180
36190
36200
36210
36220
36230
36240
36250
36260
36270
36280
36290
36300
36310
36320
36330
36340
36350
36360
36370
36380
"emulate_tas": 1,
"emulate_tpu": 0,
"emulate_tpws": 0,
"emulate_ua_intlck_ctrl": 0,
"emulate_write_cache": 0,
"enforce_pr_isids": 1,
"force_pr_aptpl": 0,
"is_nonrot": 0,
"max_unmap_block_desc_count": 1,
"max_unmap_lba_count": 8192,
"max_write_same_len": 65535,
"optimal_sectors": 2048,
"pi_prot_format": 0,
"pi_prot_type": 0,
"queue_depth": 128,
"unmap_granularity": 2048,
"unmap_granularity_alignment": 0
},
"dev": "/dev/vg1/lv-lun0002",
"name": "lun0002",
"plugin": "block",
"readonly": false,
"write_back": false,
"wwn": "50907dc1-06ff-4e27-877a-b68ffe949fd7 "
},
{
"attributes": {
"block_size": 512,
"emulate_3pc": 1,
"emulate_caw": 1,
"emulate_dpo": 0,
"emulate_fua_read": 0,
"emulate_fua_write": 1,
"emulate_model_alias": 1,
"emulate_rest_reord": 0,
"emulate_tas": 1,
"emulate_tpu": 0,
"emulate_tpws": 0,
"emulate_ua_intlck_ctrl": 0,
"emulate_write_cache": 0,
"enforce_pr_isids": 1,
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 91 36390
36400
36410
36420
36430
36440
36450
36460
36470
36480
36490
36500
36510
36520
36530
36540
36550
36560
36570
36580
36590
36600
36610
36620
36630
36640
36650
36660
36670
36680
36690
36700
36710
36720
36730
36740
36750
36760
36770
36780
36790
"force_pr_aptpl": 0,
"is_nonrot": 0,
"max_unmap_block_desc_count": 1,
"max_unmap_lba_count": 8192,
"max_write_same_len": 65535,
"optimal_sectors": 2048,
"pi_prot_format": 0,
"pi_prot_type": 0,
"queue_depth": 128,
"unmap_granularity": 2048,
"unmap_granularity_alignment": 0
},
"dev": "/dev/vg1/lv-lun0001",
"name": "lun0001",
"plugin": "block",
"readonly": false,
"write_back": false,
"wwn": "9a5889d7-4176-4e58-874e-46a15f47dc58 "
},
{
"attributes": {
"block_size": 512,
"emulate_3pc": 1,
"emulate_caw": 1,
"emulate_dpo": 0,
"emulate_fua_read": 0,
"emulate_fua_write": 1,
"emulate_model_alias": 1,
"emulate_rest_reord": 0,
"emulate_tas": 1,
"emulate_tpu": 0,
"emulate_tpws": 0,
"emulate_ua_intlck_ctrl": 0,
"emulate_write_cache": 0,
"enforce_pr_isids": 1,
"force_pr_aptpl": 0,
"is_nonrot": 0,
"max_unmap_block_desc_count": 1,
"max_unmap_lba_count": 8192,
"max_write_same_len": 65535,
"optimal_sectors": 2048,
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 92 36800
36810
36820
36830
36840
36850
36860
36870
36880
36890
36900
36910
36920
36930
36940
36950
36960
36970
36980
36990
37000
37010
37020
37030
37040
37050
37060
37070
37080
37090
37100
37110
37120
37130
37140
37150
37160
37170
37180
37190
37200
"pi_prot_format": 0,
"pi_prot_type": 0,
"queue_depth": 128,
"unmap_granularity": 2048,
"unmap_granularity_alignment": 0
},
"dev": "/dev/vg1/lv-lun0000",
"name": "lun0000",
"plugin": "block",
"readonly": false,
"write_back": false,
"wwn": "b77302c8-9db8-4889-8353-b0493cababe5 "
}
],
"targets": [
{
"fabric": "iscsi",
"tpgs": [
{
"attributes": {
"authentication": 0,
"cache_dynamic_acls": 0,
"default_cmdsn_depth": 128,
"default_erl": 0,
"demo_mode_discovery": 1,
"demo_mode_write_protect": 1,
"fabric_prot_type": 0,
"generate_node_acls": 0,
"login_timeout": 15,
"netif_timeout": 2,
"prod_mode_write_protect": 0,
"t10_pi": 0
},
"enable": true,
"luns": [
{
"index": 0,
"storage_object": "/backstores/block/lun0003"
}
],
"node_acls": [
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 93 37210
37220
37230
37240
37250
37260
37270
37280
37290
37300
37310
37320
37330
37340
37350
37360
37370
37380
37390
37400
37410
37420
37430
37440
37450
37460
37470
37480
37490
37500
37510
37520
37530
37540
37550
37560
37570
37580
37590
37600
37610
{
"attributes": {
"dataout_timeout": 3,
"dataout_timeout_retries": 5,
"default_erl": 0,
"nopin_response_timeout": 30,
"nopin_timeout": 15,
"random_datain_pdu_offsets": 0,
"random_datain_seq_offsets": 0,
"random_r2t_offsets": 0
},
"chap_password": "password-user04 ",
"chap_userid": "iscsiuser04",
"mapped_luns": [
{
"index": 0,
"tpg_lun": 0,
"write_protect": false
}
],
"node_wwn": "iqn.2016-09.com.example:initiator04 "
},
{
"attributes": {
"dataout_timeout": 3,
"dataout_timeout_retries": 5,
"default_erl": 0,
"nopin_response_timeout": 30,
"nopin_timeout": 15,
"random_datain_pdu_offsets": 0,
"random_datain_seq_offsets": 0,
"random_r2t_offsets": 0
},
"mapped_luns": [
{
"index": 0,
"tpg_lun": 0,
"write_protect": false
}
],
"node_wwn": "iqn.2016-09.com.example:initiator03 "
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 94 37620
37630
37640
37650
37660
37670
37680
37690
37700
37710
37720
37730
37740
37750
37760
37770
37780
37790
37800
37810
37820
37830
37840
37850
37860
37870
37880
37890
37900
37910
37920
37930
37940
37950
37960
37970
37980
37990
38000
38010
38020
},
{
"attributes": {
"dataout_timeout": 3,
"dataout_timeout_retries": 5,
"default_erl": 0,
"nopin_response_timeout": 30,
"nopin_timeout": 15,
"random_datain_pdu_offsets": 0,
"random_datain_seq_offsets": 0,
"random_r2t_offsets": 0
},
"chap_password": "password-user02 ",
"chap_userid": "iscsiuser02",
"mapped_luns": [
{
"index": 0,
"tpg_lun": 0,
"write_protect": false
}
],
"node_wwn": "iqn.2016-09.com.example:initiator02 "
},
{
"attributes": {
"dataout_timeout": 3,
"dataout_timeout_retries": 5,
"default_erl": 0,
"nopin_response_timeout": 30,
"nopin_timeout": 15,
"random_datain_pdu_offsets": 0,
"random_datain_seq_offsets": 0,
"random_r2t_offsets": 0
},
"chap_password": "password-user01 ",
"chap_userid": "iscsiuser01",
"mapped_luns": [
{
"index": 0,
"tpg_lun": 0,
"write_protect": false
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 95 38030
38040
38050
38060
38070
38080
38090
38100
38110
38120
38130
38140
38150
38160
38170
38180
38190
38200
38210
38220
38230
38240
38250
38260
38270
38280
38290
38300
38310
38320
38330
38340
38350
38360
38370
38380
38390
38400
38410
38420
38430
}
],
"node_wwn": "iqn.2016-09.com.example:initiator01 "
}
],
"parameters": {
"AuthMethod": "CHAP,None",
"DataDigest": "CRC32C,None",
"DataPDUInOrder": "Yes",
"DataSequenceInOrder": "Yes",
"DefaultTime2Retain": "20",
"DefaultTime2Wait": "2",
"ErrorRecoveryLevel": "0",
"FirstBurstLength": "65536",
"HeaderDigest": "CRC32C,None",
"IFMarkInt": "2048~65535",
"IFMarker": "No",
"ImmediateData": "Yes",
"InitialR2T": "Yes",
"MaxBurstLength": "262144",
"MaxConnections": "1",
"MaxOutstandingR2T": "1",
"MaxRecvDataSegmentLength": "8192",
"MaxXmitDataSegmentLength": "262144",
"OFMarkInt": "2048~65535",
"OFMarker": "No",
"TargetAlias": "LIO Target"
},
"portals": [
{
"ip_address": "10.110.88.59",
"iser": false,
"port": 3260
}
],
"tag": 1
}
],
"wwn": "iqn.2016-09.com.example:iscsitgt01-0003 "
},
{
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 96 38440
38450
38460
38470
38480
38490
38500
38510
38520
38530
38540
38550
38560
38570
38580
38590
38600
38610
38620
38630
38640
38650
38660
38670
38680
38690
38700
38710
38720
38730
38740
38750
38760
38770
38780
38790
38800
38810
38820
38830
38840
"fabric": "iscsi",
"tpgs": [
{
"attributes": {
"authentication": 0,
"cache_dynamic_acls": 0,
"default_cmdsn_depth": 128,
"default_erl": 0,
"demo_mode_discovery": 1,
"demo_mode_write_protect": 1,
"fabric_prot_type": 0,
"generate_node_acls": 0,
"login_timeout": 15,
"netif_timeout": 2,
"prod_mode_write_protect": 0,
"t10_pi": 0
},
"enable": true,
"luns": [
{
"index": 0,
"storage_object": "/backstores/block/lun0002"
}
],
"node_acls": [
{
"attributes": {
"dataout_timeout": 3,
"dataout_timeout_retries": 5,
"default_erl": 0,
"nopin_response_timeout": 30,
"nopin_timeout": 15,
"random_datain_pdu_offsets": 0,
"random_datain_seq_offsets": 0,
"random_r2t_offsets": 0
},
"chap_password": "password-user04 ",
"chap_userid": "iscsiuser04",
"mapped_luns": [
{
"index": 0,
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 97 38850
38860
38870
38880
38890
38900
38910
38920
38930
38940
38950
38960
38970
38980
38990
39000
39010
39020
39030
39040
39050
39060
39070
39080
39090
39100
39110
39120
39130
39140
39150
39160
39170
39180
39190
39200
39210
39220
39230
39240
39250
"tpg_lun": 0,
"write_protect": false
}
],
"node_wwn": "iqn.2016-09.com.example:initiator04 "
},
{
"attributes": {
"dataout_timeout": 3,
"dataout_timeout_retries": 5,
"default_erl": 0,
"nopin_response_timeout": 30,
"nopin_timeout": 15,
"random_datain_pdu_offsets": 0,
"random_datain_seq_offsets": 0,
"random_r2t_offsets": 0
},
"mapped_luns": [
{
"index": 0,
"tpg_lun": 0,
"write_protect": false
}
],
"node_wwn": "iqn.2016-09.com.example:initiator03 "
},
{
"attributes": {
"dataout_timeout": 3,
"dataout_timeout_retries": 5,
"default_erl": 0,
"nopin_response_timeout": 30,
"nopin_timeout": 15,
"random_datain_pdu_offsets": 0,
"random_datain_seq_offsets": 0,
"random_r2t_offsets": 0
},
"chap_password": "password-user02 ",
"chap_userid": "iscsiuser02",
"mapped_luns": [
{
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 98 39260
39270
39280
39290
39300
39310
39320
39330
39340
39350
39360
39370
39380
39390
39400
39410
39420
39430
39440
39450
39460
39470
39480
39490
39500
39510
39520
39530
39540
39550
39560
39570
39580
39590
39600
39610
39620
39630
39640
39650
39660
"index": 0,
"tpg_lun": 0,
"write_protect": false
}
],
"node_wwn": "iqn.2016-09.com.example:initiator02 "
},
{
"attributes": {
"dataout_timeout": 3,
"dataout_timeout_retries": 5,
"default_erl": 0,
"nopin_response_timeout": 30,
"nopin_timeout": 15,
"random_datain_pdu_offsets": 0,
"random_datain_seq_offsets": 0,
"random_r2t_offsets": 0
},
"chap_password": "password-user01 ",
"chap_userid": "iscsiuser01",
"mapped_luns": [
{
"index": 0,
"tpg_lun": 0,
"write_protect": false
}
],
"node_wwn": "iqn.2016-09.com.example:initiator01 "
}
],
"parameters": {
"AuthMethod": "CHAP,None",
"DataDigest": "CRC32C,None",
"DataPDUInOrder": "Yes",
"DataSequenceInOrder": "Yes",
"DefaultTime2Retain": "20",
"DefaultTime2Wait": "2",
"ErrorRecoveryLevel": "0",
"FirstBurstLength": "65536",
"HeaderDigest": "CRC32C,None",
"IFMarkInt": "2048~65535",
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 99 39670
39680
39690
39700
39710
39720
39730
39740
39750
39760
39770
39780
39790
39800
39810
39820
39830
39840
39850
39860
39870
39880
39890
39900
39910
39920
39930
39940
39950
39960
39970
39980
39990
40000
40010
40020
40030
40040
40050
40060
40070
"IFMarker": "No",
"ImmediateData": "Yes",
"InitialR2T": "Yes",
"MaxBurstLength": "262144",
"MaxConnections": "1",
"MaxOutstandingR2T": "1",
"MaxRecvDataSegmentLength": "8192",
"MaxXmitDataSegmentLength": "262144",
"OFMarkInt": "2048~65535",
"OFMarker": "No",
"TargetAlias": "LIO Target"
},
"portals": [
{
"ip_address": "10.110.88.59",
"iser": false,
"port": 3260
}
],
"tag": 1
}
],
"wwn": "iqn.2016-09.com.example:iscsitgt01-0002 "
},
{
"fabric": "iscsi",
"tpgs": [
{
"attributes": {
"authentication": 0,
"cache_dynamic_acls": 0,
"default_cmdsn_depth": 128,
"default_erl": 0,
"demo_mode_discovery": 1,
"demo_mode_write_protect": 1,
"fabric_prot_type": 0,
"generate_node_acls": 0,
"login_timeout": 15,
"netif_timeout": 2,
"prod_mode_write_protect": 0,
"t10_pi": 0
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 100 40080
40090
40100
40110
40120
40130
40140
40150
40160
40170
40180
40190
40200
40210
40220
40230
40240
40250
40260
40270
40280
40290
40300
40310
40320
40330
40340
40350
40360
40370
40380
40390
40400
40410
40420
40430
40440
40450
40460
40470
40480
},
"enable": true,
"luns": [
{
"index": 0,
"storage_object": "/backstores/block/lun0001"
}
],
"node_acls": [
{
"attributes": {
"dataout_timeout": 3,
"dataout_timeout_retries": 5,
"default_erl": 0,
"nopin_response_timeout": 30,
"nopin_timeout": 15,
"random_datain_pdu_offsets": 0,
"random_datain_seq_offsets": 0,
"random_r2t_offsets": 0
},
"chap_password": "password-user04 ",
"chap_userid": "iscsiuser04",
"mapped_luns": [
{
"index": 0,
"tpg_lun": 0,
"write_protect": false
}
],
"node_wwn": "iqn.2016-09.com.example:initiator04 "
},
{
"attributes": {
"dataout_timeout": 3,
"dataout_timeout_retries": 5,
"default_erl": 0,
"nopin_response_timeout": 30,
"nopin_timeout": 15,
"random_datain_pdu_offsets": 0,
"random_datain_seq_offsets": 0,
"random_r2t_offsets": 0
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 101 40490
40500
40510
40520
40530
40540
40550
40560
40570
40580
40590
40600
40610
40620
40630
40640
40650
40660
40670
40680
40690
40700
40710
40720
40730
40740
40750
40760
40770
40780
40790
40800
40810
40820
40830
40840
40850
40860
40870
40880
40890
},
"chap_password": "password-user03 ",
"chap_userid": "iscsiuser03",
"mapped_luns": [
{
"index": 0,
"tpg_lun": 0,
"write_protect": false
}
],
"node_wwn": "iqn.2016-09.com.example:initiator03 "
},
{
"attributes": {
"dataout_timeout": 3,
"dataout_timeout_retries": 5,
"default_erl": 0,
"nopin_response_timeout": 30,
"nopin_timeout": 15,
"random_datain_pdu_offsets": 0,
"random_datain_seq_offsets": 0,
"random_r2t_offsets": 0
},
"chap_password": "password-user02 ",
"chap_userid": "iscsiuser02",
"mapped_luns": [
{
"index": 0,
"tpg_lun": 0,
"write_protect": false
}
],
"node_wwn": "iqn.2016-09.com.example:initiator02 "
},
{
"attributes": {
"dataout_timeout": 3,
"dataout_timeout_retries": 5,
"default_erl": 0,
"nopin_response_timeout": 30,
"nopin_timeout": 15,
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 102 40900
40910
40920
40930
40940
40950
40960
40970
40980
40990
41000
41010
41020
41030
41040
41050
41060
41070
41080
41090
41100
41110
41120
41130
41140
41150
41160
41170
41180
41190
41200
41210
41220
41230
41240
41250
41260
41270
41280
41290
41300
"random_datain_pdu_offsets": 0,
"random_datain_seq_offsets": 0,
"random_r2t_offsets": 0
},
"chap_password": "password-user01 ",
"chap_userid": "iscsiuser01",
"mapped_luns": [
{
"index": 0,
"tpg_lun": 0,
"write_protect": false
}
],
"node_wwn": "iqn.2016-09.com.example:initiator01 "
}
],
"parameters": {
"AuthMethod": "CHAP,None",
"DataDigest": "CRC32C,None",
"DataPDUInOrder": "Yes",
"DataSequenceInOrder": "Yes",
"DefaultTime2Retain": "20",
"DefaultTime2Wait": "2",
"ErrorRecoveryLevel": "0",
"FirstBurstLength": "65536",
"HeaderDigest": "CRC32C,None",
"IFMarkInt": "2048~65535",
"IFMarker": "No",
"ImmediateData": "Yes",
"InitialR2T": "Yes",
"MaxBurstLength": "262144",
"MaxConnections": "1",
"MaxOutstandingR2T": "1",
"MaxRecvDataSegmentLength": "8192",
"MaxXmitDataSegmentLength": "262144",
"OFMarkInt": "2048~65535",
"OFMarker": "No",
"TargetAlias": "LIO Target"
},
"portals": [
{
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 103 41310
41320
41330
41340
41350
41360
41370
41380
41390
41400
41410
41420
41430
41440
41450
41460
41470
41480
41490
41500
41510
41520
41530
41540
41550
41560
41570
41580
41590
41600
41610
41620
41630
41640
41650
41660
41670
41680
41690
41700
41710
"ip_address": "10.110.88.59",
"iser": false,
"port": 3260
}
],
"tag": 1
}
],
"wwn": "iqn.2016-09.com.example:iscsitgt01-0001 "
},
{
"fabric": "iscsi",
"tpgs": [
{
"attributes": {
"authentication": 0,
"cache_dynamic_acls": 0,
"default_cmdsn_depth": 128,
"default_erl": 0,
"demo_mode_discovery": 1,
"demo_mode_write_protect": 1,
"fabric_prot_type": 0,
"generate_node_acls": 0,
"login_timeout": 15,
"netif_timeout": 2,
"prod_mode_write_protect": 0,
"t10_pi": 0
},
"enable": true,
"luns": [
{
"index": 0,
"storage_object": "/backstores/block/lun0000"
}
],
"node_acls": [
{
"attributes": {
"dataout_timeout": 3,
"dataout_timeout_retries": 5,
"default_erl": 0,
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 104 41720
41730
41740
41750
41760
41770
41780
41790
41800
41810
41820
41830
41840
41850
41860
41870
41880
41890
41900
41910
41920
41930
41940
41950
41960
41970
41980
41990
42000
42010
42020
42030
42040
42050
42060
42070
42080
42090
42100
42110
42120
"nopin_response_timeout": 30,
"nopin_timeout": 15,
"random_datain_pdu_offsets": 0,
"random_datain_seq_offsets": 0,
"random_r2t_offsets": 0
},
"chap_password": "password-user04 ",
"chap_userid": "iscsiuser04",
"mapped_luns": [
{
"index": 0,
"tpg_lun": 0,
"write_protect": false
}
],
"node_wwn": "iqn.2016-09.com.example:initiator04 "
},
{
"attributes": {
"dataout_timeout": 3,
"dataout_timeout_retries": 5,
"default_erl": 0,
"nopin_response_timeout": 30,
"nopin_timeout": 15,
"random_datain_pdu_offsets": 0,
"random_datain_seq_offsets": 0,
"random_r2t_offsets": 0
},
"chap_password": "password-user03 ",
"chap_userid": "iscsiuser03",
"mapped_luns": [
{
"index": 0,
"tpg_lun": 0,
"write_protect": false
}
],
"node_wwn": "iqn.2016-09.com.example:initiator03 "
},
{
"attributes": {
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 105 42130
42140
42150
42160
42170
42180
42190
42200
42210
42220
42230
42240
42250
42260
42270
42280
42290
42300
42310
42320
42330
42340
42350
42360
42370
42380
42390
42400
42410
42420
42430
42440
42450
42460
42470
42480
42490
42500
42510
42520
42530
"dataout_timeout": 3,
"dataout_timeout_retries": 5,
"default_erl": 0,
"nopin_response_timeout": 30,
"nopin_timeout": 15,
"random_datain_pdu_offsets": 0,
"random_datain_seq_offsets": 0,
"random_r2t_offsets": 0
},
"chap_password": "password-user02 ",
"chap_userid": "iscsiuser02",
"mapped_luns": [
{
"index": 0,
"tpg_lun": 0,
"write_protect": false
}
],
"node_wwn": "iqn.2016-09.com.example:initiator02 "
},
{
"attributes": {
"dataout_timeout": 3,
"dataout_timeout_retries": 5,
"default_erl": 0,
"nopin_response_timeout": 30,
"nopin_timeout": 15,
"random_datain_pdu_offsets": 0,
"random_datain_seq_offsets": 0,
"random_r2t_offsets": 0
},
"chap_password": "password-user01 ",
"chap_userid": "iscsiuser01",
"mapped_luns": [
{
"index": 0,
"tpg_lun": 0,
"write_protect": false
}
],
"node_wwn": "iqn.2016-09.com.example:initiator01 "
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 106 42540
42550
42560
42570
42580
42590
42600
42610
42620
42630
42640
42650
42660
42670
42680
42690
42700
42710
42720
42730
42740
42750
42760
42770
42780
42790
42800
42810
42820
42830
42840
42850
42860
42870
42880
42890
42900
42910
42920
42930
42940
}
],
"parameters": {
"AuthMethod": "CHAP,None",
"DataDigest": "CRC32C,None",
"DataPDUInOrder": "Yes",
"DataSequenceInOrder": "Yes",
"DefaultTime2Retain": "20",
"DefaultTime2Wait": "2",
"ErrorRecoveryLevel": "0",
"FirstBurstLength": "65536",
"HeaderDigest": "CRC32C,None",
"IFMarkInt": "2048~65535",
"IFMarker": "No",
"ImmediateData": "Yes",
"InitialR2T": "Yes",
"MaxBurstLength": "262144",
"MaxConnections": "1",
"MaxOutstandingR2T": "1",
"MaxRecvDataSegmentLength": "8192",
"MaxXmitDataSegmentLength": "262144",
"OFMarkInt": "2048~65535",
"OFMarker": "No",
"TargetAlias": "LIO Target"
},
"portals": [
{
"ip_address": "10.110.88.59",
"iser": false,
"port": 3260
}
],
"tag": 1
}
],
"wwn": "iqn.2016-09.com.example:iscsitgt01-0000 "
}
]
}
○
Active 機で、LIO の設定をクリアします。
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 107 42950
42960
42970
42980
42990
43000
43010
43020
43030
43040
43050
43060
43070
43080
43090
43100
43110
43120
43130
43140
43150
43160
43170
43180
43190
43200
43210
43220
43230
43240
43250
43260
43270
43280
43290
43300
43310
43320
43330
43340
43350
a
sudo targetctl clear
a
sudo targetcli ls /
o- / ..................................................................... [...]
o- backstores .......................................................... [...]
| o- block .............................................. [Storage Objects: 0]
| o- fileio ............................................. [Storage Objects: 0]
| o- pscsi .............................................. [Storage Objects: 0]
| o- ramdisk ............................................ [Storage Objects: 0]
o- iscsi ........................................................ [Targets: 0]
o- loopback ..................................................... [Targets: 0]
○
Active 機で、DRBD 上の LVM ボリュームグループを非活性化します。
a
sudo vgchange -a n vg1
0 logical volume(s) in volume group " vg1" now active
a
sudo lvs
LV
lv-drbd0
lv-lun0
lv-lun1
lv-lun2
lv-lun3
○
a
○
VG
vg0
vg1
vg1
vg1
vg1
Attr
LSize
Pool Origin Data%
-wi-ao---- 359.98g
-wi------- 323.97g
-wi------7.20g
-wi------7.20g
-wi------7.20g
Meta%
Active 機で、DRBD リソースを secondary 化（デモート）します。
sudo drbdadm secondary all
Stand-by 機で、DRBD の状態を確認し、「Ctrl + C」を押下してワッチを停止します。
Every 2.0s: cat /proc/drbd
s
○
Move Log Cpy%Sync Convert
Fri Nov 25 22:23:08 2016
version: 8.4.5 (api:1/proto:86-101)
srcversion: 1AEFF755B8BD61B81A0AF27
0: cs:Connected ro:Secondary/Secondary ds:UpToDate/UpToDate C r----ns:0 nr:228 dw:228 dr:377459420 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
Active 機と Stand-by 機で、drbd.service を停止します。
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 108 43360 a,s
43370
43380 a,s
43390
43400
43410 ○
43420
43430
a
43440
a
43450
43460
43470
43480
43490
43500
43510
43520
a
43530
43540
43550
43560
43570
43580
43590
43600
43610
43620
43630
43640
43650
43660
a
43670
a
43680
43690
43700
43710
43720
43730
43740
43750
43760
sudo systemctl stop drbd.service
cat /proc/drbd
cat: /proc/drbd: No such file or directory
Active 機で、Corosync の認証を設定し、起動します。
sudo pcs cluster auth iscsitgt01a.example.com iscsitgt01s.example.com 10.110.88.57 10.110.88.58 \
192.168.1.2 192.168.1.3 -u hacluster -p 'password' --force
iscsitgt01s.example.com : Authorized
iscsitgt01a.example.com : Authorized
10.110.88.58: Authorized
192.168.1.2: Authorized
192.168.1.3: Authorized
10.110.88.57: Authorized
sudo cat /var/lib/pcsd/tokens
{
"format_version": 2,
"data_version": 4,
"tokens": {
"10.110.88.57": "77189e9e-3be0-40ce-b81e-3e5e6525e885",
"10.110.88.58": "9e3f4ae9-b15e-49c9-b6ee-eb8c1b91783a",
"192.168.1.2": "53da862f-ad22-445b-8887-add50d385736",
"192.168.1.3": "4f78d9c6-34a4-4486-8ba1-e69f0d4e1257",
"iscsitgt01a.example.com ": "002cd1c0-2ab2-4a4c-a1a7-4bf14b61b822",
"iscsitgt01s.example.com ": "1a9981a9-04e6-461b-b904-c5df8b4c9815"
}
}
sudo pcs cluster setup --name iscsitgt01 10.110.88.57,192.168.1.2 10.110.88.58,192.168.1.3 \
--transport=udp --rrpmode=passive -u hacluster -p 'password' --force
Shutting down pacemaker/corosync services...
Redirecting to /bin/systemctl stop pacemaker.service
Redirecting to /bin/systemctl stop corosync.service
Killing any remaining services...
Removing all cluster configuration files...
10.110.88.57: Succeeded
10.110.88.58: Succeeded
Synchronizing pcsd certificates on nodes 10.110.88.57, 10.110.88.58...
10.110.88.57: Success
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 109 43770
43780
43790
43800
43810
43820
43830
43840
43850
43860
43870
43880
43890
43900
43910
43920
43930
43940
43950
43960
43970
43980
43990
44000
44010
44020
44030
44040
44050
44060
44070
44080
44090
44100
44110
44120
44130
44140
44150
44160
44170
10.110.88.58: Success
Restaring pcsd on the nodes in order to reload the certificates...
10.110.88.57: Success
10.110.88.58: Success
a
cat /etc/corosync/corosync.conf
totem {
version: 2
secauth: off
cluster_name: iscsitgt01
transport: udp
rrp_mode: passive
}
nodelist {
node {
ring0_addr: 10.110.88.57
ring1_addr: 192.168.1.2
nodeid: 1
}
node {
ring0_addr: 10.110.88.58
ring1_addr: 192.168.1.3
nodeid: 2
}
}
quorum {
provider: corosync_votequorum
two_node: 1
}
logging {
to_logfile: yes
logfile: /var/log/cluster/corosync.log
to_syslog: yes
}
a
sudo pcs cluster start --all
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 110 44180
44190
44200
44210
44220
44230
44240
44250
44260
44270
44280
44290
44300
44310
44320
44330
44340
44350
44360
44370
44380
44390
44400
44410
44420
44430
44440
44450
44460
44470
44480
44490
44500
44510
44520
44530
44540
44550
44560
44570
44580
10.110.88.57: Starting Cluster...
10.110.88.58: Starting Cluster...
a
sudo pcs status corosync
Membership information
---------------------Nodeid
Votes Name
1
1 10.110.88.57 (local)
2
1 10.110.88.58
a
sudo pcs status
Cluster name: iscsitgt01
WARNING: no stonith devices and stonith-enabled is not false
WARNING: corosync and pacemaker node names do not match (IPs used in setup?)
Stack: corosync
Current DC: iscsitgt01a.example.com (version 1.1.15-11.el7-e174ec8) - partition with quorum
Last updated: Fri Nov 25 22:26:31 2016
Last change: Fri Nov 25 22:26:26 2016 by hacluster via crmd on iscsitgt01a.example.com
2 nodes and 0 resources configured
Online: [ iscsitgt01a.example.com iscsitgt01s.example.com ]
No resources
Daemon Status:
corosync: active/disabled
pacemaker: active/disabled
pcsd: active/enabled
※　「Current DC」が表示されるまで、何回か実行します。20秒以上かかるものと思われます。
※　「Current DC」については、どちらが選ばれていてもあまり意味のある情報ではないので気にしないでください。
※　「WARNING」について、前者は後で対応します。後者は pcs のバグ（RRP mode 未対応）なので無視してください。
○
a
Active 機と Stand-by 機で、Corosync の状態とプロセスを確認します。
sudo corosync-cfgtool -s
Printing ring status.
Local node ID 1
RING ID 0
id
= 10.110.88.57
status = ring 0 active with no faults
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 111 44590
44600
44610
44620
44630
s
44640
44650
44660
44670
44680
44690
44700
44710
44720
44730 a,s
44740
44750
44760
44770
44780
44790
44800
44810
44820
44830 ○
44840
44850
a
44860
44870
44880 ○
44890
44900
a
44910
44920
44930
44940
44950
44960
44970
44980
44990
RING ID 1
id
status
= 192.168.1.2
= ring 1 active with no faults
sudo corosync-cfgtool -s
Printing ring status.
Local node ID 2
RING ID 0
id
= 10.110.88.58
status = ring 0 active with no faults
RING ID 1
id
= 192.168.1.3
status = ring 1 active with no faults
ps -ef | egrep '[c]orosync|[p]acemaker'
root
27483
1 0 22:26 ?
root
27499
1 0 22:26 ?
haclust+ 27500 27499 0 22:26 ?
root
27501 27499 0 22:26 ?
root
27502 27499 0 22:26 ?
haclust+ 27503 27499 0 22:26 ?
haclust+ 27504 27499 0 22:26 ?
haclust+ 27505 27499 0 22:26 ?
00:00:02
00:00:00
00:00:00
00:00:00
00:00:00
00:00:00
00:00:00
00:00:00
corosync
/usr/sbin/pacemakerd -f
/usr/libexec/pacemaker/cib
/usr/libexec/pacemaker/stonithd
/usr/libexec/pacemaker/lrmd
/usr/libexec/pacemaker/attrd
/usr/libexec/pacemaker/pengine
/usr/libexec/pacemaker/crmd
Active 機で、クラスタにリソースを登録します。
sudo /etc/ha.d/crm.sh
Adding ms_drbd_r0 p_lvm (kind: Mandatory) (Options: first-action=promote then-action=start)
Active 機で、状態を確認します。
sudo pcs status
Cluster name: iscsitgt01
WARNING: corosync and pacemaker node names do not match (IPs used in setup?)
Stack: corosync
Current DC: iscsitgt01a.example.com (version 1.1.15-11.el7-e174ec8) - partition with quorum
Last updated: Fri Nov 25 22:31:49 2016
Last change: Fri Nov 25 22:31:30 2016 by root via cibadmin on iscsitgt01s.example.com
2 nodes and 5 resources configured
Online: [ iscsitgt01a.example.com iscsitgt01s.example.com ]
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 112 45000
45010
45020
45030
45040
45050
45060
45070
45080
45090
45100
45110
45120
45130
45140
45150
45160
45170
45180
45190
45200
45210
45220
45230
45240
45250
45260
45270
45280
45290
45300
45310
45320
45330
45340
45350
45360
45370
45380
45390
45400
Full list of resources:
Master/Slave Set: ms_drbd_r0 [p_drbd_r0]
p_drbd_r0 (ocf::linbit:drbd):
FAILED iscsitgt01a.example.com (unmanaged)
p_drbd_r0 (ocf::linbit:drbd):
FAILED iscsitgt01s.example.com (unmanaged)
Resource Group: g_tgt
p_lvm
(ocf::heartbeat:LVM):
Stopped
p_lio
(ocf::heartbeat:LIO):
Stopped
p_vip
(ocf::heartbeat:VIP):
Stopped
Failed Actions:
* p_drbd_r0_stop_0 on iscsitgt01a.example.com 'not configured' (6): call=6, status=complete, exitreason='none',
last-rc-change='Sat Oct 29 18:33:24 2016', queued=0ms, exec=24ms
* p_lvm_start_0 on iscsitgt01a.example.com 'unknown error' (1): call=11, status=complete, exitreason='Volume group
[vg1] does not exist or contains error!
Volume group " vg1" not found',
last-rc-change='Sat Oct 29 18:33:24 2016', queued=0ms, exec=101ms
* p_drbd_r0_stop_0 on iscsitgt01s.example.com 'not configured' (6): call=6, status=complete, exitreason='none',
last-rc-change='Sat Oct 29 18:33:24 2016', queued=0ms, exec=23ms
* p_lvm_start_0 on iscsitgt01s.example.com 'unknown error' (1): call=15, status=complete, exitreason='Volume group
[vg1] does not exist or contains error!
Volume group " vg1" not found',
last-rc-change='Tue Oct 29 18:33:24 2016', queued=0ms, exec=190ms
Daemon Status:
corosync: active/disabled
pacemaker: active/disabled
pcsd: active/enabled
○
a
○
a
Active 機で、リソースのエラー情報をクリアします。
sudo pcs resource cleanup
Waiting for 1 replies from the CRMd. OK
Active 機で、状態を確認します。
sudo pcs status
Cluster name: iscsitgt01
WARNING: corosync and pacemaker node names do not match (IPs used in setup?)
Stack: corosync
Current DC: iscsitgt01a.example.com (version 1.1.15-11.el7-e174ec8) - partition with quorum
Last updated: Fri Nov 25 22:35:07 2016
Last change: Fri Nov 25 22:31:30 2016 by hacluster via crmd on iscsitgt01s.example.com
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 113 45410
45420
45430
45440
45450
45460
45470
45480
45490
45500
45510
45520
45530
45540
45550
45560
45570
45580
45590
45600
45610 ○
45620
45630 a,s
45640
45650
45660
45670
45680
45690
45700
45710
45720
45730
45740
45750
45760
45770
45780
45790
45800
45810
2 nodes and 5 resources configured
Online: [ iscsitgt01a.example.com iscsitgt01s.example.com ]
Full list of resources:
Master/Slave Set: ms_drbd_r0 [p_drbd_r0]
Masters: [ iscsitgt01a.example.com ]
Slaves: [ iscsitgt01s.example.com ]
Resource Group: g_tgt
p_lvm
(ocf::heartbeat:LVM):
Started iscsitgt01a.example.com
p_lio
(ocf::heartbeat:LIO):
Started iscsitgt01a.example.com
p_vip
(ocf::heartbeat:VIP):
Started iscsitgt01a.example.com
Daemon Status:
corosync: active/disabled
pacemaker: active/disabled
pcsd: active/enabled
Active 機と Stand-by 機で、設定情報を保存します。
sudo pcs config | sudo tee /etc/ha.d/crm.conf
Cluster Name: iscsitgt01
Corosync Nodes:
10.110.88.57 10.110.88.58
Pacemaker Nodes:
iscsitgt01a.example.com iscsitgt01s.example.com
Resources:
Master: ms_drbd_r0
Meta Attrs: master-node-max=1 clone-max=2 clone-node-max=1 master-max=1 notify=true target-role=Started is-managed=true
Resource: p_drbd_r0 (class=ocf provider=linbit type=drbd)
Attributes: drbd_resource=r0
Operations: start interval=0s timeout=240 (p_drbd_r0-start-interval-0s)
monitor interval=10 role=Master timeout=20 (p_drbd_r0-monitor-interval-10)
monitor interval=20 role=Slave timeout=20 (p_drbd_r0-monitor-interval-20)
notify interval=0s timeout=90 (p_drbd_r0-notify-interval-0s)
stop interval=0s timeout=100 (p_drbd_r0-stop-interval-0s)
promote interval=0s timeout=90 (p_drbd_r0-promote-interval-0s)
demote interval=0s timeout=90 (p_drbd_r0-demote-interval-0s)
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 114 45820
45830
45840
45850
45860
45870
45880
45890
45900
45910
45920
45930
45940
45950
45960
45970
45980
45990
46000
46010
46020
46030
46040
46050
46060
46070
46080
46090
46100
46110
46120
46130
46140
46150
46160
46170
46180
46190
46200
46210
46220
Group: g_tgt
Resource: p_lvm (class=ocf provider=heartbeat type=LVM)
Attributes: volgrpname=vg1
Operations: start interval=0s timeout=30 (p_lvm-start-interval-0s)
monitor interval=5 timeout=10 (p_lvm-monitor-interval-5)
stop interval=0s timeout=30 (p_lvm-stop-interval-0s)
Resource: p_lio (class=ocf provider=heartbeat type=LIO)
Operations: start interval=0s timeout=10 (p_lio-start-interval-0s)
monitor interval=5 timeout=5 (p_lio-monitor-interval-5)
stop interval=0s timeout=10 (p_lio-stop-interval-0s)
Resource: p_vip (class=ocf provider=heartbeat type=VIP)
Attributes: ip=10.110.88.59 cidr_netmask=26 nic=bond0 iflabel=1 arp_interval=200 arp_count=5
Operations: start interval=0s timeout=20 (p_vip-start-interval-0s)
monitor interval=5 timeout=10 (p_vip-monitor-interval-5)
stop interval=0s timeout=20 (p_vip-stop-interval-0s)
Stonith Devices:
Fencing Levels:
Location Constraints:
Resource: g_tgt
Enabled on: iscsitgt01a.example.com (score:100) (id:lc_tgt)
Ordering Constraints:
promote ms_drbd_r0 then start p_lvm (kind:Mandatory) (id:order-ms_drbd_r0-p_lvm-mandatory)
Colocation Constraints:
g_tgt with ms_drbd_r0 (score:INFINITY) (with-rsc-role:Master) (id:colocation-g_tgt-ms_drbd_r0-INFINITY)
Ticket Constraints:
Alerts:
No alerts defined
Resources Defaults:
resource-stickiness: 200
migration-threshold: 2
Operations Defaults:
No defaults set
Cluster Properties:
batch-limit: 30
cluster-delay: 60
cluster-infrastructure: corosync
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 115 46230
46240
46250
46260
46270
46280
46290
46300
46310
46320
46330
46340
46350
46360
46370
46380
46390
46400
46410
46420
46430
46440
46450
46460
46470
46480
46490
46500
46510
46520
46530
46540
46550
46560
46570
46580
46590
46600
46610
46620
46630
cluster-name: iscsitgt01
cluster-recheck-interval: 15min
crmd-finalization-timeout: 30min
crmd-integration-timeout: 3min
crmd-transition-delay: 0s
dc-deadtime: 20s
dc-version: 1.1.15-11.el7-e174ec8
default-action-timeout: 20
election-timeout: 2min
enable-acl: true
enable-startup-probes: true
have-watchdog: false
is-managed-default: true
load-threshold: 80%
maintenance-mode: false
migration-limit: -1
no-quorum-policy: ignore
node-action-limit: 0
node-health-green: 0
node-health-red: -INFINITY
node-health-strategy: none
node-health-yellow: 0
notification-agent: /dev/null
pe-error-series-max: 100
pe-input-series-max: 100
pe-warn-series-max: 100
placement-strategy: default
remove-after-stop: false
shutdown-escalation: 20min
start-failure-is-fatal: true
startup-fencing: true
stonith-action: reboot
stonith-enabled: false
stonith-timeout: 60
stop-all-resources: false
stop-orphan-actions: true
stop-orphan-resources: true
symmetric-cluster: true
Quorum:
Options:
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 116 46640
46650 a,s
46660
46670 ○
46680
46690
a
46700
46710
46720
46730
46740
46750 ○
46760
46770
a
46780
46790
46800
46810
46820
46830
46840
46850
46860
46870
46880
46890
46900
46910
46920
46930
46940
46950
46960
46970
46980
46990
47000
47010
47020
47030
47040
sudo cp -a /etc{,~}/ha.d/crm.conf
Active 機で、スイッチオーバ（手動フェイルオーバ）させます。
sudo pcs resource move g_tgt
Warning: Creating location constraint cli-ban-g_tgt-on- iscsitgt01a.example.com with a score of -INFINITY for resource g_tgt on
node iscsitgt01a.example.com .
This will prevent g_tgt from running on iscsitgt01a.example.com until the constraint is removed. This will be the case even if
iscsitgt01a.example.com is the last node in the cluster.
Active 機で、状態を確認します。
sudo pcs status
Cluster name: iscsitgt01
WARNING: corosync and pacemaker node names do not match (IPs used in setup?)
Stack: corosync
Current DC: iscsitgt01a.example.com (version 1.1.15-11.el7-e174ec8) - partition with quorum
Last updated: Fri Nov 25 22:45:12 2016
Last change: Fri Nov 25 22:44:04 2016 by root via crm_resource on iscsitgt01a.example.com
2 nodes and 5 resources configured
Online: [ iscsitgt01a.example.com iscsitgt01s.example.com ]
Full list of resources:
Master/Slave Set: ms_drbd_r0 [p_drbd_r0]
Masters: [ iscsitgt01s.example.com ]
Slaves: [ iscsitgt01a.example.com ]
Resource Group: g_tgt
p_lvm
(ocf::heartbeat:LVM):
Started iscsitgt01s.example.com
p_lio
(ocf::heartbeat:LIO):
Started iscsitgt01s.example.com
p_vip
(ocf::heartbeat:VIP):
Started iscsitgt01s.example.com
Daemon Status:
corosync: active/disabled
pacemaker: active/disabled
pcsd: active/enabled
※　「p_vip」のノードが変わるまで、何回か実行します。
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 117 47050
47060
47070
47080
47090
47100
47110
47120
47130
47140
47150
47160
47170
47180
47190
47200
47210
47220
47230
47240
47250
47260
47270
47280
47290
47300
47310
47320
47330
47340
47350
47360
47370
47380
47390
47400
47410
47420
47430
47440
47450
○
a
○
a
a
○
a
○
a
Active 機で、設定変更を確認します。
diff <(grep -v last-lrm-refresh /etc/ha.d/crm.conf) <(sudo pcs config | grep -v last-lrm-refresh)
40a41
>
Disabled on: iscsitgt01a.example.com (score:-INFINITY) (role: Started) (id:cli-ban-g_tgt-on- iscsitgt01a.example.com )
Active 機で、設定変更を元に戻します。
sudo pcs resource clear g_tgt
diff <(grep -v last-lrm-refresh /etc/ha.d/crm.conf) <(sudo pcs config | grep -v last-lrm-refresh)
Active 機でリソースが起動した状態でない場合のみ、スイッチバック（フェイルバック）させます。
sudo pcs resource move g_tgt; sleep 5; sudo pcs resource clear g_tgt
Warning: Creating location constraint cli-ban-g_tgt-on- iscsitgt01s.example.com with a score of -INFINITY for resource g_tgt on
node iscsitgt01s.example.com .
This will prevent g_tgt from running on iscsitgt01s.example.com until the constraint is removed. This will be the case even if
iscsitgt01s.example.com is the last node in the cluster.
Active 機で、状態を確認します。
sudo pcs status
Cluster name: iscsitgt01
WARNING: corosync and pacemaker node names do not match (IPs used in setup?)
Stack: corosync
Current DC: iscsitgt01a.example.com (version 1.1.15-11.el7-e174ec8) - partition with quorum
Last updated: Fri Nov 25 22:48:03 2016
Last change: Fri Nov 25 22:47:34 2016 by root via crm_resource on iscsitgt01a.example.com
2 nodes and 5 resources configured
Online: [ iscsitgt01a.example.com iscsitgt01s.example.com ]
Full list of resources:
Master/Slave Set: ms_drbd_r0 [p_drbd_r0]
Masters: [ iscsitgt01a.example.com ]
Slaves: [ iscsitgt01s.example.com ]
Resource Group: g_tgt
p_lvm
(ocf::heartbeat:LVM):
Started iscsitgt01a.example.com
p_lio
(ocf::heartbeat:LIO):
Started iscsitgt01a.example.com
p_vip
(ocf::heartbeat:VIP):
Started iscsitgt01a.example.com
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 118 47460
47470
47480
47490
47500
47510
47520 ○
47530
47540
a
47550
47560 ○
47570
47580 a,s
47590
47600
47610
47620
47630
47640
47650
47660
47670
47680
47690
47700
47710
47720
47730
47740
47750
47760
47770
47780
47790
47800
47810
47820
47830
47840 a,s
47850
47860 ○
Daemon Status:
corosync: active/disabled
pacemaker: active/disabled
pcsd: active/enabled
Active 機で、設定変更を確認します。
diff <(grep -v last-lrm-refresh /etc/ha.d/crm.conf) <(sudo pcs config | grep -v last-lrm-refresh)
Active 機と Stand-by 機で、状態を記録します。
sudo pcs status | sudo tee /etc/ha.d/crm.status
Cluster name: iscsitgt01
WARNING: corosync and pacemaker node names do not match (IPs used in setup?)
Stack: corosync
Current DC: iscsitgt01a.example.com (version 1.1.15-11.el7-e174ec8) - partition with quorum
Last updated: Fri Nov 25 22:49:57 2016
Last change: Fri Nov 25 22:47:34 2016 by root via crm_resource on iscsitgt01a.example.com
2 nodes and 5 resources configured
Online: [ iscsitgt01a.example.com iscsitgt01s.example.com ]
Full list of resources:
Master/Slave Set: ms_drbd_r0 [p_drbd_r0]
Masters: [ iscsitgt01a.example.com ]
Slaves: [ iscsitgt01s.example.com ]
Resource Group: g_tgt
p_lvm
(ocf::heartbeat:LVM):
Started iscsitgt01a.example.com
p_lio
(ocf::heartbeat:LIO):
Started iscsitgt01a.example.com
p_vip
(ocf::heartbeat:VIP):
Started iscsitgt01a.example.com
Daemon Status:
corosync: active/disabled
pacemaker: active/disabled
pcsd: active/enabled
sudo cp -a /etc{,~}/ha.d/crm.status
Active 機で、クラスタを停止します。
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 119 47870
47880
a
47890
47900
47910
47920
47930
47940 ○
47950
47960 a,s
47970
47980 ○
47990
48000
a
48010
a
48020
48030
s
48040
s
48050
48060 ○
48070
48080
a
48090
48100
48110
48120 ○
48130
48140
a
48150
48160
48170
48180
48190
48200
48210
48220
48230
48240
48250
48260
48270
sudo pcs cluster stop --all
10.110.88.57: Stopping Cluster
10.110.88.58: Stopping Cluster
10.110.88.58: Stopping Cluster
10.110.88.57: Stopping Cluster
(pacemaker)...
(pacemaker)...
(corosync)...
(corosync)...
Active 機と Stand-by 機で、再起動します。
sudo reboot
Active 機と Stand-by 機へ、管理者用一般ユーザにて、ssh でログインします。
ssh admin@10.110.88.57
admin@10.110.88.57's password: ********
ssh admin@10.110.88.58
admin@10.110.88.58's password: ********
Active 機で、クラスタを起動します。
sudo pcs cluster start --all
10.110.88.57: Starting Cluster...
10.110.88.58: Starting Cluster...
Active 機で、状態を確認します。
sudo pcs status
Cluster name: iscsitgt01
WARNING: corosync and pacemaker node names do not match (IPs used in setup?)
Stack: corosync
Current DC: iscsitgt01s.example.com (version 1.1.15-11.el7-e174ec8) - partition with quorum
Last updated: Fri Nov 25 23:06:32 2016
Last change: Fri Nov 25 23:06:30 2016 by root via crm_resource on iscsitgt01s.example.com
2 nodes and 5 resources configured
Online: [ iscsitgt01a.example.com iscsitgt01s.example.com ]
Full list of resources:
Master/Slave Set: ms_drbd_r0 [p_drbd_r0]
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 120 48280
48290
48300
48310
48320
48330
48340
48350
48360
48370
48380
48390
Masters: [ iscsitgt01a.example.com ]
Slaves: [ iscsitgt01s.example.com ]
Resource Group: g_tgt
p_lvm
(ocf::heartbeat:LVM):
Started iscsitgt01a.example.com
p_lio
(ocf::heartbeat:LIO):
Started iscsitgt01a.example.com
p_vip
(ocf::heartbeat:VIP):
Started iscsitgt01a.example.com
Daemon Status:
corosync: active/disabled
pacemaker: active/disabled
pcsd: active/enabled
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 121 48400 ○
48410
48420 ○
48430
48440 a,s
48450
48460 ○
48470
48480 a,s
48490
48500
48510 ○
48520
48530
o
48540
o
48550
o
48560
o
48570
o
48580
o
48590
o
48600
o
48610
o
48620
o
48630
o
48640
o
48650
o
48660
o
48670
o
48680
o
48690
o
48700
o
48710
o
48720
o
48730
o
48740
o
48750
o
48760
o
48770
48780 ○
48790
48800 a,s
【Munin のインストールと初期設定】
以下のインストーラを DVD ドライブにセットします。
# V834394-01.iso (Oracle Linux 7.3)
インストーラをマウントします。
sudo mount /dev/cdrom /mnt
mount: /dev/sr0 is write-protected, mounting read-only
インターネットと接続可能な端末で以下のコマンドを実行する等して、必要なパッケージを収集します。
curl
curl
curl
curl
curl
curl
curl
curl
curl
curl
curl
curl
curl
curl
curl
curl
curl
curl
curl
curl
curl
curl
curl
curl
-O
-O
-O
-O
-O
-O
-O
-O
-O
-O
-O
-O
-O
-O
-O
-O
-O
-O
-O
-O
-O
-O
-O
-O
http://yum.oracle.com/repo/OracleLinux/OL7/optional/latest/x86_64/getPackage/perl-Crypt-DES-2.05-20.el7.x86_64.rpm
http://yum.oracle.com/repo/OracleLinux/OL7/optional/latest/x86_64/getPackage/perl-File-Copy-Recursive-0.38-14.el7.noarch.rpm
http://yum.oracle.com/repo/OracleLinux/OL7/optional/latest/x86_64/getPackage/perl-Taint-Runtime-0.03-19.el7.x86_64.rpm
http://yum.oracle.com/repo/OracleLinux/OL7/optional/latest/x86_64/getPackage/perl-XML-DOM-1.44-19.el7.noarch.rpm
http://yum.oracle.com/repo/OracleLinux/OL7/optional/latest/x86_64/getPackage/perl-XML-RegExp-0.04-2.el7.noarch.rpm
http://yum.oracle.com/repo/OracleLinux/OL7/optional/latest/x86_64/getPackage/rrdtool-perl-1.4.8-9.el7.x86_64.rpm
https://dl.fedoraproject.org/pub/epel/7/x86_64/m/munin-2.0.25-11.el7.noarch.rpm
https://dl.fedoraproject.org/pub/epel/7/x86_64/m/munin-common-2.0.25-11.el7.noarch.rpm
https://dl.fedoraproject.org/pub/epel/7/x86_64/m/munin-node-2.0.25-11.el7.noarch.rpm
https://dl.fedoraproject.org/pub/epel/7/x86_64/p/perl-Cache-Cache-1.06-12.el7.noarch.rpm
https://dl.fedoraproject.org/pub/epel/7/x86_64/p/perl-Email-Date-Format-1.002-15.el7.noarch.rpm
https://dl.fedoraproject.org/pub/epel/7/x86_64/p/perl-HTML-Template-2.95-1.el7.noarch.rpm
https://dl.fedoraproject.org/pub/epel/7/x86_64/p/perl-IO-Multiplex-1.13-6.el7.noarch.rpm
https://dl.fedoraproject.org/pub/epel/7/x86_64/p/perl-IPC-ShareLite-0.17-12.el7.x86_64.rpm
https://dl.fedoraproject.org/pub/epel/7/x86_64/p/perl-Log-Dispatch-2.41-1.el7.1.noarch.rpm
https://dl.fedoraproject.org/pub/epel/7/x86_64/p/perl-Log-Dispatch-FileRotate-1.19-13.el7.noarch.rpm
https://dl.fedoraproject.org/pub/epel/7/x86_64/p/perl-Log-Log4perl-1.42-2.el7.noarch.rpm
https://dl.fedoraproject.org/pub/epel/7/x86_64/p/perl-MIME-Lite-3.030-1.el7.noarch.rpm
https://dl.fedoraproject.org/pub/epel/7/x86_64/p/perl-MIME-Types-1.38-2.el7.noarch.rpm
https://dl.fedoraproject.org/pub/epel/7/x86_64/p/perl-Mail-Sender-0.8.23-1.el7.noarch.rpm
https://dl.fedoraproject.org/pub/epel/7/x86_64/p/perl-Mail-Sendmail-0.79-21.el7.noarch.rpm
https://dl.fedoraproject.org/pub/epel/7/x86_64/p/perl-Net-CIDR-0.18-1.el7.noarch.rpm
https://dl.fedoraproject.org/pub/epel/7/x86_64/p/perl-Net-SNMP-6.0.1-7.el7.noarch.rpm
https://dl.fedoraproject.org/pub/epel/7/x86_64/p/perl-Net-Server-2.007-2.el7.noarch.rpm
収集したパッケージをホームディレクトリにコピーし、確認します。
scp xxxx@yyy:perl-Crypt-DES- 2.05-20.el7.x86_64.rpm .
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 122 48810
48820
48830
48840
48850
48860
48870
48880
48890
48900
48910
48920
48930
48940
48950
48960
48970
48980
48990
49000
49010
49020
49030
49040
49050
49060
49070
49080
49090
49100
49110
49120
49130
49140
49150
49160
49170
49180
49190
49200
49210
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
scp
scp
scp
scp
scp
scp
scp
scp
scp
scp
scp
scp
scp
scp
scp
scp
scp
scp
scp
scp
scp
scp
scp
xxxx@yyy:perl-File-Copy-Recursive- 0.38-14.el7.noarch.rpm .
xxxx@yyy:perl-Taint-Runtime- 0.03-19.el7.x86_64.rpm .
xxxx@yyy:perl-XML-DOM- 1.44-19.el7.noarch.rpm .
xxxx@yyy:perl-XML-RegExp- 0.04-2.el7.noarch.rpm .
xxxx@yyy:rrdtool-perl- 1.4.8-9.el7.x86_64.rpm .
xxxx@yyy:munin-2.0.25-11.el7 .noarch.rpm .
xxxx@yyy:munin-common- 2.0.25-11.el7 .noarch.rpm .
xxxx@yyy:munin-node-2.0.25-11.el7 .noarch.rpm .
xxxx@yyy:perl-Cache-Cache- 1.06-12.el7.noarch.rpm .
xxxx@yyy:perl-Email-Date-Format- 1.002-15.el7.noarch.rpm .
xxxx@yyy:perl-HTML-Template- 2.95-1.el7.noarch.rpm .
xxxx@yyy:perl-IO-Multiplex- 1.13-6.el7.noarch.rpm .
xxxx@yyy:perl-IPC-ShareLite- 0.17-12.el7.x86_64.rpm .
xxxx@yyy:perl-Log-Dispatch- 2.41-1.el7.1.noarch.rpm .
xxxx@yyy:perl-Log-Dispatch-FileRotate- 1.19-13.el7.noarch.rpm .
xxxx@yyy:perl-Log-Log4perl- 1.42-2.el7.noarch.rpm .
xxxx@yyy:perl-MIME-Lite- 3.030-1.el7.noarch.rpm .
xxxx@yyy:perl-MIME-Types- 1.38-2.el7.noarch.rpm .
xxxx@yyy:perl-Mail-Sender- 0.8.23-1.el7.noarch.rpm .
xxxx@yyy:perl-Mail-Sendmail- 0.79-21.el7.noarch.rpm .
xxxx@yyy:perl-Net-CIDR- 0.18-1.el7.noarch.rpm .
xxxx@yyy:perl-Net-SNMP- 6.0.1-7.el7.noarch.rpm .
xxxx@yyy:perl-Net-Server- 2.007-2.el7.noarch.rpm .
a,s
ls -l *.rpm
-rw-rw-r-- 1
-rw-rw-r-- 1
-rw-rw-r-- 1
-rw-rw-r-- 1
-rw-rw-r-- 1
-rw-rw-r-- 1
-rw-rw-r-- 1
-rw-rw-r-- 1
-rw-rw-r-- 1
-rw-rw-r-- 1
-rw-rw-r-- 1
-rw-rw-r-- 1
-rw-rw-r-- 1
-rw-rw-r-- 1
-rw-rw-r-- 1
-rw-rw-r-- 1
admin
admin
admin
admin
admin
admin
admin
admin
admin
admin
admin
admin
admin
admin
admin
admin
admin 204328 Nov 25 23:11 munin-2.0.25-11.el7.noarch.rpm
admin 93672 Nov 25 23:12 munin-common-2.0.25-11.el7.noarch.rpm
admin 408204 Nov 25 23:12 munin-node-2.0.25-11.el7.noarch.rpm
admin 93340 Nov 25 23:12 perl-Cache-Cache-1.06-12.el7.noarch.rpm
admin 19920 Nov 25 23:11 perl-Crypt-DES-2.05-20.el7.x86_64.rpm
admin 17524 Nov 25 23:12 perl-Email-Date-Format-1.002-15.el7.noarch.rpm
admin 23164 Nov 25 23:11 perl-File-Copy-Recursive-0.38-14.el7.noarch.rpm
admin 77780 Nov 25 23:12 perl-HTML-Template-2.95-1.el7.noarch.rpm
admin 25616 Nov 25 23:12 perl-IO-Multiplex-1.13-6.el7.noarch.rpm
admin 31004 Nov 25 23:12 perl-IPC-ShareLite-0.17-12.el7.x86_64.rpm
admin 84300 Nov 25 23:12 perl-Log-Dispatch-2.41-1.el7.1.noarch.rpm
admin 25232 Nov 25 23:12 perl-Log-Dispatch-FileRotate-1.19-13.el7.noarch.rpm
admin 433560 Nov 25 23:12 perl-Log-Log4perl-1.42-2.el7.noarch.rpm
admin 60212 Nov 25 23:12 perl-Mail-Sender-0.8.23-1.el7.noarch.rpm
admin 29540 Nov 25 23:12 perl-Mail-Sendmail-0.79-21.el7.noarch.rpm
admin 98316 Nov 25 23:12 perl-MIME-Lite-3.030-1.el7.noarch.rpm
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 123 49220
49230
49240
49250
49260
49270
49280
49290
49300
49310
49320
49330
49340
49350
49360
49370
49380
49390
49400
49410
49420
49430
49440
49450
49460
49470
49480
49490
49500
49510
49520
49530
49540
49550
49560
49570
49580
49590
49600
49610
49620
-rw-rw-r--rw-rw-r--rw-rw-r--rw-rw-r--rw-rw-r--rw-rw-r--rw-rw-r--rw-rw-r-a,s
○
a,s
a,s
a,s
1
1
1
1
1
1
1
1
admin
admin
admin
admin
admin
admin
admin
admin
admin 39184 Nov 25 23:12 perl-MIME-Types-1.38-2.el7.noarch.rpm
admin 19640 Nov 25 23:12 perl-Net-CIDR-0.18-1.el7.noarch.rpm
admin 213136 Nov 25 23:12 perl-Net-Server-2.007-2.el7.noarch.rpm
admin 105348 Nov 25 23:12 perl-Net-SNMP-6.0.1-7.el7.noarch.rpm
admin 22496 Nov 25 23:11 perl-Taint-Runtime-0.03-19.el7.x86_64.rpm
admin 141504 Nov 25 23:11 perl-XML-DOM-1.44-19.el7.noarch.rpm
admin 10628 Nov 25 23:11 perl-XML-RegExp-0.04-2.el7.noarch.rpm
admin 42864 Nov 25 23:11 rrdtool-perl-1.4.8-9.el7.x86_64.rpm
file *.rpm
munin-2.0.25-11.el7.noarch.rpm:
munin-common-2.0.25-11.el7.noarch.rpm:
munin-node-2.0.25-11.el7.noarch.rpm:
perl-Cache-Cache-1.06-12.el7.noarch.rpm:
perl-Crypt-DES-2.05-20.el7.x86_64.rpm:
perl-Email-Date-Format-1.002-15.el7.noarch.rpm:
perl-File-Copy-Recursive-0.38-14.el7.noarch.rpm:
perl-HTML-Template-2.95-1.el7.noarch.rpm:
perl-IO-Multiplex-1.13-6.el7.noarch.rpm:
perl-IPC-ShareLite-0.17-12.el7.x86_64.rpm:
perl-Log-Dispatch-2.41-1.el7.1.noarch.rpm:
perl-Log-Dispatch-FileRotate-1.19-13.el7.noarch.rpm:
perl-Log-Log4perl-1.42-2.el7.noarch.rpm:
perl-Mail-Sender-0.8.23-1.el7.noarch.rpm:
perl-Mail-Sendmail-0.79-21.el7.noarch.rpm:
perl-MIME-Lite-3.030-1.el7.noarch.rpm:
perl-MIME-Types-1.38-2.el7.noarch.rpm:
perl-Net-CIDR-0.18-1.el7.noarch.rpm:
perl-Net-Server-2.007-2.el7.noarch.rpm:
perl-Net-SNMP-6.0.1-7.el7.noarch.rpm:
perl-Taint-Runtime-0.03-19.el7.x86_64.rpm:
perl-XML-DOM-1.44-19.el7.noarch.rpm:
perl-XML-RegExp-0.04-2.el7.noarch.rpm:
rrdtool-perl-1.4.8-9.el7.x86_64.rpm:
RPM
RPM
RPM
RPM
RPM
RPM
RPM
RPM
RPM
RPM
RPM
RPM
RPM
RPM
RPM
RPM
RPM
RPM
RPM
RPM
RPM
RPM
RPM
RPM
v3.0
v3.0
v3.0
v3.0
v3.0
v3.0
v3.0
v3.0
v3.0
v3.0
v3.0
v3.0
v3.0
v3.0
v3.0
v3.0
v3.0
v3.0
v3.0
v3.0
v3.0
v3.0
v3.0
v3.0
bin
bin
bin
bin
bin
bin
bin
bin
bin
bin
bin
bin
bin
bin
bin
bin
bin
bin
bin
bin
bin
bin
bin
bin
noarch munin-2.0.25-11.el7
noarch munin-common-2.0.25-11.el7
noarch munin-node-2.0.25-11.el7
noarch perl-Cache-Cache-1.06-12.el7
i386/x86_64 perl-Crypt-DES-2.05-20.el7
noarch perl-Email-Date-Format-1.002-15.el7
noarch perl-File-Copy-Recursive-0.38-14.el7
noarch perl-HTML-Template-2.95-1.el7
noarch perl-IO-Multiplex-1.13-6.el7
i386/x86_64 perl-IPC-ShareLite-0.17-12.el7
noarch perl-Log-Dispatch-2.41-1.el7.1
noarch perl-Log-Dispatch-FileRotate-1.19-13.el7
noarch perl-Log-Log4perl-1.42-2.el7
noarch perl-Mail-Sender-0.8.23-1.el7
noarch perl-Mail-Sendmail-0.79-21.el7
noarch perl-MIME-Lite-3.030-1.el7
noarch perl-MIME-Types-1.38-2.el7
noarch perl-Net-CIDR-0.18-1.el7
noarch perl-Net-Server-2.007-2.el7
noarch perl-Net-SNMP-6.0.1-7.el7
i386/x86_64 perl-Taint-Runtime-0.03-19.el7
noarch perl-XML-DOM-1.44-19.el7
noarch perl-XML-RegExp-0.04-2.el7
i386/x86_64 rrdtool-perl-1.4.8-9.el7
Munin をインストールします。Oracle 社サポート外のパッケージです。
sudo yum -y --disablerepo=\* --enablerepo=media install httpd
sudo yum -y --disablerepo=\* --enablerepo=media localinstall munin-*.rpm perl-*.rpm rrdtool-perl-*.rpm
sudo mv munin-*.rpm perl-*.rpm rrdtool-perl-*.rpm /opt/packages/
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 124 49630
49640
49650
49660
49670
49680
49690
49700
49710
49720
49730
49740
49750
49760
49770
49780
49790
49800
49810
49820
49830
49840
49850
49860
49870
49880
49890
49900
49910
49920
49930
49940
49950
49960
49970
49980
49990
50000
50010
50020
50030
○
a,s
○
a,s
○
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
○
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
インストーラをアンマウントします。
sudo umount /mnt
インストーラをDVDドライブから外します。
# Eject DVD
追加インストールしたパッケージの設定をバックアップします。
sudo
sudo
sudo
sudo
sudo
sudo
sudo
sudo
sudo
sudo
sudo
sudo
sudo
sudo
sudo
sudo
sudo
cp
cp
cp
cp
cp
cp
cp
cp
cp
cp
cp
cp
cp
cp
cp
cp
cp
-a
-a
-a
-a
-a
-a
-a
-a
-a
-a
-a
-a
-a
-a
-a
-a
-a
/etc{,~}/cron.d/munin
/etc{,~}/fonts
/etc{,~}/httpd
/etc{,~}/logrotate.d/httpd
/etc{,~}/logrotate.d/munin
/etc{,~}/logrotate.d/munin-node
/etc{,~}/munin
/etc{,~}/sysconfig/htcacheclean
/etc{,~}/sysconfig/httpd
/etc/passwd
/etc~/passwd_$(date +%Y%m%d_%H%M%S)
/etc/passwd- /etc~/passwd-_$(date +%Y%m%d_%H%M%S)
/etc/shadow
/etc~/shadow_$(date +%Y%m%d_%H%M%S)
/etc/shadow- /etc~/shadow-_$(date +%Y%m%d_%H%M%S)
/etc/group
/etc~/group_$(date +%Y%m%d_%H%M%S)
/etc/group/etc~/group-_$(date +%Y%m%d_%H%M%S)
/etc/gshadow /etc~/gshadow_$(date +%Y%m%d_%H%M%S)
/etc/gshadow- /etc~/gshadow-_$(date +%Y%m%d_%H%M%S)
DRBD, LIO に関するプラグインを作成します。
cat << 'EOF' | sudo tee /usr/share/munin/plugins/drbd
#!/usr/bin/perl
#%# family=auto
#%# capabilities=autoconf
# http://www.drbd.org/en/doc/users-guide-84/ch-admin#s-performance-indicators
use strict;
my $file="/proc/drbd";
my $store = {};
my $rid;
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 125 50040
50050
50060
50070
50080
50090
50100
50110
50120
50130
50140
50150
50160
50170
50180
50190
50200
50210
50220
50230
50240
50250
50260
50270
50280
50290
50300
50310
50320
50330
50340
50350
50360
50370
50380
50390
50400
50410
50420
50430
50440
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
&crunch;
&display;
sub display{
if ($ARGV[0] and $ARGV[0] eq "config"){
print "graph_title DRBD\n";
print "graph_category DRBD\n";
print "graph_info Graph DRBD\n";
print "graph_vlabel Graph DRBD (Bytes/sec)\n";
print "graph_scale yes\n";
print "graph_args --base 1024 --lower-limit 0\n";
print "graph_period second\n";
print "graph_height 200\n";
print "graph_width 400\n";
print "graph_printf %7.2lf\n";
foreach my $key ( keys %$store ){
my $drbdname = 'drbd'.$key;
print $drbdname."dr.label $drbdname Disk Read\n";
print $drbdname."dw.label $drbdname Disk Write\n";
print $drbdname."ns.label $drbdname Network Send\n";
print $drbdname."nr.label $drbdname Network Receive\n";
print $drbdname."os.label $drbdname Out of Sync\n";
print $drbdname."dr.cdef ".$drbdname."dr,1024,*\n";
print $drbdname."dw.cdef ".$drbdname."dw,1024,*\n";
print $drbdname."ns.cdef ".$drbdname."ns,1024,*\n";
print $drbdname."nr.cdef ".$drbdname."nr,1024,*\n";
print $drbdname."os.cdef ".$drbdname."os,1024,*\n";
print $drbdname."dr.min 0\n";
print $drbdname."dw.min 0\n";
print $drbdname."ns.min 0\n";
print $drbdname."nr.min 0\n";
print $drbdname."os.min 0\n";
print $drbdname."dr.type DERIVE\n";
print $drbdname."dw.type DERIVE\n";
print $drbdname."ns.type DERIVE\n";
print $drbdname."nr.type DERIVE\n";
print $drbdname."os.type DERIVE\n";
}
exit 0;
}
foreach my $key ( keys %$store ){
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 126 50450
50460
50470
50480
50490
50500
50510
50520
50530
50540
50550
50560
50570
50580
50590
50600
50610
50620
50630
50640
50650
50660
50670
50680
50690
50700
50710
50720
50730
50740
50750
50760
50770
50780
50790
50800
50810
50820
50830
50840
50850
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
my $drbdname = 'drbd'.$key;
print $drbdname."dw.value ".$store->{$key}->{'dw'}."\n";
print $drbdname."dr.value ".$store->{$key}->{'dr'}."\n";
print $drbdname."ns.value ".$store->{$key}->{'ns'}."\n";
print $drbdname."nr.value ".$store->{$key}->{'nr'}."\n";
print $drbdname."os.value ".$store->{$key}->{'os'}."\n";
}
}
sub crunch{
open (IN, $file) || die "Could not open $file for reading:
if ($ARGV[0] and $ARGV[0] eq "autoconf"){
close (IN);
print "yes\n";
exit 0;
}
while (<IN>){
next if /version:|GIT-hash:/;
chomp;
my ($drbd) = $_ =~ /^\s+(\d):/;
$rid = $drbd if $drbd =~ /\d/;
my ($ns) = $_ =~ /ns:(\d*)/; $store->{ $rid }->{'ns'} =
my ($nr) = $_ =~ /nr:(\d*)/; $store->{ $rid }->{'nr'} =
my ($dw) = $_ =~ /dw:(\d*)/; $store->{ $rid }->{'dw'} =
my ($dr) = $_ =~ /dr:(\d*)/; $store->{ $rid }->{'dr'} =
my ($os) = $_ =~ /oos:(\d*)/; $store->{ $rid }->{'os'} =
}
close (IN);
}
$!";
$ns
$nr
$dw
$dr
$os
if
if
if
if
if
$ns
$ns
$dw
$dr
$os
ne
ne
ne
ne
ne
undef;
undef;
undef;
undef;
undef;
exit 0;
EOF
sudo chmod 755 /usr/share/munin/plugins/drbd
cat << 'EOF' | sudo tee /usr/share/munin/plugins/drbd_al
#!/usr/bin/perl
#%# family=auto
#%# capabilities=autoconf
# http://www.drbd.org/en/doc/users-guide-84/ch-admin#s-performance-indicators
use strict;
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 127 50860
50870
50880
50890
50900
50910
50920
50930
50940
50950
50960
50970
50980
50990
51000
51010
51020
51030
51040
51050
51060
51070
51080
51090
51100
51110
51120
51130
51140
51150
51160
51170
51180
51190
51200
51210
51220
51230
51240
51250
51260
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
my $file="/proc/drbd";
my $store = {};
my $rid;
&crunch;
&display;
sub display{
if ($ARGV[0] and $ARGV[0] eq "config"){
print "graph_title DRBD (Activity Log)\n";
print "graph_category DRBD\n";
print "graph_info Graph DRBD (Activity Log)\n";
print "graph_vlabel Graph DRBD (Activity Log)\n";
print "graph_scale yes\n";
print "graph_args --base 1024 --lower-limit 0\n";
print "graph_period second\n";
print "graph_height 200\n";
print "graph_width 400\n";
print "graph_printf %7.2lf\n";
foreach my $key ( keys %$store ){
my $drbdname = 'drbd'.$key;
print $drbdname."al.label $drbdname Activity log\n";
print $drbdname."al.min 0\n";
#
print $drbdname."al.type DERIVE\n";
}
exit 0;
}
foreach my $key ( keys %$store ){
my $drbdname = 'drbd'.$key;
print $drbdname."al.value ".$store->{$key}->{'al'}."\n";
}
}
sub crunch{
open (IN, $file ) || die "Could not open $file for reading: $!";
if ($ARGV[0] and $ARGV[0] eq "autoconf"){
close (IN);
print "yes\n";
exit 0;
}
while (<IN>){
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 128 51270
51280
51290
51300
51310
51320
51330
51340
51350
51360
51370
51380
51390
51400
51410
51420
51430
51440
51450
51460
51470
51480
51490
51500
51510
51520
51530
51540
51550
51560
51570
51580
51590
51600
51610
51620
51630
51640
51650
51660
51670
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
next if /version:|GIT-hash:/;
chomp;
my ($drbd) = $_ =~ /^\s+(\d):/;
$rid = $drbd if $drbd =~ /\d/;
my ($al) = $_ =~ /al:(\d*)/; $store->{ $rid }->{'al'} = $al if $al ne undef;
}
close (IN);
}
exit 0;
EOF
sudo chmod 755 /usr/share/munin/plugins/drbd_al
cat << 'EOF' | sudo tee /usr/share/munin/plugins/drbd_ext
#!/usr/bin/perl
#%# family=auto
#%# capabilities=autoconf
# http://www.drbd.org/en/doc/users-guide-84/ch-admin#s-performance-indicators
use strict;
my $file="/proc/drbd";
my $store = {};
my $rid;
&crunch;
&display;
sub display{
if ($ARGV[0] and $ARGV[0] eq "config"){
print "graph_title DRBD (Ext)\n";
print "graph_category DRBD\n";
print "graph_info Graph DRBD (Ext)\n";
print "graph_vlabel Graph DRBD (Ext)\n";
print "graph_scale yes\n";
print "graph_args --base 1024 --lower-limit 0\n";
print "graph_period second\n";
print "graph_height 200\n";
print "graph_width 400\n";
print "graph_printf %7.2lf\n";
foreach my $key ( keys %$store ){
my $drbdname = 'drbd'.$key;
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 129 51680
51690
51700
51710
51720
51730
51740
51750
51760
51770
51780
51790
51800
51810
51820
51830
51840
51850
51860
51870
51880
51890
51900
51910
51920
51930
51940
51950
51960
51970
51980
51990
52000
52010
52020
52030
52040
52050
52060
52070
52080
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
print
print
print
print
print
print
$drbdname."bm.label
$drbdname."lo.label
$drbdname."pe.label
$drbdname."ua.label
$drbdname."ap.label
$drbdname."ep.label
$drbdname
$drbdname
$drbdname
$drbdname
$drbdname
$drbdname
Bit Map\n";
Local count\n";
Pending\n";
UnAcknowledged\n";
Application Pending\n";
Epochs\n";
}
exit 0;
}
foreach my $key ( keys %$store ){
my $drbdname = 'drbd'.$key;
print $drbdname."bm.value ".$store->{$key}->{'bm'}."\n";
print $drbdname."lo.value ".$store->{$key}->{'lo'}."\n";
print $drbdname."pe.value ".$store->{$key}->{'pe'}."\n";
print $drbdname."ua.value ".$store->{$key}->{'ua'}."\n";
print $drbdname."ap.value ".$store->{$key}->{'ap'}."\n";
print $drbdname."ep.value ".$store->{$key}->{'ep'}."\n";
}
}
sub crunch{
open (IN, $file ) || die "Could not open $file for reading: $!";
if ($ARGV[0] and $ARGV[0] eq "autoconf"){
close (IN);
print "yes\n";
exit 0;
}
while (<IN>){
next if /version:|GIT-hash:/;
chomp;
my ($drbd) = $_ =~ /^\s+(\d):/;
$rid = $drbd if $drbd =~ /\d/;
my ($bm) = $_ =~ /bm:(\d*)/; $store->{ $rid }->{'bm'} = $bm if
my ($lo) = $_ =~ /lo:(\d*)/; $store->{ $rid }->{'lo'} = $lo if
my ($pe) = $_ =~ /pe:(\d*)/; $store->{ $rid }->{'pe'} = $pe if
my ($ua) = $_ =~ /ua:(\d*)/; $store->{ $rid }->{'ua'} = $ua if
my ($ap) = $_ =~ /ap:(\d*)/; $store->{ $rid }->{'ap'} = $ap if
my ($ep) = $_ =~ /ep:(\d*)/; $store->{ $rid }->{'ep'} = $ep if
}
close (IN);
}
$bm
$lo
$pe
$ua
$ap
$ep
ne
ne
ne
ne
ne
ne
undef;
undef;
undef;
undef;
undef;
undef;
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 130 52090
52100
52110
52120
52130
52140
52150
52160
52170
52180
52190
52200
52210
52220
52230
52240
52250
52260
52270
52280
52290
52300
52310
52320
52330
52340
52350
52360
52370
52380
52390
52400
52410
52420
52430
52440
52450
52460
52470
52480
52490
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
exit 0;
EOF
sudo chmod 755 /usr/share/munin/plugins/drbd_ext
cat << 'EOF' | sudo tee /usr/share/munin/plugins/lio_read
#!/bin/sh
#%# family=auto
#%# capabilities=autoconf
if [ "$1" = "autoconf" ]; then
if [ -d /sys/kernel/config/target/iscsi/iqn.*/tpgt_1 ]; then
echo yes
else
echo 'no (no iscsi target)'
fi
exit 0
fi
if [ "$1" = "config" ]; then
echo 'graph_title LIO (Read)'
echo 'graph_category LIO'
echo 'graph_info Graph LIO (Read)'
echo 'graph_vlabel Graph LIO (Bytes/sec)'
echo 'graph_scale yes'
echo 'graph_args --base 1024 --lower-limit 0'
echo 'graph_period second'
# echo 'graph_height 200'
# echo 'graph_width 400'
echo 'graph_printf %7.2lf'
TGT_=
INI_=
for i in $(echo /sys/kernel/config/target/iscsi/iqn.*/tpgt_1/acls/iqn.*/*/statistics/scsi_auth_intr/read_mbytes | LANG=C sort)
do
TGT=$(echo $i | cut -d/ -f7)
INI=$(echo $i | cut -d/ -f10)
LUN=$(echo $i | cut -d/ -f11)
if [ "$TGT_" = "$TGT" ]; then
if [ "$INI_" = "$INI" ]; then
:
else
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 131 52500
52510
52520
52530
52540
52550
52560
52570
52580
52590
52600
52610
52620
52630
52640
52650
52660
52670
52680
52690
52700
52710
52720
52730
52740
52750
52760
52770
52780
52790
52800
52810
52820
52830
52840
52850
52860
52870
52880
52890
52900
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
INI_=$INI
INI_F=$(echo $INI | tr "[:upper:]" "[:lower:]" | sed -e 's/-/_/g' -e 's![^a-z0-9_]!!g')
fi
else
TGT_=$TGT
TGT_F=$(echo $TGT | tr "[:upper:]" "[:lower:]" | sed -e 's/-/_/g' -e 's![^a-z0-9_]!!g')
INI_=$INI
INI_F=$(echo $INI | tr "[:upper:]" "[:lower:]" | sed -e 's/-/_/g' -e 's![^a-z0-9_]!!g')
for j in $(echo /sys/kernel/config/target/iscsi/$TGT/tpgt_1/lun/*/statistics/scsi_tgt_port/read_mbytes | LANG=C sort)
do
LUN_=$(echo $j | cut -d/ -f10)
echo ${TGT_F}$LUN_.label $(echo $TGT | cut -d: -f2) \($LUN_\) Read
echo ${TGT_F}$LUN_.cdef ${TGT_F}$LUN_,1048576,\*
echo ${TGT_F}$LUN_.min 0
echo ${TGT_F}$LUN_.type DERIVE
done
fi
echo
echo
echo
echo
done
exit 0
${TGT_F}${INI_F}$LUN.label $(echo $TGT | cut -d: -f2) - $(echo $INI | cut -d: -f2) \($LUN\) Read
${TGT_F}${INI_F}$LUN.cdef ${TGT_F}${INI_F}$LUN,1048576,\*
${TGT_F}${INI_F}$LUN.min 0
${TGT_F}${INI_F}$LUN.type DERIVE
fi
TGT_=
INI_=
for i in $(echo /sys/kernel/config/target/iscsi/iqn.*/tpgt_1/acls/iqn.*/*/statistics/scsi_auth_intr/read_mbytes | LANG=C sort)
do
TGT=$(echo $i | cut -d/ -f7)
INI=$(echo $i | cut -d/ -f10)
LUN=$(echo $i | cut -d/ -f11)
if [ "$TGT_" = "$TGT" ]; then
if [ "$INI_" = "$INI" ]; then
:
else
INI_=$INI
INI_F=$(echo $INI | tr "[:upper:]" "[:lower:]" | sed -e 's/-/_/g' -e 's![^a-z0-9_]!!g')
fi
else
TGT_=$TGT
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 132 52910
52920
52930
52940
52950
52960
52970
52980
52990
53000
53010
53020
53030
53040
53050
53060
53070
53080
53090
53100
53110
53120
53130
53140
53150
53160
53170
53180
53190
53200
53210
53220
53230
53240
53250
53260
53270
53280
53290
53300
53310
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
TGT_F=$(echo $TGT | tr "[:upper:]" "[:lower:]" | sed -e 's/-/_/g' -e 's![^a-z0-9_]!!g')
INI_=$INI
INI_F=$(echo $INI | tr "[:upper:]" "[:lower:]" | sed -e 's/-/_/g' -e 's![^a-z0-9_]!!g')
for j in $(echo /sys/kernel/config/target/iscsi/$TGT/tpgt_1/lun/*/statistics/scsi_tgt_port/read_mbytes | LANG=C sort)
do
LUN_=$(echo $j | cut -d/ -f10)
echo -n "${TGT_F}$LUN_.value "
cat $j
done
fi
echo -n "${TGT_F}${INI_F}$LUN.value "
cat $i
done
exit 0;
EOF
sudo chmod 755 /usr/share/munin/plugins/lio_read
cat << 'EOF' | sudo tee /usr/share/munin/plugins/lio_write
#!/bin/sh
#%# family=auto
#%# capabilities=autoconf
if [ "$1" = "autoconf" ]; then
if [ -d /sys/kernel/config/target/iscsi/iqn.*/tpgt_1 ]; then
echo yes
else
echo 'no (no iscsi target)'
fi
exit 0
fi
if [ "$1" = "config" ]; then
echo 'graph_title LIO (Write)'
echo 'graph_category LIO'
echo 'graph_info Graph LIO (Write)'
echo 'graph_vlabel Graph LIO (Bytes/sec)'
echo 'graph_scale yes'
echo 'graph_args --base 1024 --lower-limit 0'
echo 'graph_period second'
# echo 'graph_height 200'
# echo 'graph_width 400'
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 133 53320
53330
53340
53350
53360
53370
53380
53390
53400
53410
53420
53430
53440
53450
53460
53470
53480
53490
53500
53510
53520
53530
53540
53550
53560
53570
53580
53590
53600
53610
53620
53630
53640
53650
53660
53670
53680
53690
53700
53710
53720
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
echo 'graph_printf %7.2lf'
TGT_=
INI_=
for i in $(echo /sys/kernel/config/target/iscsi/iqn.*/tpgt_1/acls/iqn.*/*/statistics/scsi_auth_intr/write_mbytes | LANG=C sort)
do
TGT=$(echo $i | cut -d/ -f7)
INI=$(echo $i | cut -d/ -f10)
LUN=$(echo $i | cut -d/ -f11)
if [ "$TGT_" = "$TGT" ]; then
if [ "$INI_" = "$INI" ]; then
:
else
INI_=$INI
INI_F=$(echo $INI | tr "[:upper:]" "[:lower:]" | sed -e 's/-/_/g' -e 's![^a-z0-9_]!!g')
fi
else
TGT_=$TGT
TGT_F=$(echo $TGT | tr "[:upper:]" "[:lower:]" | sed -e 's/-/_/g' -e 's![^a-z0-9_]!!g')
INI_=$INI
INI_F=$(echo $INI | tr "[:upper:]" "[:lower:]" | sed -e 's/-/_/g' -e 's![^a-z0-9_]!!g')
for j in $(echo /sys/kernel/config/target/iscsi/$TGT/tpgt_1/lun/*/statistics/scsi_tgt_port/write_mbytes | LANG=C sort)
do
LUN_=$(echo $j | cut -d/ -f10)
echo ${TGT_F}$LUN_.label $(echo $TGT | cut -d: -f2) \($LUN_\) Write
echo ${TGT_F}$LUN_.cdef ${TGT_F}$LUN_,1048576,\*
echo ${TGT_F}$LUN_.min 0
echo ${TGT_F}$LUN_.type DERIVE
done
fi
echo
echo
echo
echo
done
exit 0
${TGT_F}${INI_F}$LUN.label $(echo $TGT | cut -d: -f2) - $(echo $INI | cut -d: -f2) \($LUN\) Write
${TGT_F}${INI_F}$LUN.cdef ${TGT_F}${INI_F}$LUN,1048576,\*
${TGT_F}${INI_F}$LUN.min 0
${TGT_F}${INI_F}$LUN.type DERIVE
fi
TGT_=
INI_=
for i in $(echo /sys/kernel/config/target/iscsi/iqn.*/tpgt_1/acls/iqn.*/*/statistics/scsi_auth_intr/write_mbytes | LANG=C sort)
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 134 53730
53740
53750
53760
53770
53780
53790
53800
53810
53820
53830
53840
53850
53860
53870
53880
53890
53900
53910
53920
53930
53940
53950
53960
53970
53980
53990
54000
54010
54020
54030
54040
54050
54060
54070
54080
54090
54100
54110
54120
54130
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
○
do
a,s
a,s
sudo rm /etc/munin/plugins/postfix_mail*
sudo rm /etc/munin/plugins/fw_packets
○
a,s
a,s
a,s
TGT=$(echo $i | cut -d/ -f7)
INI=$(echo $i | cut -d/ -f10)
LUN=$(echo $i | cut -d/ -f11)
if [ "$TGT_" = "$TGT" ]; then
if [ "$INI_" = "$INI" ]; then
:
else
INI_=$INI
INI_F=$(echo $INI | tr "[:upper:]" "[:lower:]" | sed -e 's/-/_/g' -e 's![^a-z0-9_]!!g')
fi
else
TGT_=$TGT
TGT_F=$(echo $TGT | tr "[:upper:]" "[:lower:]" | sed -e 's/-/_/g' -e 's![^a-z0-9_]!!g')
INI_=$INI
INI_F=$(echo $INI | tr "[:upper:]" "[:lower:]" | sed -e 's/-/_/g' -e 's![^a-z0-9_]!!g')
for j in $(echo /sys/kernel/config/target/iscsi/$TGT/tpgt_1/lun/*/statistics/scsi_tgt_port/write_mbytes | LANG=C sort)
do
LUN_=$(echo $j | cut -d/ -f10)
echo -n "${TGT_F}$LUN_.value "
cat $j
done
fi
echo -n "${TGT_F}${INI_F}$LUN.value "
cat $i
done
exit 0;
EOF
sudo chmod 755 /usr/share/munin/plugins/lio_write
有効化されている不要なプラグインを無効化します。
Munin の稼働状況をグラフ化するプラグインを有効化します。
cat << 'EOF' | sudo tee /etc/munin/plugin-conf.d/munin-node
[diskstats]
user munin
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 135 54140
54150
54160
54170
54180
54190
54200
54210
54220
54230
54240
54250
54260
54270
54280
54290
54300
54310
54320
54330
54340
54350
54360
54370
54380
54390
54400
54410
54420
54430
54440
54450
54460
54470
54480
54490
54500
54510
54520
54530
54540
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
○
a,s
○
a,s
a,s
[iostat_ios]
user root
[munin_*]
user munin
EOF
cat << 'EOF' | sudo tee -a /etc/munin/plugin-conf.d/munin-node
[http_loadtime]
env.target http://127.0.0.1/server-status
env.requisites true
EOF
cat << 'EOF' | sudo tee /etc/httpd/conf.d/status.conf
<IfModule mod_status.c>
ExtendedStatus On
<Location /server-status>
SetHandler server-status
Order deny,allow
Deny from all
Allow from 127.0.0.1
</Location>
</IfModule>
EOF
sudo ln -s '/usr/share/munin/plugins/apache_accesses' '/etc/munin/plugins/apache_accesses'
sudo ln -s '/usr/share/munin/plugins/apache_processes' '/etc/munin/plugins/apache_processes'
sudo ln -s '/usr/share/munin/plugins/apache_volume' '/etc/munin/plugins/apache_volume'
sudo ln -s '/usr/share/munin/plugins/http_loadtime' '/etc/munin/plugins/http_loadtime'
sudo ln -s '/usr/share/munin/plugins/munin_stats' '/etc/munin/plugins/munin_stats'
sudo ln -s '/usr/share/munin/plugins/munin_update' '/etc/munin/plugins/munin_update'
sudo ln -s '/usr/share/munin/plugins/iostat' '/etc/munin/plugins/iostat'
sudo ln -s '/usr/share/munin/plugins/iostat_ios' '/etc/munin/plugins/iostat_ios'
ネットワーク統計の詳細情報をグラフ化するプラグインを有効化します。
sudo ln -s '/usr/share/munin/plugins/netstat_multi' '/etc/munin/plugins/netstat_multi'
DRBD の稼働状況をグラフ化するプラグインを有効化します。
sudo ln -s '/usr/share/munin/plugins/drbd' '/etc/munin/plugins/drbd'
sudo ln -s '/usr/share/munin/plugins/drbd_al' '/etc/munin/plugins/drbd_al'
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 136 54550
54560
54570
54580
54590
54600
54610
54620
54630
54640
54650
54660
54670
54680
54690
54700
54710
54720
54730
54740
54750
54760
54770
54780
54790
54800
54810
54820
54830
54840
54850
54860
54870
54880
54890
54900
54910
54920
54930
54940
54950
a,s
○
a,s
a,s
○
a,s
a,s
○
a,s
a,s
a,s
○
a,s
a,s
○
a
a
a
a
○
a
○
a
○
s
sudo ln -s '/usr/share/munin/plugins/drbd_ext' '/etc/munin/plugins/drbd_ext'
Munin にホスト名を登録します。
sudo sed -i -e "s/^host_name .*\$/host_name $(uname -n)/" /etc/munin/munin-node.conf
sudo sed -i -e "s/^\\[localhost/[$(uname -n)/" /etc/munin/munin.conf
PrivateTmp 機能を無効化します。
sudo sed -i -e 's/^PrivateTmp=.*$/PrivateTmp=false/' /usr/lib/systemd/system/munin-node.service
sudo systemctl daemon-reload
ベーシック認証設定を行います。
sudo htpasswd -c -b /etc/munin/munin-htpasswd munin 'password'
sudo htpasswd -b /etc/munin/munin-htpasswd admin 'password'
sudo htpasswd -b /etc/munin/munin-htpasswd monitor 'password'
LIO の稼働状況をグラフ化するプラグインを有効化します。
sudo ln -s '/usr/share/munin/plugins/lio_read' '/etc/munin/plugins/lio_read'
sudo ln -s '/usr/share/munin/plugins/lio_write' '/etc/munin/plugins/lio_write'
Active 機で、Munin 関連サービスを自動起動するように変更し、起動します。
sudo
sudo
sudo
sudo
systemctl
systemctl
systemctl
systemctl
enable munin-node.service
enable httpd.service
start munin-node.service
start httpd.service
数十分待ってから、ブラウザでアクセスし、動作を確認します。
# http://10.110.88.57/munin
リソースをスイッチオーバします。
sudo pcs resource move g_tgt; sleep 5; sudo pcs resource clear g_tgt
Stand-by 機で、Munin 関連サービスを自動起動するように変更し、起動します。
sudo systemctl enable munin-node.service
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 137 54960
54970
54980
54990
55000
55010
55020
55030
55040
55050
55060
55070
55080
55090
55100
55110
55120
55130
55140
55150
55160
55170
55180
55190
55200
55210
55220
55230
55240
55250
55260
55270
55280
55290
55300
55310
55320
55330
55340
55350
55360
s
s
s
○
s
○
a
○
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
a,s
sudo systemctl enable httpd.service
sudo systemctl start munin-node.service
sudo systemctl start httpd.service
数十分待ってから、ブラウザでアクセスし、動作を確認します。
# http://10.110.88.58/munin
リソースをスイッチバックします。
sudo pcs resource move g_tgt; sleep 5; sudo pcs resource clear g_tgt
LIO の統計情報を定期保存する設定を行います。
sudo mkdir -p /etc/lio
sudo mkdir -p /var/log/lio/
cat << 'EOF' | sudo tee /etc/lio/save
#!/bin/sh
FILE=/dev/shm/lio-$(date +%Y%m%d%H%M)
for i in $(find /sys/kernel/config/target ! -type d | LANG=C sort)
do echo [$i]; cat $i; echo; done > $FILE 2> /dev/null
gzip $FILE
mv $FILE.gz /var/log/lio/
EOF
sudo chmod 755 /etc/lio/save
cat << 'EOF' | sudo tee /etc/lio/statistics
#!/bin/sh
FILE=/dev/shm/lio-statistics-$(date +%Y%m%d%H%M)
YYYYMMDD=$(echo $FILE | sed -e 's/^.*\([0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]\)[0-9][0-9][0-9][0-9]$/\1/')
for i in $(for k in /sys/kernel/config/target/{core/*/*/,iscsi/*/{fabric_,tpgt_1/{acls/*/{fabric_,*/},lun/*/}}}statistics; do echo $k; done | LANG=C sort)
do for j in $(find $i ! -type d | LANG=C sort); do echo [$j]; cat $j; echo; done; done > $FILE 2> /dev/null
gzip $FILE
mkdir -p /var/log/lio/$YYYYMMDD/
mv $FILE.gz /var/log/lio/$YYYYMMDD/
EOF
sudo chmod 755 /etc/lio/statistics
cat << 'EOF' | sudo tee /etc/cron.d/lio
59 * * * * root nice -n 19 /etc/lio/save
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 138 55370 a,s
55380 a,s
55390 a,s
55400
* * * * * root nice -n 19 /etc/lio/statistics
58 23 * * * root nice -n 19 /bin/find /var/log/lio -mtime +365 -print0 | xargs -0 rm -rfv 2> /dev/null
EOF
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.
http://dba-ha.blogspot.jp/ (メシの種 - DBAの落書き帳) LIO, DRBD, Pacemaker による冗長化 iSCSI Target 構築手順 - 139 -
当文書で紹介した構成で初期構築をご希望の方は、メール(mailto: si@pc-office.net)にてお問い合わせください。
リモートアクセスによる構築サービスを、弊社提示のヒアリング事項（当文書の緑色で記載した部分）以外のカスタマイズはなしとの前提で、
サーバ1台あたり10万円（税別）という超特価にてご提供中です。
1号機と2号機(Active 機と Stand-by 機)サーバの物理構成は同一構成との前提です。
カスタマイズやドキュメントの提供、個別訪問等は、別途ご相談となります。
クラスタの起動停止、スイッチオーバ(手動フェイルオーバ)以外の動作確認、結合試験、障害試験、性能試験等は、別途ご相談となります。
インストールメディアからインストールできない環境やコンソールにリモートアクセスできない環境でのインストールも別途ご相談となります。
監視設定（障害監視、リソース監視、セキュリティ監視等）については、別途ご相談となります。
UEFI 対応、ハードウェア固有のドライバや管理ソフト等のインストールについては、別途ご相談となります。
当該サーバには、消失したら困るデータは存在していない前提での作業となります。
既に動いている CentOS を置き換えるインストールの場合、ヒアリング事項を弊社で調査して提示することも可能です。
例えば、IBM Bluemix(SoftLayer) のベアメタルサーバの場合、CentOS7をあらかじめインストールした状態で弊社へお引き渡し頂ければ、
お客様の手間を省くことができます。
弊社での正式サポートは、お客様が当該サーバ用の Oracle Linux サブスクリプションを契約中か NRI OpenStandia に相談窓口をお持ちで、
代理で問い合わせを行うという前提を取らさせていただくことになります。費用はご相談ください。
サーバの調達・CE作業費用、OS 等のライセンス費用、サブスクリプション費用等はすべて別料金となります。
OS を Oracle Linux (UEK カーネル) に置き換えずに、RHEL・CentOS で構築することも可能ですが、制限事項についてご相談ください。
MySQL や PostgreSQL、Oracle の冗長構成構築サービスも鋭意開発中です。商品開発に関するリクエストがあればお知らせください。
検討させていただきます。
貴社のアプリケーション、サービス等を冗長化する共同開発も承ります。
Copyright © 2016 PC Office Co., Ltd. All Rights Reserved.

放水量の計算 Q=K×D2×√10.2P Q=放水量(L/min） K=0.653 Q= (L

みなさんこんにちは

ハンズオン -‐‑‒ 準備編

PowerPoint プレゼンテーション

drbd_proxy_wan

～SIESTAのインストール方法～

耐震診断義務付け対象建築物の耐震診断の結果の公表について

drbd-users-guide-ja-8.4