Oracle 11gR2 RAC 添加节点最佳实践
OS信息:
[grid@11grac1 ~]$ uname -a
Linux 11grac1 2.6.32-300.10.1.el5uek #1 SMP Wed Feb 22 17:22:40 EST 2012 i686 i686 i386 GNU/Linux
DB信息:
SQL> select * from v$version where rownumBANNER
------------------------------------------------------------------------
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - Production
PL/SQL Release 11.2.0.1.0 - Production
CORE 11.2.0.1.0 Production
TNS for Linux: Version 11.2.0.1.0 - Production
（一）设置网络
（二）添加组、目录和用户
（三）修改相关OS参数
（四）修改用户环境变量
（五）设置存储
（六）配置节点互信
（七）安装RPM包
注：1-7步略(其中的修改需要参照其他节点的配置)
（八）安装GI前检测
[grid@11grac1 bin]$ pwd
/u01/11.2.0/grid/bin
[grid@11grac1 bin]$ ./cluvfy stage -pre nodeadd -n 11grac2 -fixup -verbose
Performing pre-checks for node addition
Checking node reachability...
Check: Node reachability from node "11grac1"
Destination Node Reachable?
------------------------------------ ------------------------
11grac2 yes
Result: Node reachability check passed from node "11grac1"
Checking user equivalence...
Check: User equivalence for user "grid"
Node Name Comment
------------------------------------ ------------------------
11grac2 passed
Result: User equivalence check passed for user "grid"
Checking node connectivity...
Checking hosts config file...
Node Name Status Comment
------------ ------------------------ ------------------------
11grac2 passed
11grac1 passed
Verification of the hosts config file successful
Interface information for node "11grac1"
Name IP Address Subnet Gateway Def. Gateway HW Address MTU
------ --------------- --------------- --------------- --------------- ----------------- ------
eth0 192.168.137.161 192.168.137.0 0.0.0.0 192.168.137.1 00:0C:29:31:BE:BA 1500
eth0 192.168.137.163 192.168.137.0 0.0.0.0 192.168.137.1 00:0C:29:31:BE:BA 1500
eth0 192.168.137.200 192.168.137.0 0.0.0.0 192.168.137.1 00:0C:29:31:BE:BA 1500
eth1 192.168.136.161 192.168.136.0 0.0.0.0 192.168.137.1 00:0C:29:31:BE:C4 1500
Interface information for node "11grac2"
Name IP Address Subnet Gateway Def. Gateway HW Address MTU
------ --------------- --------------- --------------- --------------- ----------------- ------
eth0 192.168.137.162 192.168.137.0 0.0.0.0 192.168.137.1 00:0C:29:D6:B6:09 1500
eth1 192.168.136.162 192.168.136.0 0.0.0.0 192.168.137.1 00:0C:29:D6:B6:13 1500
Check: Node connectivity for interface "eth0"
Source Destination Connected?
------------------------------ ------------------------------ ----------------
11grac1:eth0 11grac1:eth0 yes
11grac1:eth0 11grac1:eth0 yes
11grac1:eth0 11grac2:eth0 yes
11grac1:eth0 11grac1:eth0 yes
11grac1:eth0 11grac2:eth0 yes
11grac1:eth0 11grac2:eth0 yes
Result: Node connectivity passed for interface "eth0"
Result: Node connectivity check passed
Checking CRS integrity...
The Oracle clusterware is healthy on node "11grac1"
CRS integrity check passed
Checking shared resources...
Checking CRS home location...
The location "/u01/11.2.0/grid" is not shared but is present/creatable on all nodes
Result: Shared resources check for node addition passed
Checking node connectivity...
Checking hosts config file...
Node Name Status Comment
------------ ------------------------ ------------------------
11grac2 passed
11grac1 passed
Verification of the hosts config file successful
Interface information for node "11grac1"
Name IP Address Subnet Gateway Def. Gateway HW Address MTU
------ --------------- --------------- --------------- --------------- ----------------- ------
eth0 192.168.137.161 192.168.137.0 0.0.0.0 192.168.137.1 00:0C:29:31:BE:BA 1500
eth0 192.168.137.163 192.168.137.0 0.0.0.0 192.168.137.1 00:0C:29:31:BE:BA 1500
eth0 192.168.137.200 192.168.137.0 0.0.0.0 192.168.137.1 00:0C:29:31:BE:BA 1500
eth1 192.168.136.161 192.168.136.0 0.0.0.0 192.168.137.1 00:0C:29:31:BE:C4 1500
Interface information for node "11grac2"
Name IP Address Subnet Gateway Def. Gateway HW Address MTU
------ --------------- --------------- --------------- --------------- ----------------- ------
eth0 192.168.137.162 192.168.137.0 0.0.0.0 192.168.137.1 00:0C:29:D6:B6:09 1500
eth1 192.168.136.162 192.168.136.0 0.0.0.0 192.168.137.1 00:0C:29:D6:B6:13 1500
Check: Node connectivity of subnet "192.168.137.0"
Source Destination Connected?
------------------------------ ------------------------------ ----------------
11grac1:eth0 11grac1:eth0 yes
11grac1:eth0 11grac1:eth0 yes
11grac1:eth0 11grac2:eth0 yes
11grac1:eth0 11grac1:eth0 yes
11grac1:eth0 11grac2:eth0 yes
11grac1:eth0 11grac2:eth0 yes
Result: Node connectivity passed for subnet "192.168.137.0" with node(s) 11grac1,11grac2
Check: TCP connectivity of subnet "192.168.137.0"
Source Destination Connected?
------------------------------ ------------------------------ ----------------
11grac1:192.168.137.161 11grac1:192.168.137.163 passed
11grac1:192.168.137.161 11grac1:192.168.137.200 passed
11grac1:192.168.137.161 11grac2:192.168.137.162 passed
Result: TCP connectivity check passed for subnet "192.168.137.0"
Check: Node connectivity of subnet "192.168.136.0"
Source Destination Connected?
------------------------------ ------------------------------ ----------------
11grac1:eth1 11grac2:eth1 yes
Result: Node connectivity passed for subnet "192.168.136.0" with node(s) 11grac1,11grac2
Check: TCP connectivity of subnet "192.168.136.0"
Source Destination Connected?
------------------------------ ------------------------------ ----------------
11grac1:192.168.136.161 11grac2:192.168.136.162 passed
Result: TCP connectivity check passed for subnet "192.168.136.0"
Interfaces found on subnet "192.168.137.0" that are likely candidates for VIP are:
11grac1 eth0:192.168.137.161 eth0:192.168.137.163 eth0:192.168.137.200
11grac2 eth0:192.168.137.162
Interfaces found on subnet "192.168.136.0" that are likely candidates for a private interconnect are:
11grac1 eth1:192.168.136.161
11grac2 eth1:192.168.136.162
Result: Node connectivity check passed
Check: Total memory
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
11grac2 1.97GB (2061864.0KB) 1.5GB (1572864.0KB) passed
Result: Total memory check passed
Check: Available memory
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
11grac2 1.88GB (1970204.0KB) 50MB (51200.0KB) passed
Result: Available memory check passed
Check: Swap space
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
11grac2 4GB (4192956.0KB) 2.95GB (3092796.0KB) passed
Result: Swap space check passed
Check: Free disk space for "11grac2:/tmp"
Path Node Name Mount point Available Required Comment
---------------- ------------ ------------ ------------ ------------ ------------
/tmp 11grac2 / 15.54GB 1GB passed
Result: Free disk space check passed for "11grac2:/tmp"
Check: User existence for "grid"
Node Name Status Comment
------------ ------------------------ ------------------------
11grac2 exists passed
Result: User existence check passed for "grid"
Check: Run level
Node Name run level Required Comment
------------ ------------------------ ------------------------ ----------
11grac2 3 3,5 passed
Result: Run level check passed
Check: Hard limits for "maximum open file descriptors"
Node Name Type Available Required Comment
---------------- ------------ ------------ ------------ ----------------
11grac2 hard 65536 65536 passed
Result: Hard limits check passed for "maximum open file descriptors"
Check: Soft limits for "maximum open file descriptors"
Node Name Type Available Required Comment
---------------- ------------ ------------ ------------ ----------------
11grac2 soft 1024 1024 passed
Result: Soft limits check passed for "maximum open file descriptors"
Check: Hard limits for "maximum user processes"
Node Name Type Available Required Comment
---------------- ------------ ------------ ------------ ----------------
11grac2 hard 16384 16384 passed
Result: Hard limits check passed for "maximum user processes"
Check: Soft limits for "maximum user processes"
Node Name Type Available Required Comment
---------------- ------------ ------------ ------------ ----------------
11grac2 soft 2047 2047 passed
Result: Soft limits check passed for "maximum user processes"
Check: System architecture
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
11grac2 i686 x86 passed
Result: System architecture check passed
Check: Kernel version
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
11grac2 2.6.32-300.10.1.el5uek 2.6.18 passed
Result: Kernel version check passed
Check: Kernel parameter for "semmsl"
Node Name Configured Required Comment
------------ ------------------------ ------------------------ ----------
11grac2 250 250 passed
Result: Kernel parameter check passed for "semmsl"
Check: Kernel parameter for "semmns"
Node Name Configured Required Comment
------------ ------------------------ ------------------------ ----------
11grac2 32000 32000 passed
Result: Kernel parameter check passed for "semmns"
Check: Kernel parameter for "semopm"
Node Name Configured Required Comment
------------ ------------------------ ------------------------ ----------
11grac2 100 100 passed
Result: Kernel parameter check passed for "semopm"
Check: Kernel parameter for "semmni"
Node Name Configured Required Comment
------------ ------------------------ ------------------------ ----------
11grac2 128 128 passed
Result: Kernel parameter check passed for "semmni"
Check: Kernel parameter for "shmmax"
Node Name Configured Required Comment
------------ ------------------------ ------------------------ ----------
11grac2 1073741824 536870912 passed
Result: Kernel parameter check passed for "shmmax"
Check: Kernel parameter for "shmmni"
Node Name Configured Required Comment
------------ ------------------------ ------------------------ ----------
11grac2 4096 4096 passed
Result: Kernel parameter check passed for "shmmni"
Check: Kernel parameter for "shmall"
Node Name Configured Required Comment
------------ ------------------------ ------------------------ ----------
11grac2 8388608 2097152 passed
Result: Kernel parameter check passed for "shmall"
Check: Kernel parameter for "file-max"
Node Name Configured Required Comment
------------ ------------------------ ------------------------ ----------
11grac2 6815744 6815744 passed
Result: Kernel parameter check passed for "file-max"
Check: Kernel parameter for "ip_local_port_range"
Node Name Configured Required Comment
------------ ------------------------ ------------------------ ----------
11grac2 between 9000 & 65500 between 9000 & 65500 passed
Result: Kernel parameter check passed for "ip_local_port_range"
Check: Kernel parameter for "rmem_default"
Node Name Configured Required Comment
------------ ------------------------ ------------------------ ----------
11grac2 262144 262144 passed
Result: Kernel parameter check passed for "rmem_default"
Check: Kernel parameter for "rmem_max"
Node Name Configured Required Comment
------------ ------------------------ ------------------------ ----------
11grac2 4194304 4194304 passed
Result: Kernel parameter check passed for "rmem_max"
Check: Kernel parameter for "wmem_default"
Node Name Configured Required Comment
------------ ------------------------ ------------------------ ----------
11grac2 262144 262144 passed
Result: Kernel parameter check passed for "wmem_default"
Check: Kernel parameter for "wmem_max"
Node Name Configured Required Comment
------------ ------------------------ ------------------------ ----------
11grac2 1048586 1048576 passed
Result: Kernel parameter check passed for "wmem_max"
Check: Kernel parameter for "aio-max-nr"
Node Name Configured Required Comment
------------ ------------------------ ------------------------ ----------
11grac2 1048576 1048576 passed
Result: Kernel parameter check passed for "aio-max-nr"
Check: Package existence for "make-3.81"
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
11grac2 make-3.81-3.el5 make-3.81 passed
Result: Package existence check passed for "make-3.81"
Check: Package existence for "binutils-2.17.50.0.6"
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
11grac2 binutils-2.17.50.0.6-20.el5 binutils-2.17.50.0.6 passed
Result: Package existence check passed for "binutils-2.17.50.0.6"
Check: Package existence for "gcc-4.1.2"
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
11grac2 gcc-4.1.2-52.el5 gcc-4.1.2 passed
Result: Package existence check passed for "gcc-4.1.2"
Check: Package existence for "gcc-c++-4.1.2"
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
11grac2 gcc-c++-4.1.2-52.el5 gcc-c++-4.1.2 passed
Result: Package existence check passed for "gcc-c++-4.1.2"
Check: Package existence for "libgomp-4.1.2"
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
11grac2 libgomp-4.4.6-3.el5.1 libgomp-4.1.2 passed
Result: Package existence check passed for "libgomp-4.1.2"
Check: Package existence for "libaio-0.3.106"
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
11grac2 libaio-0.3.106-5 libaio-0.3.106 passed
Result: Package existence check passed for "libaio-0.3.106"
Check: Package existence for "glibc-2.5-24"
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
11grac2 glibc-2.5-81 glibc-2.5-24 passed
Result: Package existence check passed for "glibc-2.5-24"
Check: Package existence for "compat-libstdc++-33-3.2.3"
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
11grac2 compat-libstdc++-33-3.2.3-61 compat-libstdc++-33-3.2.3 passed
Result: Package existence check passed for "compat-libstdc++-33-3.2.3"
Check: Package existence for "elfutils-libelf-0.125"
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
11grac2 elfutils-libelf-0.137-3.el5 elfutils-libelf-0.125 passed
Result: Package existence check passed for "elfutils-libelf-0.125"
Check: Package existence for "elfutils-libelf-devel-0.125"
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
11grac2 elfutils-libelf-devel-0.137-3.el5 elfutils-libelf-devel-0.125 passed
Result: Package existence check passed for "elfutils-libelf-devel-0.125"
Check: Package existence for "glibc-common-2.5"
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
11grac2 glibc-common-2.5-81 glibc-common-2.5 passed
Result: Package existence check passed for "glibc-common-2.5"
Check: Package existence for "glibc-devel-2.5"
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
11grac2 glibc-devel-2.5-81 glibc-devel-2.5 passed
Result: Package existence check passed for "glibc-devel-2.5"
Check: Package existence for "glibc-headers-2.5"
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
11grac2 glibc-headers-2.5-81 glibc-headers-2.5 passed
Result: Package existence check passed for "glibc-headers-2.5"
Check: Package existence for "libaio-devel-0.3.106"
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
11grac2 libaio-devel-0.3.106-5 libaio-devel-0.3.106 passed
Result: Package existence check passed for "libaio-devel-0.3.106"
Check: Package existence for "libgcc-4.1.2"
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
11grac2 libgcc-4.1.2-52.el5 libgcc-4.1.2 passed
Result: Package existence check passed for "libgcc-4.1.2"
Check: Package existence for "libstdc++-4.1.2"
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
11grac2 libstdc++-4.1.2-52.el5 libstdc++-4.1.2 passed
Result: Package existence check passed for "libstdc++-4.1.2"
Check: Package existence for "libstdc++-devel-4.1.2"
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
11grac2 libstdc++-devel-4.1.2-52.el5 libstdc++-devel-4.1.2 passed
Result: Package existence check passed for "libstdc++-devel-4.1.2"
Check: Package existence for "sysstat-7.0.2"
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
11grac2 sysstat-7.0.2-11.el5 sysstat-7.0.2 passed
Result: Package existence check passed for "sysstat-7.0.2"
Check: Package existence for "unixODBC-2.2.11"
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
11grac2 unixODBC-2.2.11-10.el5 unixODBC-2.2.11 passed
Result: Package existence check passed for "unixODBC-2.2.11"
Check: Package existence for "unixODBC-devel-2.2.11"
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
11grac2 unixODBC-devel-2.2.11-10.el5 unixODBC-devel-2.2.11 passed
Result: Package existence check passed for "unixODBC-devel-2.2.11"
Check: Package existence for "ksh-20060214"
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
11grac2 ksh-20100621-5.el5 ksh-20060214 passed
Result: Package existence check passed for "ksh-20060214"
Checking for multiple users with UID value 0
Result: Check for multiple users with UID value 0 passed
Checking to make sure user "grid" is not in "root" group
Node Name Status Comment
------------ ------------------------ ------------------------
11grac2 does not exist passed
Result: User "grid" is not part of "root" group. Check passed
Starting Clock synchronization checks using Network Time Protocol(NTP)...
NTP Configuration file check started...
Network Time Protocol(NTP) configuration file not found on any of the nodes. Oracle Cluster Time Synchronization Service(CTSS) can be used instead of NTP for time synchronization on the cluster nodes
Result: Clock synchronization check using Network Time Protocol(NTP) passed
Pre-check for node addition was successful.
[grid@11grac1 bin]$