当前位置:  数据库>oracle

Oracle RAC 中vip网关引起错误的解决

    来源: 互联网  发布时间:2017-06-11

    本文导语: 在虚拟机上刚装完Oracle RAC,重新启动系统后。重新启动了oracle所有的服务,一开始所有的服务都正常。 但是过了一会,节点2的vip又再次飘移到了节点1上,并且节点2的监听服务也OFFLINE了。反复启动了几次都是同样的效果的。[ro...

在虚拟机上刚装完Oracle RAC,重新启动系统后。重新启动了oracle所有的服务,一开始所有的服务都正常。
但是过了一会,节点2的vip又再次飘移到了节点1上,并且节点2的监听服务也OFFLINE了。反复启动了几次都是同样的效果的。
[root@node2 bin]# ./crs_stat -t
 Name          Type          Target    State    Host       
 ------------------------------------------------------------
 ora.devdb.db  application    ONLINE    ONLINE    node2     
 ora....b1.inst application    ONLINE    ONLINE    node1     
 ora....b2.inst application    ONLINE    ONLINE    node2     
 ora....SM1.asm application    ONLINE    ONLINE    node1     
 ora....E1.lsnr application    ONLINE    ONLINE    node1     
 ora.node1.gsd  application    ONLINE    ONLINE    node1     
 ora.node1.ons  application    ONLINE    ONLINE    node1     
 ora.node1.vip  application    ONLINE    ONLINE    node1     
 ora....SM2.asm application    ONLINE    ONLINE    node2     
ora....E2.lsnr application    ONLINE    OFFLINE         
 ora.node2.gsd  application    ONLINE    ONLINE    node2     
 ora.node2.ons  application    ONLINE    ONLINE    node2     
ora.node2.vip  application    ONLINE    ONLINE    node1     

节点2上的监听没有起,并且节点2的vip起在节点1上。之前监听器是在节点1上建立的一个集群监听。
 尝试将节点2的vip停止下来再重新启动,发现只能在节点1上启动。
/u01/app/oracle/crs/log/node2/crsd/crsd.log的日志报错:
2014-11-03 13:41:33.386: [ CRSRES][2659736480]0Attempting to start `ora.node2.vip` on member `node2`
 2014-11-03 13:41:46.720: [ CRSAPP][2659736480]0StartResource error for ora.rac10g2.vip error code = 1
 2014-11-03 13:41:51.405: [ CRSRES][2659736480]0Start of `ora.node2.vip` on member `node2` failed.
 2014-11-03 13:41:51.532: [ CRSRES][2659736480]0Attempting to start `ora.node2.vip` on member `node1`
 2014-11-03 13:41:55.388: [ CRSRES][2659736480]0Start of `ora.node2.vip` on member `node1` succeeded.
将所有服务都停止,然后一一重启就好了。
[root@node2 bin]# ./crs_stat -t
 Name          Type          Target    State    Host       
 ------------------------------------------------------------
 ora.devdb.db  application    ONLINE    ONLINE    node2     
 ora....b1.inst application    ONLINE    ONLINE    node1     
 ora....b2.inst application    ONLINE    ONLINE    node2     
 ora....SM1.asm application    ONLINE    ONLINE    node1     
 ora....E1.lsnr application    ONLINE    ONLINE    node1     
 ora.node1.gsd  application    ONLINE    ONLINE    node1     
 ora.node1.ons  application    ONLINE    ONLINE    node1     
 ora.node1.vip  application    ONLINE    ONLINE    node1     
 ora....SM2.asm application    ONLINE    ONLINE    node2     
 ora....E2.lsnr application    ONLINE    ONLINE    node2     
 ora.node2.gsd  application    ONLINE    ONLINE    node2     
 ora.node2.ons  application    ONLINE    ONLINE    node2     
 ora.node2.vip  application    ONLINE    ONLINE    node2     

但是过了一会,节点2的vip又再次飘移到了节点1上,并且节点2的监听服务也OFFLINE了。

在/u01/app/oracle/crs/log/node2/racg下的ora.node2.vip.log日志中发现如下报错:


[root@node2 bin]# cat /u01/app/oracle/product/crs/log/node2/racg/ora.node2.vip.log
 Oracle Database 10g CRS Release 10.2.0.1.0 Production Copyright 1996, 2005 Oracle.  All rights reserved.
 2014-11-03 09:58:20.864: [    RACG][3067430592] [8114][3067430592][ora.node2.vip]: Interface eth0 checked failed (host=node2)
 Invalid parameters, or failed to bring up VIP (host=node2)

2014-11-03 09:58:20.864: [    RACG][3067430592] [8114][3067430592][ora.node2.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/u01/app/oracle/product/crs

2014-11-03 09:58:20.864: [    RACG][3067430592] [8114][3067430592][ora.node2.vip]: clsrcexecut: cmd = /u01/app/oracle/product/crs/bin/racgeut -e _USR_ORA_DEBUG=0 54 /u01/app/oracle/product/crs/bin/racgvip check node2

2014-11-03 09:58:20.864: [    RACG][3067430592] [8114][3067430592][ora.node2.vip]: clsrcexecut: rc = 1, time = 6.330s

2014-11-03 09:58:20.864: [    RACG][3067430592] [8114][3067430592][ora.node2.vip]: end for resource = ora.node2.vip, action = check, status = 1, time = 6.340s

2014-11-03 10:01:36.674: [    RACG][3068290752] [12884][3068290752][ora.node2.vip]: IP:143.168.0.15 is already up in the network (host=node2)

2014-11-03 10:01:36.674: [    RACG][3068290752] [12884][3068290752][ora.node2.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/u01/app/oracle/product/crs

2014-11-03 10:01:36.674: [    RACG][3068290752] [12884][3068290752][ora.node2.vip]: clsrcexecut: cmd = /u01/app/oracle/product/crs/bin/racgeut -e _USR_ORA_DEBUG=0 54 /u01/app/oracle/product/crs/bin/racgvip start node2

2014-11-03 10:01:36.674: [    RACG][3068290752] [12884][3068290752][ora.node2.vip]: clsrcexecut: rc = 1, time = 2.240s

2014-11-03 10:01:38.818: [    RACG][3068290752] [12884][3068290752][ora.node2.vip]: IP:143.168.0.15 is already up in the network (host=node2)

2014-11-03 10:01:38.818: [    RACG][3068290752] [12884][3068290752][ora.node2.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/u01/app/oracle/product/crs

2014-11-03 10:01:38.818: [    RACG][3068290752] [12884][3068290752][ora.node2.vip]: clsrcexecut: cmd = /u01/app/oracle/product/crs/bin/racgeut -e _USR_ORA_DEBUG=0 54 /u01/app/oracle/product/crs/bin/racgvip check node2

2014-11-03 10:01:38.818: [    RACG][3068290752] [12884][3068290752][ora.node2.vip]: clsrcexecut: rc = 1, time = 2.150s

2014-11-03 10:01:38.818: [    RACG][3068290752] [12884][3068290752][ora.node2.vip]: end for resource = ora.node2.vip, action = start, status = 1, time = 4.530s

2014-11-03 10:01:41.418: [    RACG][3068180160] [13098][3068180160][ora.node2.vip]: bounce eth0 (host=node2)

2014-11-03 10:05:41.296: [    RACG][3067840192] [18850][3067840192][ora.node2.vip]: IP:143.168.0.15 is already up in the network (host=node2)

2014-11-03 10:05:41.296: [    RACG][3067840192] [18850][3067840192][ora.node2.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/u01/app/oracle/product/crs

2014-11-03 10:05:41.296: [    RACG][3067840192] [18850][3067840192][ora.node2.vip]: clsrcexecut: cmd = /u01/app/oracle/product/crs/bin/racgeut -e _USR_ORA_DEBUG=0 54 /u01/app/oracle/product/crs/bin/racgvip start node2

2014-11-03 10:05:41.296: [    RACG][3067840192] [18850][3067840192][ora.node2.vip]: clsrcexecut: rc = 1, time = 2.150s

2014-11-03 10:05:43.434: [    RACG][3067840192] [18850][3067840192][ora.node2.vip]: IP:143.168.0.15 is already up in the network (host=node2)

2014-11-03 10:05:43.434: [    RACG][3067840192] [18850][3067840192][ora.node2.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/u01/app/oracle/product/crs

2014-11-03 10:05:43.434: [    RACG][3067840192] [18850][3067840192][ora.node2.vip]: clsrcexecut: cmd = /u01/app/oracle/product/crs/bin/racgeut -e _USR_ORA_DEBUG=0 54 /u01/app/oracle/product/crs/bin/racgvip check node2

2014-11-03 10:05:43.434: [    RACG][3067840192] [18850][3067840192][ora.node2.vip]: clsrcexecut: rc = 1, time = 2.140s

2014-11-03 10:05:43.434: [    RACG][3067840192] [18850][3067840192][ora.node2.vip]: end for resource = ora.node2.vip, action = start, status = 1, time = 4.350s

2014-11-03 10:05:45.791: [    RACG][3067639488] [19001][3067639488][ora.node2.vip]: bounce eth0 (host=node2)

2014-11-03 10:53:28.997: [    RACG][3067741888] [2298][3067741888][ora.node2.vip]: Interface eth0 checked failed (host=node2)
 Invalid parameters, or failed to bring up VIP (host=node2)

2014-11-03 10:53:28.997: [    RACG][3067741888] [2298][3067741888][ora.node2.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/u01/app/oracle/product/crs

2014-11-03 10:53:28.997: [    RACG][3067741888] [2298][3067741888][ora.node2.vip]: clsrcexecut: cmd = /u01/app/oracle/product/crs/bin/racgeut -e _USR_ORA_DEBUG=0 54 /u01/app/oracle/product/crs/bin/racgvip check node2

2014-11-03 10:53:28.997: [    RACG][3067741888] [2298][3067741888][ora.node2.vip]: clsrcexecut: rc = 1, time = 6.390s

2014-11-03 10:53:28.997: [    RACG][3067741888] [2298][3067741888][ora.node2.vip]: end for resource = ora.node2.vip, action = check, status = 1, time = 9.930s

2014-11-03 10:56:47.744: [    RACG][3067373248] [7351][3067373248][ora.node2.vip]: Interface eth0 checked failed (host=node2)
 Invalid parameters, or failed to bring up VIP (host=node2)

2014-11-03 10:56:47.745: [    RACG][3067373248] [7351][3067373248][ora.node2.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/u01/app/oracle/product/crs

2014-11-03 10:56:47.745: [    RACG][3067373248] [7351][3067373248][ora.node2.vip]: clsrcexecut: cmd = /u01/app/oracle/product/crs/bin/racgeut -e _USR_ORA_DEBUG=0 54 /u01/app/oracle/product/crs/bin/racgvip check node2

2014-11-03 10:56:47.745: [    RACG][3067373248] [7351][3067373248][ora.node2.vip]: clsrcexecut: rc = 1, time = 6.320s

2014-11-03 10:56:47.745: [    RACG][3067373248] [7351][3067373248][ora.node2.vip]: end for resource = ora.node2.vip, action = check, status = 1, time = 6.340s

2014-11-03 10:58:35.730: [    RACG][3067692736] [9859][3067692736][ora.node2.vip]: Interface eth0 checked failed (host=node2)
 Invalid parameters, or failed to bring up VIP (host=node2)

2014-11-03 10:58:35.730: [    RACG][3067692736] [9859][3067692736][ora.node2.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/u01/app/oracle/product/crs

2014-11-03 10:58:35.730: [    RACG][3067692736] [9859][3067692736][ora.node2.vip]: clsrcexecut: cmd = /u01/app/oracle/product/crs/bin/racgeut -e _USR_ORA_DEBUG=0 54 /u01/app/oracle/product/crs/bin/racgvip start node2

2014-11-03 10:58:35.730: [    RACG][3067692736] [9859][3067692736][ora.node2.vip]: clsrcexecut: rc = 1, time = 9.240s

2014-11-03 10:58:42.058: [    RACG][3067692736] [9859][3067692736][ora.node2.vip]: Interface eth0 checked failed (host=node2)
 Invalid parameters, or failed to bring up VIP (host=node2)

2014-11-03 10:58:42.058: [    RACG][3067692736] [9859][3067692736][ora.node2.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/u01/app/oracle/product/crs

2014-11-03 10:58:42.058: [    RACG][3067692736] [9859][3067692736][ora.node2.vip]: clsrcexecut: cmd = /u01/app/oracle/product/crs/bin/racgeut -e _USR_ORA_DEBUG=0 54 /u01/app/oracle/product/crs/bin/racgvip check node2

2014-11-03 10:58:42.058: [    RACG][3067692736] [9859][3067692736][ora.node2.vip]: clsrcexecut: rc = 1, time = 6.330s

2014-11-03 10:58:42.058: [    RACG][3067692736] [9859][3067692736][ora.node2.vip]: end for resource = ora.node2.vip, action = start, status = 1, time = 15.590s

2014-11-03 11:01:20.158: [  OCRMSG][3068126912]prom_rpc: CLSC send failure..ret code 11
 2014-11-03 11:01:20.187: [  OCRMSG][3068126912]prom_rpc: possible OCR retry scenario
 2014-11-03 11:01:21.188: [  OCRAPI][3068126912]procr_open: Node Failure. Attempting retry #0
 2014-11-03 11:01:21.380: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 2014-11-03 11:01:22.382: [  OCRAPI][3068126912]procr_open: Node Failure. Attempting retry #1
 2014-11-03 11:01:22.567: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 2014-11-03 11:01:23.569: [  OCRAPI][3068126912]procr_open: Node Failure. Attempting retry #2
 2014-11-03 11:01:23.757: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 2014-11-03 11:01:24.758: [  OCRAPI][3068126912]procr_open: Node Failure. Attempting retry #3
 2014-11-03 11:01:24.946: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 2014-11-03 11:01:25.947: [  OCRAPI][3068126912]procr_open: Node Failure. Attempting retry #4
 2014-11-03 11:01:26.138: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 2014-11-03 11:01:27.139: [  OCRAPI][3068126912]procr_open: Node Failure. Attempting retry #5
 2014-11-03 11:01:27.327: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 2014-11-03 11:01:28.328: [  OCRAPI][3068126912]procr_open: Node Failure. Attempting retry #6
 2014-11-03 11:01:28.523: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 2014-11-03 11:01:29.525: [  OCRAPI][3068126912]procr_open: Node Failure. Attempting retry #7
 2014-11-03 11:01:29.708: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 2014-11-03 11:01:30.709: [  OCRAPI][3068126912]procr_open: Node Failure. Attempting retry #8
 2014-11-03 11:01:30.893: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 2014-11-03 11:01:31.894: [  OCRAPI][3068126912]procr_open: Node Failure. Attempting retry #9
 2014-11-03 11:01:32.080: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 2014-11-03 11:01:33.081: [  OCRAPI][3068126912]procr_open: Node Failure. Attempting retry #10
 2014-11-03 11:01:33.269: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 2014-11-03 11:01:34.271: [  OCRAPI][3068126912]procr_open: Node Failure. Attempting retry #11
 2014-11-03 11:01:34.459: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 2014-11-03 11:01:35.461: [  OCRAPI][3068126912]procr_open: Node Failure. Attempting retry #12
 2014-11-03 11:01:35.647: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 2014-11-03 11:01:36.649: [  OCRAPI][3068126912]procr_open: Node Failure. Attempting retry #13
 2014-11-03 11:01:36.836: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 2014-11-03 11:01:37.837: [  OCRAPI][3068126912]procr_open: Node Failure. Attempting retry #14
 2014-11-03 11:01:38.024: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 [  OCRAPI][3068126912]a_get_last_error: Buf too small (0, 95)
 2014-11-03 11:01:38.024: [    RACG][3068126912] [13698][3068126912][ora.node2.vip]: clsrccssgetnodename: clssnsqname() failed. rc=21

2014-11-03 11:01:38.216: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 2014-11-03 11:01:39.217: [  OCRAPI][3068126912]procr_open: Node Failure. Attempting retry #0
 2014-11-03 11:01:39.399: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 2014-11-03 11:01:40.400: [  OCRAPI][3068126912]procr_open: Node Failure. Attempting retry #1
 2014-11-03 11:01:40.584: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 2014-11-03 11:01:41.587: [  OCRAPI][3068126912]procr_open: Node Failure. Attempting retry #2
 2014-11-03 11:01:41.779: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 2014-11-03 11:01:42.780: [  OCRAPI][3068126912]procr_open: Node Failure. Attempting retry #3
 2014-11-03 11:01:42.970: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 2014-11-03 11:01:43.972: [  OCRAPI][3068126912]procr_open: Node Failure. Attempting retry #4
 2014-11-03 11:01:44.159: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 2014-11-03 11:01:45.162: [  OCRAPI][3068126912]procr_open: Node Failure. Attempting retry #5
 2014-11-03 11:01:45.347: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 2014-11-03 11:01:46.348: [  OCRAPI][3068126912]procr_open: Node Failure. Attempting retry #6
 2014-11-03 11:01:46.541: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 2014-11-03 11:01:47.542: [  OCRAPI][3068126912]procr_open: Node Failure. Attempting retry #7
 2014-11-03 11:01:47.729: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 2014-11-03 11:01:48.731: [  OCRAPI][3068126912]procr_open: Node Failure. Attempting retry #8
 2014-11-03 11:01:48.919: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 2014-11-03 11:01:49.920: [  OCRAPI][3068126912]procr_open: Node Failure. Attempting retry #9
 2014-11-03 11:01:50.103: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 2014-11-03 11:01:51.107: [  OCRAPI][3068126912]procr_open: Node Failure. Attempting retry #10
 2014-11-03 11:01:51.294: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 2014-11-03 11:01:52.295: [  OCRAPI][3068126912]procr_open: Node Failure. Attempting retry #11
 2014-11-03 11:01:52.488: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 2014-11-03 11:01:53.489: [  OCRAPI][3068126912]procr_open: Node Failure. Attempting retry #12
 2014-11-03 11:01:53.676: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 2014-11-03 11:01:54.677: [  OCRAPI][3068126912]procr_open: Node Failure. Attempting retry #13
 2014-11-03 11:01:54.865: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 2014-11-03 11:01:55.867: [  OCRAPI][3068126912]procr_open: Node Failure. Attempting retry #14
 2014-11-03 11:01:56.050: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 [  OCRAPI][3068126912]a_get_last_error: Buf too small (0, 95)
 2014-11-03 11:01:56.051: [    RACG][3068126912] [13698][3068126912][ora.node2.vip]: clsrccssgetnodename: clssnsqname() failed. rc=21

2014-11-03 11:01:56.051: [    RACG][3068126912] [13698][3068126912][ora.node2.vip]: clsrcep: clsrcgetnodename error: leaving node name null

2014-11-03 11:01:56.237: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 2014-11-03 11:01:57.239: [  OCRAPI][3068126912]procr_open: Node Failure. Attempting retry #0
 2014-11-03 11:01:57.434: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 2014-11-03 11:01:58.436: [  OCRAPI][3068126912]procr_open: Node Failure. Attempting retry #1
 2014-11-03 11:01:58.629: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 2014-11-03 11:01:59.630: [  OCRAPI][3068126912]procr_open: Node Failure. Attempting retry #2
 2014-11-03 11:01:59.817: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 2014-11-03 11:02:00.819: [  OCRAPI][3068126912]procr_open: Node Failure. Attempting retry #3
 2014-11-03 11:02:01.006: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 2014-11-03 11:02:02.008: [  OCRAPI][3068126912]procr_open: Node Failure. Attempting retry #4
 2014-11-03 11:02:02.192: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 2014-11-03 11:02:03.193: [  OCRAPI][3068126912]procr_open: Node Failure. Attempting retry #5
 2014-11-03 11:02:03.402: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 2014-11-03 11:02:04.403: [  OCRAPI][3068126912]procr_open: Node Failure. Attempting retry #6
 2014-11-03 11:02:04.587: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 2014-11-03 11:02:05.589: [  OCRAPI][3068126912]procr_open: Node Failure. Attempting retry #7
 2014-11-03 11:02:05.775: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 2014-11-03 11:02:06.777: [  OCRAPI][3068126912]procr_open: Node Failure. Attempting retry #8
 2014-11-03 11:02:06.965: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 2014-11-03 11:02:07.966: [  OCRAPI][3068126912]procr_open: Node Failure. Attempting retry #9
 2014-11-03 11:02:08.155: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 2014-11-03 11:02:09.156: [  OCRAPI][3068126912]procr_open: Node Failure. Attempting retry #10
 2014-11-03 11:02:09.348: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 2014-11-03 11:02:10.349: [  OCRAPI][3068126912]procr_open: Node Failure. Attempting retry #11
 2014-11-03 11:02:10.535: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 2014-11-03 11:02:11.536: [  OCRAPI][3068126912]procr_open: Node Failure. Attempting retry #12
 2014-11-03 11:02:11.722: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 2014-11-03 11:02:12.722: [  OCRAPI][3068126912]procr_open: Node Failure. Attempting retry #13
 2014-11-03 11:02:12.908: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 2014-11-03 11:02:13.909: [  OCRAPI][3068126912]procr_open: Node Failure. Attempting retry #14
 2014-11-03 11:02:14.096: [  OCRCLI][3068126912]oac_reconnect_server: Could not connect to server. clsc ret 9
 [  OCRAPI][3068126912]a_get_last_error: Buf too small (0, 95)
 2014-11-03 11:02:14.096: [    RACG][3068126912] [13698][3068126912][ora.node2.vip]: clsrccssalive: Unable to get node number stat=21

2014-11-03 11:22:23.844: [    RACG][3068208832] [27259][3068208832][ora.node2.vip]: Interface eth0 checked failed (host=node2)
 Invalid parameters, or failed to bring up VIP (host=node2)

2014-11-03 11:22:23.844: [    RACG][3068208832] [27259][3068208832][ora.node2.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/u01/app/oracle/product/crs

2014-11-03 11:22:23.844: [    RACG][3068208832] [27259][3068208832][ora.node2.vip]: clsrcexecut: cmd = /u01/app/oracle/product/crs/bin/racgeut -e _USR_ORA_DEBUG=0 54 /u01/app/oracle/product/crs/bin/racgvip check node2

2014-11-03 11:22:23.844: [    RACG][3068208832] [27259][3068208832][ora.node2.vip]: clsrcexecut: rc = 1, time = 6.330s

2014-11-03 11:22:23.844: [    RACG][3068208832] [27259][3068208832][ora.node2.vip]: end for resource = ora.node2.vip, action = check, status = 1, time = 6.350s

2014-11-03 11:24:23.113: [    RACG][3068204736] [30064][3068204736][ora.node2.vip]: Interface eth0 checked failed (host=node2)
 Invalid parameters, or failed to bring up VIP (host=node2)

2014-11-03 11:24:23.114: [    RACG][3068204736] [30064][3068204736][ora.node2.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/u01/app/oracle/product/crs

2014-11-03 11:24:23.114: [    RACG][3068204736] [30064][3068204736][ora.node2.vip]: clsrcexecut: cmd = /u01/app/oracle/product/crs/bin/racgeut -e _USR_ORA_DEBUG=0 54 /u01/app/oracle/product/crs/bin/racgvip start node2

2014-11-03 11:24:23.114: [    RACG][3068204736] [30064][3068204736][ora.node2.vip]: clsrcexecut: rc = 1, time = 9.310s

2014-11-03 11:24:32.479: [    RACG][3068204736] [30064][3068204736][ora.node2.vip]: Interface eth0 checked failed (host=node2)
 Invalid parameters, or failed to bring up VIP (host=node2)

2014-11-03 11:24:32.479: [    RACG][3068204736] [30064][3068204736][ora.node2.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/u01/app/oracle/product/crs

2014-11-03 11:24:32.479: [    RACG][3068204736] [30064][3068204736][ora.node2.vip]: clsrcexecut: cmd = /u01/app/oracle/product/crs/bin/racgeut -e _USR_ORA_DEBUG=0 54 /u01/app/oracle/product/crs/bin/racgvip check node2

2014-11-03 11:24:32.479: [    RACG][3068204736] [30064][3068204736][ora.node2.vip]: clsrcexecut: rc = 1, time = 9.360s

2014-11-03 11:24:32.479: [    RACG][3068204736] [30064][3068204736][ora.node2.vip]: end for resource = ora.node2.vip, action = start, status = 1, time = 18.690s

2014-11-03 13:49:31.224: [    RACG][3067307712] [4955][3067307712][ora.node2.vip]: Interface eth0 checked failed (host=node2)
 Invalid parameters, or failed to bring up VIP (host=node2)

2014-11-03 13:49:31.266: [    RACG][3067307712] [4955][3067307712][ora.node2.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/u01/app/oracle/product/crs

2014-11-03 13:49:31.266: [    RACG][3067307712] [4955][3067307712][ora.node2.vip]: clsrcexecut: cmd = /u01/app/oracle/product/crs/bin/racgeut -e _USR_ORA_DEBUG=0 54 /u01/app/oracle/product/crs/bin/racgvip start node2

2014-11-03 13:49:31.266: [    RACG][3067307712] [4955][3067307712][ora.node2.vip]: clsrcexecut: rc = 1, time = 9.680s

2014-11-03 13:49:40.790: [    RACG][3067307712] [4955][3067307712][ora.node2.vip]: Interface eth0 checked failed (host=node2)
 Invalid parameters, or failed to bring up VIP (host=node2)

2014-11-03 13:49:40.790: [    RACG][3067307712] [4955][3067307712][ora.node2.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/u01/app/oracle/product/crs

2014-11-03 13:49:40.790: [    RACG][3067307712] [4955][3067307712][ora.node2.vip]: clsrcexecut: cmd = /u01/app/oracle/product/crs/bin/racgeut -e _USR_ORA_DEBUG=0 54 /u01/app/oracle/product/crs/bin/racgvip check node2

2014-11-03 13:49:40.790: [    RACG][3067307712] [4955][3067307712][ora.node2.vip]: clsrcexecut: rc = 1, time = 9.530s

2014-11-03 13:49:40.790: [    RACG][3067307712] [4955][3067307712][ora.node2.vip]: end for resource = ora.node2.vip, action = start, status = 1, time = 19.720s

2014-11-03 13:53:22.690: [    RACG][3068077760] [10724][3068077760][ora.node2.vip]: Interface eth0 checked failed (host=node2)
 Invalid parameters, or failed to bring up VIP (host=node2)

2014-11-03 13:53:22.690: [    RACG][3068077760] [10724][3068077760][ora.node2.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/u01/app/oracle/product/crs

2014-11-03 13:53:22.690: [    RACG][3068077760] [10724][3068077760][ora.node2.vip]: clsrcexecut: cmd = /u01/app/oracle/product/crs/bin/racgeut -e _USR_ORA_DEBUG=0 54 /u01/app/oracle/product/crs/bin/racgvip check node2

2014-11-03 13:53:22.690: [    RACG][3068077760] [10724][3068077760][ora.node2.vip]: clsrcexecut: rc = 1, time = 6.440s

2014-11-03 13:53:22.690: [    RACG][3068077760] [10724][3068077760][ora.node2.vip]: end for resource = ora.node2.vip, action = check, status = 1, time = 6.460s

2014-11-03 14:16:51.547: [    RACG][3067770560] [12993][3067770560][ora.node2.vip]: Interface eth0 checked failed (host=node2)
 Invalid parameters, or failed to bring up VIP (host=node2)

2014-11-03 14:16:51.547: [    RACG][3067770560] [12993][3067770560][ora.node2.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/u01/app/oracle/product/crs

2014-11-03 14:16:51.547: [    RACG][3067770560] [12993][3067770560][ora.node2.vip]: clsrcexecut: cmd = /u01/app/oracle/product/crs/bin/racgeut -e _USR_ORA_DEBUG=0 54 /u01/app/oracle/product/crs/bin/racgvip start node2

2014-11-03 14:16:51.547: [    RACG][3067770560] [12993][3067770560][ora.node2.vip]: clsrcexecut: rc = 1, time = 9.250s

2014-11-03 14:20:23.748: [    RACG][3067922112] [18191][3067922112][ora.node2.vip]: Interface eth0 checked failed (host=node2)
 Invalid parameters, or failed to bring up VIP (host=node2)

2014-11-03 14:20:23.749: [    RACG][3067922112] [18191][3067922112][ora.node2.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/u01/app/oracle/product/crs

2014-11-03 14:20:23.749: [    RACG][3067922112] [18191][3067922112][ora.node2.vip]: clsrcexecut: cmd = /u01/app/oracle/product/crs/bin/racgeut -e _USR_ORA_DEBUG=0 54 /u01/app/oracle/product/crs/bin/racgvip check node2

2014-11-03 14:20:23.749: [    RACG][3067922112] [18191][3067922112][ora.node2.vip]: clsrcexecut: rc = 1, time = 6.220s

2014-11-03 14:20:23.749: [    RACG][3067922112] [18191][3067922112][ora.node2.vip]: end for resource = ora.node2.vip, action = check, status = 1, time = 6.240s

2014-11-03 14:28:08.920: [    RACG][3068122816] [29510][3068122816][ora.node2.vip]: Interface eth0 checked failed (host=node2)
 Invalid parameters, or failed to bring up VIP (host=node2)

2014-11-03 14:28:08.921: [    RACG][3068122816] [29510][3068122816][ora.node2.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/u01/app/oracle/product/crs

2014-11-03 14:28:08.921: [    RACG][3068122816] [29510][3068122816][ora.node2.vip]: clsrcexecut: cmd = /u01/app/oracle/product/crs/bin/racgeut -e _USR_ORA_DEBUG=0 54 /u01/app/oracle/product/crs/bin/racgvip check node2

2014-11-03 14:28:08.921: [    RACG][3068122816] [29510][3068122816][ora.node2.vip]: clsrcexecut: rc = 1, time = 6.360s

2014-11-03 14:28:08.921: [    RACG][3068122816] [29510][3068122816][ora.node2.vip]: end for resource = ora.node2.vip, action = check, status = 1, time = 6.370s

2014-11-03 14:37:35.267: [    RACG][3067410112] [10640][3067410112][ora.node2.vip]: Interface eth0 checked failed (host=node2)
 Invalid parameters, or failed to bring up VIP (host=node2)

2014-11-03 14:37:35.267: [    RACG][3067410112] [10640][3067410112][ora.node2.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/u01/app/oracle/product/crs

2014-11-03 14:37:35.267: [    RACG][3067410112] [10640][3067410112][ora.node2.vip]: clsrcexecut: cmd = /u01/app/oracle/product/crs/bin/racgeut -e _USR_ORA_DEBUG=0 54 /u01/app/oracle/product/crs/bin/racgvip check node2

2014-11-03 14:37:35.267: [    RACG][3067410112] [10640][3067410112][ora.node2.vip]: clsrcexecut: rc = 1, time = 6.280s

2014-11-03 14:37:35.267: [    RACG][3067410112] [10640][3067410112][ora.node2.vip]: end for resource = ora.node2.vip, action = check, status = 1, time = 6.300s

2014-11-03 14:40:36.266: [    RACG][3067541184] [15021][3067541184][ora.node2.vip]: Interface eth0 checked failed (host=node2)
 Invalid parameters, or failed to bring up VIP (host=node2)

2014-11-03 14:40:36.266: [    RACG][3067541184] [15021][3067541184][ora.node2.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/u01/app/oracle/product/crs

2014-11-03 14:40:36.266: [    RACG][3067541184] [15021][3067541184][ora.node2.vip]: clsrcexecut: cmd = /u01/app/oracle/product/crs/bin/racgeut -e _USR_ORA_DEBUG=0 54 /u01/app/oracle/product/crs/bin/racgvip check node2

2014-11-03 14:40:36.266: [    RACG][3067541184] [15021][3067541184][ora.node2.vip]: clsrcexecut: rc = 1, time = 6.310s

2014-11-03 14:40:36.267: [    RACG][3067541184] [15021][3067541184][ora.node2.vip]: end for resource = ora.node2.vip, action = check, status = 1, time = 6.380s

2014-11-03 14:43:27.050: [    RACG][3067307712] [19015][3067307712][ora.node2.vip]: Interface eth0 checked failed (host=node2)
 Invalid parameters, or failed to bring up VIP (host=node2)

2014-11-03 14:43:27.050: [    RACG][3067307712] [19015][3067307712][ora.node2.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/u01/app/oracle/product/crs

2014-11-03 14:43:27.050: [    RACG][3067307712] [19015][3067307712][ora.node2.vip]: clsrcexecut: cmd = /u01/app/oracle/product/crs/bin/racgeut -e _USR_ORA_DEBUG=0 54 /u01/app/oracle/product/crs/bin/racgvip start node2

2014-11-03 14:43:27.050: [    RACG][3067307712] [19015][3067307712][ora.node2.vip]: clsrcexecut: rc = 1, time = 9.280s

2014-11-03 14:47:19.942: [    RACG][3067995840] [23291][3067995840][ora.node2.vip]: Interface eth0 checked failed (host=node2)
 Invalid parameters, or failed to bring up VIP (host=node2)

2014-11-03 14:47:19.942: [    RACG][3067995840] [23291][3067995840][ora.node2.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/u01/app/oracle/product/crs

2014-11-03 14:47:19.942: [    RACG][3067995840] [23291][3067995840][ora.node2.vip]: clsrcexecut: cmd = /u01/app/oracle/product/crs/bin/racgeut -e _USR_ORA_DEBUG=0 54 /u01/app/oracle/product/crs/bin/racgvip check node2

2014-11-03 14:47:19.942: [    RACG][3067995840] [23291][3067995840][ora.node2.vip]: clsrcexecut: rc = 1, time = 6.230s

2014-11-03 14:47:19.942: [    RACG][3067995840] [23291][3067995840][ora.node2.vip]: end for resource = ora.node2.vip, action = check, status = 1, time = 6.250s

2014-11-03 14:51:49.744: [    RACG][3067389632] [30018][3067389632][ora.node2.vip]: Interface eth0 checked failed (host=node2)
 Invalid parameters, or failed to bring up VIP (host=node2)

2014-11-03 14:51:49.745: [    RACG][3067389632] [30018][3067389632][ora.node2.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/u01/app/oracle/product/crs

2014-11-03 14:51:49.745: [    RACG][3067389632] [30018][3067389632][ora.node2.vip]: clsrcexecut: cmd = /u01/app/oracle/product/crs/bin/racgeut -e _USR_ORA_DEBUG=0 54 /u01/app/oracle/product/crs/bin/racgvip check node2

2014-11-03 14:51:49.745: [    RACG][3067389632] [30018][3067389632][ora.node2.vip]: clsrcexecut: rc = 1, time = 6.520s

2014-11-03 14:51:49.745: [    RACG][3067389632] [30018][3067389632][ora.node2.vip]: end for resource = ora.node2.vip, action = check, status = 1, time = 6.540s

2014-11-03 15:54:22.213: [    RACG][3067971264] [5646][3067971264][ora.node2.vip]: Interface eth0 checked failed (host=node2)
 Invalid parameters, or failed to bring up VIP (host=node2)

2014-11-03 15:54:22.218: [    RACG][3067971264] [5646][3067971264][ora.node2.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/u01/app/oracle/product/crs

2014-11-03 15:54:22.218: [    RACG][3067971264] [5646][3067971264][ora.node2.vip]: clsrcexecut: cmd = /u01/app/oracle/product/crs/bin/racgeut -e _USR_ORA_DEBUG=0 54 /u01/app/oracle/product/crs/bin/racgvip check node2

2014-11-03 15:54:22.218: [    RACG][3067971264] [5646][3067971264][ora.node2.vip]: clsrcexecut: rc = 1, time = 6.380s

2014-11-03 15:54:22.218: [    RACG][3067971264] [5646][3067971264][ora.node2.vip]: end for resource = ora.node2.vip, action = check, status = 1, time = 6.430s

2014-11-03 15:59:28.335: [    RACG][3067631296] [13317][3067631296][ora.node2.vip]: Interface eth0 checked failed (host=node2)
 Invalid parameters, or failed to bring up VIP (host=node2)

2014-11-03 15:59:28.335: [    RACG][3067631296] [13317][3067631296][ora.node2.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/u01/app/oracle/product/crs

2014-11-03 15:59:28.335: [    RACG][3067631296] [13317][3067631296][ora.node2.vip]: clsrcexecut: cmd = /u01/app/oracle/product/crs/bin/racgeut -e _USR_ORA_DEBUG=0 54 /u01/app/oracle/product/crs/bin/racgvip start node2

2014-11-03 15:59:28.335: [    RACG][3067631296] [13317][3067631296][ora.node2.vip]: clsrcexecut: rc = 1, time = 6.510s

2014-11-03 16:37:22.661: [    RACG][3067848384] [31375][3067848384][ora.node2.vip]: Interface eth0 checked failed (host=node2)
 Invalid parameters, or failed to bring up VIP (host=node2)

2014-11-03 16:37:22.661: [    RACG][3067848384] [31375][3067848384][ora.node2.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/u01/app/oracle/product/crs

2014-11-03 16:37:22.661: [    RACG][3067848384] [31375][3067848384][ora.node2.vip]: clsrcexecut: cmd = /u01/app/oracle/product/crs/bin/racgeut -e _USR_ORA_DEBUG=0 54 /u01/app/oracle/product/crs/bin/racgvip check node2

2014-11-03 16:37:22.661: [    RACG][3067848384] [31375][3067848384][ora.node2.vip]: clsrcexecut: rc = 1, time = 51.290s

2014-11-03 16:37:22.661: [    RACG][3067848384] [31375][3067848384][ora.node2.vip]: end for resource = ora.node2.vip, action = check, status = 1, time = 113.690s

2014-11-03 16:37:57.544: [    RACG][3067524800] [32281][3067524800][ora.node2.vip]: Interface eth0 checked failed (host=node2)
 Invalid parameters, or failed to bring up VIP (host=node2)

2014-11-03 16:37:57.545: [    RACG][3067524800] [32281][3067524800][ora.node2.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/u01/app/oracle/product/crs

2014-11-03 16:37:57.545: [    RACG][3067524800] [32281][3067524800][ora.node2.vip]: clsrcexecut: cmd = /u01/app/oracle/product/crs/bin/racgeut -e _USR_ORA_DEBUG=0 54 /u01/app/oracle/product/crs/bin/racgvip check node2

2014-11-03 16:37:57.545: [    RACG][3067524800] [32281][3067524800][ora.node2.vip]: clsrcexecut: rc = 1, time = 9.720s

2014-11-03 16:37:57.545: [    RACG][3067524800] [32281][3067524800][ora.node2.vip]: end for resource = ora.node2.vip, action = check, status = 1, time = 11.210s

2014-11-03 16:43:06.214: [    RACG][3067926208] [5514][3067926208][ora.node2.vip]: Interface eth0 checked failed (host=node2)
 Invalid parameters, or failed to bring up VIP (host=node2)

2014-11-03 16:43:06.215: [    RACG][3067926208] [5514][3067926208][ora.node2.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/u01/app/oracle/product/crs

2014-11-03 16:43:06.215: [    RACG][3067926208] [5514][3067926208][ora.node2.vip]: clsrcexecut: cmd = /u01/app/oracle/product/crs/bin/racgeut -e _USR_ORA_DEBUG=0 54 /u01/app/oracle/product/crs/bin/racgvip start node2

2014-11-03 16:43:06.215: [    RACG][3067926208] [5514][3067926208][ora.node2.vip]: clsrcexecut: rc = 1, time = 9.250s

2014-11-03 16:44:25.848: [    RACG][3067700928] [7520][3067700928][ora.node2.vip]: Interface eth0 checked failed (host=node2)
 Invalid parameters, or failed to bring up VIP (host=node2)

2014-11-03 16:44:25.848: [    RACG][3067700928] [7520][3067700928][ora.node2.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/u01/app/oracle/product/crs

2014-11-03 16:44:25.848: [    RACG][3067700928] [7520][3067700928][ora.node2.vip]: clsrcexecut: cmd = /u01/app/oracle/product/crs/bin/racgeut -e _USR_ORA_DEBUG=0 54 /u01/app/oracle/product/crs/bin/racgvip check node2

2014-11-03 16:44:25.848: [    RACG][3067700928] [7520][3067700928][ora.node2.vip]: clsrcexecut: rc = 1, time = 6.380s

2014-11-03 16:44:25.848: [    RACG][3067700928] [7520][3067700928][ora.node2.vip]: end for resource = ora.node2.vip, action = check, status = 1, time = 6.400s

2014-11-03 16:49:22.955: [    RACG][3067414208] [14413][3067414208][ora.node2.vip]: Interface eth0 checked failed (host=node2)
 Invalid parameters, or failed to bring up VIP (host=node2)

2014-11-03 16:49:22.955: [    RACG][3067414208] [14413][3067414208][ora.node2.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/u01/app/oracle/product/crs

2014-11-03 16:49:22.955: [    RACG][3067414208] [14413][3067414208][ora.node2.vip]: clsrcexecut: cmd = /u01/app/oracle/product/crs/bin/racgeut -e _USR_ORA_DEBUG=0 54 /u01/app/oracle/product/crs/bin/racgvip start node2

2014-11-03 16:49:22.955: [    RACG][3067414208] [14413][3067414208][ora.node2.vip]: clsrcexecut: rc = 1, time = 9.320s

2014-11-03 16:49:32.234: [    RACG][3067414208] [14413][3067414208][ora.node2.vip]: Interface eth0 checked failed (host=node2)
 Invalid parameters, or failed to bring up VIP (host=node2)

2014-11-03 16:49:32.234: [    RACG][3067414208] [14413][3067414208][ora.node2.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/u01/app/oracle/product/crs

2014-11-03 16:49:32.234: [    RACG][3067414208] [14413][3067414208][ora.node2.vip]: clsrcexecut: cmd = /u01/app/oracle/product/crs/bin/racgeut -e _USR_ORA_DEBUG=0 54 /u01/app/oracle/product/crs/bin/racgvip check node2

2014-11-03 16:49:32.234: [    RACG][3067414208] [14413][3067414208][ora.node2.vip]: clsrcexecut: rc = 1, time = 9.280s

2014-11-03 16:49:32.234: [    RACG][3067414208] [14413][3067414208][ora.node2.vip]: end for resource = ora.node2.vip, action = start, status = 1, time = 18.630s

2014-11-03 17:29:29.895: [    RACG][3067782848] [5942][3067782848][ora.node2.vip]: Interface eth0 checked failed (host=node2)
 Invalid parameters, or failed to bring up VIP (host=node2)

2014-11-03 17:29:29.895: [    RACG][3067782848] [5942][3067782848][ora.node2.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/u01/app/oracle/product/crs

2014-11-03 17:29:29.895: [    RACG][3067782848] [5942][3067782848][ora.node2.vip]: clsrcexecut: cmd = /u01/app/oracle/product/crs/bin/racgeut -e _USR_ORA_DEBUG=0 54 /u01/app/oracle/product/crs/bin/racgvip check node2

2014-11-03 17:29:29.895: [    RACG][3067782848] [5942][3067782848][ora.node2.vip]: clsrcexecut: rc = 1, time = 6.370s

2014-11-03 17:29:29.895: [    RACG][3067782848] [5942][3067782848][ora.node2.vip]: end for resource = ora.node2.vip, action = check, status = 1, time = 6.390s

2014-11-03 17:38:01.381: [    RACG][3067832000] [18181][3067832000][ora.node2.vip]: Interface eth0 checked failed (host=node2)
 Invalid parameters, or failed to bring up VIP (host=node2)

2014-11-03 17:38:01.381: [    RACG][3067832000] [18181][3067832000][ora.node2.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/u01/app/oracle/product/crs

2014-11-03 17:38:01.381: [    RACG][3067832000] [18181][3067832000][ora.node2.vip]: clsrcexecut: cmd = /u01/app/oracle/product/crs/bin/racgeut -e _USR_ORA_DEBUG=0 54 /u01/app/oracle/product/crs/bin/racgvip check node2

2014-11-03 17:38:01.381: [    RACG][3067832000] [18181][3067832000][ora.node2.vip]: clsrcexecut: rc = 1, time = 6.210s

2014-11-03 17:38:01.381: [    RACG][3067832000] [18181][3067832000][ora.node2.vip]: end for resource = ora.node2.vip, action = check, status = 1, time = 6.260s


发现eth0 ping的网关不正常。
 将eth1上的网关设置删除后,还是出现相同的报错,怀疑是vip的默认网关配置出了问题。

 

 

将所有的服务都停止。
 对节点2的vip进行debug,最后的5是_USR_ORA_DEBUG flag ,表示级别。
[root@rac10g2 ~]# crsctl debug log res "ora.node2.vip:5"
 Set Resource Debug Module: ora.node2.vip Level: 5
启动ora.node2.vip。
[root@rac10g2 ~]# crs_start ora.node2.vip
查看trace日志。trace日志在$ORA_CRS_HOME/log//racg/
果然发现默认网关设置不正常,设置为192.168.94.1,并不是预计的192.168.1.1。

2014-11-03 15:53:15.830: [ RACG][3086924000] [2200][3086924000][ora.rac10g2.vip]: Fri Nov 03 15:53:09 CST 2014 [ 2204 ] /sbin/mii-tool eth0 error
 Fri Nov 18 15:53:09 CST 2014 [ 2204 ] defaultgw: started
Fri Nov 03 15:53:09 CST 2014 [ 2204 ] defaultgw: completed with 192.168.94.1

2014-11-03 15:53:15.830: [ RACG][3086924000] [2200][3086924000][ora.node2.vip]: ping to 192.168.94.1 via eth0 failed, rc = 1 (host=node2)
 ping to 192.168.94.1 via eth0 failed, rc = 1 (host=rac10g2)
 Fri Nov 03 15:53:15 CST 2014 [ 2204 ] checkIf: ping and RX packets checked if=eth0 failed
 Interface eth0 checked failed (host=node2)

2014-11-03 15:53:15.830: [ RACG][3086924000] [2200][3086924000][ora.node2.vip]: Fri Dec 18 15:53:15 CST 2014 [ 2204 ] checkIf: end for if=eth0
 Invalid parameters, or failed to bring up VIP (host=node2)

修改默认网关:
 修改文件$ORA_CRS_HOME/bin/racgvip,将以下内容修改为想要指派的新的网关。
DEFAULTGW原本是没有值的,现在修改为192.168.1.1。
# hard code default gateway here if needed
DEFAULTGW=192.168.1.1

修改之后,在trace看到该部分已经检测ok。
2014-11-03 16:11:40.984: [ RACG][3086924000] [7808][3086924000][ora.rac10g2.vip]: Fri Nov 03 16:11:40 CST 2014 [ 7812 ] /sbin/mii-tool eth0 error
 Fri Nov 03 16:11:40 CST 2014 [ 7812 ] checkIf: ping checked if=eth0 ok
 Fri Nov 03 16:11:40 CST 2014 [ 7812 ] checkIf: end for if=eth0

该修改需要在节点1和节点2上同时修改,观察很长一段时间之后,没有再发现之前的问题。

其实该问题的出现,主要的原因是在eth1上配置了默认网关,用于私有网络的网卡不建议配置网关。所以该问题中,如果没有特别的必要,将私有网卡上的网关删除,删除默认路由网关,重启vip即可解决该问题。

在CentOS 6.4下安装Oracle 11gR2(x64)

Oracle 11gR2 在VMWare虚拟机中安装步骤

Debian 下 安装 Oracle 11g XE R2


    
 
 
 
本站(WWW.)旨在分享和传播互联网科技相关的资讯和技术,将尽最大努力为读者提供更好的信息聚合和浏览方式。
本站(WWW.)站内文章除注明原创外,均为转载、整理或搜集自网络。欢迎任何形式的转载,转载请注明出处。












  • 相关文章推荐
  • 安装Oracle加载数据库错误areasQueries的解决
  • 我在安装oracle的时候出现了这个错误,请懂行人赐教~~附图!~~
  • 记录Linux下一次oracle启动错误
  • shell如何获取oracle错误码 在线等
  • Linux系统下安装Oracle X11错误解决办法
  • 求助前辈们:solaris 10 x86 安装oracle 10g 错误
  • 有 ORACLE 错误码文档吗? 谢谢大家!
  • Eclipse连接Oracle数据库的ORA-00604 ORA-12705错误
  • ORACLE安装时/tmp/orainstRoot.sh 执行发生错误
  • 关于提高Oracle数据库性能的四个错误认识
  • C++使用OCCI连Oracle10g的错误
  • Linux上管理本机Oracle的时候出现找不到ServiceName的错误
  • Oracle的sqlcode对应的错误信息sqlerrm
  • 连接Oracle817数据库的错误提示
  • Oracle AS关键字 提示错误
  • rh9上装oracle817犯了个低级错误,如何挽回?
  • oracle错误迷惑
  • FC6下安装Oracle9i错误,请高手指教 在线等 解决即结贴
  • Oracle10201 RAC升级到10204后导出数据时报EXP-00056错误
  • 在tomcat下连接oracle8i的javabean出现如下错误,急!
  • Oracle 12c发布简单介绍及官方下载地址
  • 在linux下安装oracle,如何设置让oracle自动启动!也就是让oracle那个服务自动启动,不是手动的
  • oracle 11g最新版官方下载地址
  • 请问su oracle 和su - oracle有什么不同?
  • Oracle 数据库(oracle Database)Select 多表关联查询方式
  • 关于提高Oracle数据库性能的四个错误认识 iis7站长之家
  • Oracle数据库(Oracle Database)体系结构及基本组成介绍
  • Oracle 数据库开发工具 Oracle SQL Developer
  • 如何设置让Oracle SQL Developer显示的时间包含时分秒
  • Oracle EBS R12 支持 Oracle Database 11g
  • Oracle 10g和Oracle 11g网格技术介绍


  • 站内导航:


    特别声明:169IT网站部分信息来自互联网,如果侵犯您的权利,请及时告知,本站将立即删除!

    ©2012-2021,