告警日志中除了这个ORA-600(KSLGES_3)的错误,还有一堆的ORA-2730N的操作系统相关错误。
数据库版本为RAC 10204 FOR HP UNIX
详细的错误信息为:
Fri Oct 15 17:37:35 2010
Errors in file /oracle10g/admin/B1ACDB/bdump/b1acdb1_lmon_27909.trc:
ORA-00600: internal error code, arguments: [KSLGES_3], [], [], [], [], [], [], []
ORA-27300: OS system dependent operation:semop failed with status: 22
ORA-27301: OS failure message: Invalid argument
ORA-27302: failure occurred at: sskgpwwait3
ORA-27303: additional information: ctx(8000000100012cb8); wid(c00000009d1b7898); flags(200)
semid(0xb48d); sem_num(9); oldval(-1)
Fri Oct 15 17:37:40 2010
Errors in file /oracle10g/admin/B1ACDB/bdump/b1acdb1_ckpt_27990.trc:
ORA-27157: OS post/wait facility removed
ORA-27300: OS system dependent operation:semop failed with status: 36
ORA-27301: OS failure message: Identifier removed
ORA-27302: failure occurred at: sskgpwwait1
Fri Oct 15 17:37:40 2010
CKPT: terminating instance due to error 27157
Fri Oct 15 17:37:41 2010
System state dump is made for local instance
System State dumped to trace file /oracle10g/admin/B1ACDB/bdump/b1acdb1_diag_27895.trc
Fri Oct 15 17:37:43 2010
Errors in file /oracle10g/admin/B1ACDB/bdump/b1acdb1_lmon_27909.trc:
ORA-00600: internal error code, arguments: [KSLGES_3], [], [], [], [], [], [], []
ORA-27300: OS system dependent operation:semop failed with status: 22
ORA-27301: OS failure message: Invalid argument
ORA-27302: failure occurred at: sskgpwwait3
ORA-27303: additional information: ctx(8000000100012cb8); wid(c00000009d1b7898); flags(200)
semid(0xb48d); sem_num(9); oldval(-1)
Fri Oct 15 17:37:44 2010
Trace dumping is performing id=[cdmp_20101015173741]
Fri Oct 15 17:37:46 2010
Starting ORACLE instance (normal)
Fri Oct 15 17:37:51 2010
Instance terminated by CKPT, pid = 27990
Fri Oct 15 17:37:51 2010
Errors in file /oracle10g/admin/B1ACDB/bdump/b1acdb1_ckpt_27990.trc:
ORA-27300: OS system dependent operation:semctl failed with status: 22
ORA-27301: OS failure message: Invalid argument
ORA-27302: failure occurred at: sskgpwrm1
ORA-27157: OS post/wait facility removed
ORA-27300: OS system dependent operation:semop failed with status: 36
ORA-27301: OS failure message: Identifier removed
ORA-27302: failure occurred at: sskgpwwait1
虽然开始报了一个ORA-600的错误,但是真正导致的应该是后面的这些ORA-2730N的错误,根据错误查询metalink文档,文档ID 438205.1描述的就是这个问题。
简单的说,就是Oracle使用的共享内存段被删除了,可能是人为的操作,也可能是硬件问题导致。
问题发生的时间离现在已经比较远了,当时发生了什么,执行了什么操作已经不可考了,不过alert文件中,发生错误前后记录的系统时间发生了错乱,说明当时系统或人为肯定进行了某些操作,至少改变过系统时间。
这个仅发生过一次,基本上可以排除Oracle自身的问题,应该是操作系统上的操作或错误导致了这个问题的产生。