客户的9i物理STANDBY出现了ORA-600(2878)错误,简单记录一下解决的过程。
详细的错误信息如下:
Tue Apr 19 15:14:20 2011
Errors in file /u01/oracle/admin/oric/bdump/oric_dbw0_25026.trc:
ORA-00600: internal error code, arguments: [2878], [13], [7680000], [4194303], [], [], [], []
Tue Apr 19 15:14:20 2011
Errors in file /u01/oracle/admin/oric/bdump/oric_dbw0_25026.trc:
ORA-00600: internal error code, arguments: [2878], [13], [7680000], [4194303], [], [], [], []
Tue Apr 19 15:14:20 2011
DBW0: terminating instance due to error 600
Instance terminated by DBW0, pid = 25026
STANDBY数据库只能启动到NOMOUNT状态,当执行ALTER DATABASE MOUNT STANDBY DATABASE就会出现这种ORA-600(2878)错误。
查询了一下metalink,发现这种错误本身很少见,一般都是控制文件损坏造成的。
检查了一下这个数据库之前的操作:
Tue Apr 19 10:58:01 2011
RFS: Successfully opened standby logfile 9: '/u03/oradata/oric/redo_stb_9_1.log'
Tue Apr 19 10:58:08 2011
ARC0: Completed archiving log 7 thread 2 sequence 277
Tue Apr 19 12:09:58 2011
alter tablespace DANGANJIEKO add datafile '/u02/oradata/oric/DANGANJIEKO.dbf' size 30000M autoextend off
Tue Apr 19 12:09:58 2011
ORA-1109 signalled during: alter tablespace DANGANJIEKO add datafile '/u02/or...
Tue Apr 19 12:31:10 2011
RFS: Successfully opened standby logfile 8: '/u03/oradata/oric/redo_stb_8_1.log'
Tue Apr 19 09:01:32 2011
ARC0: Completed archiving log 9 thread 1 sequence 474
Tue Apr 19 10:58:00 2011
ARC0: Evaluating archive log 7 thread 2 sequence 277
ARC0: Beginning to archive log 7 thread 2 sequence 277
Creating archive destination LOG_ARCHIVE_DEST_1: '/u01/oracle/oradata/oric/archivelog/2_277.dbf'
Tue Apr 19 10:58:01 2011
RFS: Successfully opened standby logfile 9: '/u03/oradata/oric/redo_stb_9_1.log'
Tue Apr 19 10:58:08 2011
ARC0: Completed archiving log 7 thread 2 sequence 277
Tue Apr 19 12:09:58 2011
alter tablespace DANGANJIEKO add datafile '/u02/oradata/oric/DANGANJIEKO.dbf' size 30000M autoextend off
Tue Apr 19 12:09:58 2011
ORA-1109 signalled during: alter tablespace DANGANJIEKO add datafile '/u02/or...
Tue Apr 19 12:31:10 2011
alter database rename file '/dev/raw/DANGANJIEKO.dbf' to '/u02/oradata/oric/DANGANJIEKO1.dbf'
Tue Apr 19 12:31:10 2011
ORA-1511 signalled during: alter database rename file '/dev/raw/DANGANJIEKO.d...
Tue Apr 19 12:32:08 2011
Errors in file /u01/oracle/admin/oric/bdump/oric_dbw0_16090.trc:
ORA-01157: cannot identify/lock data file 13 - see DBWR trace file
ORA-01110: data file 13: '/dev/raw/DANGANJIEKO.dbf'
ORA-27037: unable to obtain file status
Linux Error: 2: No such file or directory
Additional information: 3
Tue Apr 19 12:32:08 2011
Errors in file /u01/oracle/admin/oric/bdump/oric_dbw0_16090.trc:
ORA-01157: cannot identify/lock data file 13 - see DBWR trace file
ORA-01110: data file 13: '/dev/raw/DANGANJIEKO.dbf'
ORA-27037: unable to obtain file status
Linux Error: 2: No such file or directory
Additional information: 3
Tue Apr 19 13:59:00 2011
alter database rename file '/dev/raw/DANGANJIEKO.dbf' to '/u02/oradata/oric/DANGANJIEKO.dbf'
Tue Apr 19 13:59:05 2011
Starting control autobackup
Control autobackup failed with following error :
ORA-00236: snapshot operation disallowed: mounted controlfile is a backup
Completed: alter database rename file '/dev/raw/DANGANJIEKO.d
Tue Apr 19 14:09:47 2011
alter database create datafile 13 as '/u02/oradata/oric/DANGANJIEKO.dbf' size 60000M autoextend off
Tue Apr 19 14:09:47 2011
ORA-1119 signalled during: alter database create datafile 13 as '/u02/oradata...
Tue Apr 19 14:13:36 2011
alter database create datafile 13 as '/u02/oradata/oric/DANGANJIEKO.dbf' size 60000M autoextend off
Tue Apr 19 14:49:01 2011
Starting control autobackup
Control autobackup failed with following error :
ORA-00236: snapshot operation disallowed: mounted controlfile is a backup
Completed: alter database create datafile 13 as '/u02/oradata
由于这是一个RAC环境的STANDBY数据库,且主库采用裸设备,因此备库设置了手工管理STANDBY数据文件。
而上面的手工创建文件操作显然是错误的,正是这个错误导致了STANDBY控制文件的损坏。
由于控制文件无法在没有MOUNT的状态下进行修改,所以这个STANDBY环境只有通过重建控制文件来解决。
在主库上新建STANDBY控制文件,利用备库的初始化参数启动,并将主库新增的数据文件在备库上进行还原,最后应用日志,问题解决。