Last week I had performed switchover activity of database on version 11.2.0.3 The switchover was performed using dgmgrl “swicthover to standby” command. After sometime we started receiving “ORA-00600: [ktbdchk1: bad dscn]” on the primary database.
Tue Dec 16 10:33:26 2014 Errors in file /ora_software/diag/rdbms/db02_dbv/dbv/trace/db02_ora_16271.trc (incident=434103): ORA-00600: internal error code, arguments: [ktbdchk1: bad dscn], [], [], [], [], [], [], [], [], [], [], [] Incident details in: /ora_software/diag/rdbms/db02_dbv/dbv/incident/incdir_434103/db02_ora_16271_i434103.trc
The trace file showed
*** ACTION NAME:() 2014-12-16 10:33:26.857
Dump continued from file: /ora_software/diag/rdbms/db02_dbv/dbv/trace/db02_ora_16271.trc
ORA-00600: internal error code, arguments: [ktbdchk1: bad dscn], [], [], [], [], [], [], [], [], [], [], []
========= Dump for incident 434103 (ORA 600 [ktbdchk1: bad dscn]) ========
----- Beginning of Customized Incident Dump(s) -----
[ktbdchk] -- ktbgcl4 -- bad dscn
dependent scn: 0x0008.f197f24e recent scn: 0x0008.c7313a4c current scn: 0x0008.c7313a4c
----- End of Customized Incident Dump(s) -----
*** 2014-12-16 10:33:26.961
dbkedDefDump(): Starting incident default dumps (flags=0x2, level=3, mask=0x0)
----- Current SQL Statement for this session (sql_id=dmrpvzvbbsuy5) -----
INSERT INTO MSG_OPEN( SITE_ID, CLIENT_ID, CAMP_ID, MESSAGE_ID, ID, USER_ID, ADDRESS, DATE_OPENED) VALUES( :B7 , :B6 , :B5 , :B4 , LOG_SEQ.NEXTVAL, :B3 , :B2 , :B1 )
----- PL/SQL Stack -----
----- PL/SQL Call Stack -----
object line object
handle number name
0x32276ee18 611 package body TP.EM_PKG
0x31cbac388 3 anonymous block
----- Call Stack Trace -----
calling call entry argument values in hex
location type point (? means dubious value)
-------------------- -------- -------------------- ----------------------------
skdstdst()+36 call kgdsdst() 000000000 ? 000000000 ?
7FFFFDA13028 ? 000000001 ?
000000001 ? 000000002 ?
ksedst1()+98 call skdstdst() 000000000 ? 000000000 ?
7FFFFDA13028 ? 000000001 ?
000000000 ? 000000002 ?
ksedst()+34 call ksedst1() 000000000 ? 000000001 ?
7FFFFDA13028 ? 000000001 ?
000000000 ? 000000002 ?
dbkedDefDump()+2741 call ksedst() 000000000 ? 000000001 ?
7FFFFDA13028 ? 000000001 ?
000000000 ? 000000002 ?
........................
Searching for the issue on Metalink pointed to the below Document:-
ALERT Description and fix for Bug 8895202: ORA-1555 / ORA-600 [ktbdchk1: bad dscn] ORA-600 [2663] in Physical Standby after switchover (Doc ID 1608167.1)
As per metalink
In a Data Guard environment with Physical Standby (including Active Data Guard), invalid SCNs can be introduced in index blocks after a switchover.
Symptoms ORA-1555 / ORA-600 [ktbdchk1: bad dscn] / ktbGetDependentScn / Dependent scn violations as the block ITL has higher COMMIT SCN than block SCN. DBVERIFY reports the next error when the fix of Bug 7517208 is present; reference Note 7517208.8 for interim patches: itl[] has higher commit scn(aaa.bbb) than block scn (xx.yy) Page failed with check code 6056 There is NO DATA CORRUPTION in the block.
To Resolve
The fix of Bug 8895202 is the workaround.
Although the fix of Bug 8895202 is included in patchset 11.2.0.2 and later, the fix needs to be enabled by setting parameter _ktb_debug_flags = 8.
SQL> alter system set "_ktb_debug_flags"=8 scope=both sid='*'; System altered. SQL> exit
If you are using Oracle version less than 11.2.0.2, then rebuilding index is the option, as we did for one of the client on 11.1.0.7 version.
One thing to note is —
In rare cases blocks healed by this fix may cause queries to fail with an ORA-600 [ktbgcl1_KTUCLOMINSCN_1] as described in Note 13513004.8 / Bug 13513004.
For more detail Metalink Doc 1608167.1
Learn. Share. Repeat. 
You are not the only one 😉
Go figure : http://laimisnd.wordpress.com/2014/02/27/11g-ora-1555-on-activated-standby-after-migration-from-10g/
Good One buddy… !!
Thanks for the detailed description! I also have the same problem and the database is running in 10.x compatible mode. How could you find the name of the index that needs to be rebuilt from the page number provided by the DBVERIFY utility? Thanks!
Hi Gheja,
Running dbverify normally provides RDBA (Relative data block address) of the affected block.
Usually you would see
Page XX is marked corrupt
Corrupt block relative dba: 0x02c00022 (file RFN, block BL)
If you have got something similar, you can use
select file_id AFN, relative_fno RFN,filename,tablespace_name from dba_data_files where relative_fno= RFN;
The above command would give you the Actual File_id. Using it you can run the below SQL to identify the segment :-
select *
from dba_extents
where file_id = AFN
and BL between block_id AND block_id + blocks – 1;
Hi Anand,
Thanks for the quick reply. Unfortunately the DBVERIFY did not return any rdba number just the page number, also when I looked in the dba_extents I was unable to find the problematic block.
In the alert log I found the name of the trace file for the incident and that contained the SQL that was interrupted due to the error. I rebuilt all the indexes and now the problem seems to be solved. By re-running the SQL after each rebuild it turned out that one in “valid” state was the problematic one.
Thanks again for the article and the help!
Gabor
Thanks a lot from Colombia!!