DB startup/shutdown error after downgrade from 11gR2 to 10gR2

I was asked to work on a 2-node RAC db which was downgraded to 10.2.0.5 from 11.2.0.3. While working, i had stopped the database using “shutdown immediate” command from sqlplus instead of srvctl. While start the db using srvctl –

wdlab1: (matrix1) /u01/app/oracle/product/rdbms/10205> srvctl  start database -d matrix_lab -o open
PRKR-1001 : cluster database matrix_lab does not exist
PRKO-2005 : Application error: Failure in getting Cluster Database Configuration for: matrix_lab
wdlab1: (matrix1) /u01/app/oracle/product/rdbms/10205/bin> which srvctl
/u01/app/oracle/product/rdbms/10205/bin/srvctl

Hmmm…lets try starting it using srvctl from 11gR2 grid home

wdlab1: (matrix1) /u01/app/grid/11.2.0/bin> ./srvctl start database -d matrix_lab -o open
PRCR-1079 : Failed to start resource ora.matrix_lab.db
CRS-5017: The resource action "ora.matrix_lab.db start" encountered the following error:
ORA-02095: specified initialization parameter cannot be modified
. For details refer to "(:CLSN00107:)" in "/u01/app/grid/11.2.0/log/wdlab1/agent/crsd/oraagent_oracle/oraagent_oracle.log".

CRS-2674: Start of 'ora.matrix_lab.db' on 'wdlab1' failed
CRS-5017: The resource action "ora.matrix_lab.db start" encountered the following error:
ORA-02095: specified initialization parameter cannot be modified
. For details refer to "(:CLSN00107:)" in "/u01/app/grid/11.2.0/log/wdlab2/agent/crsd/oraagent_oracle/oraagent_oracle.log".

CRS-2632: There are no more servers to try to place resource 'ora.matrix_lab.db' on that would satisfy its placement policy
CRS-2674: Start of 'ora.matrix_lab.db' on 'wdlab2' failed

Hmmm…lets try to see what is the issue

wdlab1: (matrix1) /u01/app/grid/11.2.0/bin> ./crsctl stat res -t
..........
ora.matrix_lab.db
      1        ONLINE  OFFLINE                               Instance Shutdown
      2        ONLINE  OFFLINE                               Instance Shutdown
..........

wdlab1: (matrix1) /u01/app/grid/11.2.0/bin> ./crsctl stat res ora.matrix_lab.db -p
....................
GEN_USR_ORA_INST_NAME@SERVERNAME(wdlab1)=matrix1
GEN_USR_ORA_INST_NAME@SERVERNAME(wdlab2)=matrix2
HOSTING_MEMBERS=
INSTANCE_FAILOVER=0
....................
USR_ORA_INST_NAME=
USR_ORA_INST_NAME@SERVERNAME(wdlab1)=matrix1
USR_ORA_INST_NAME@SERVERNAME(wdlab2)=matrix2
USR_ORA_OPEN_MODE=open
....................

For a pre-11gR2 database registered to the 11gR2 grid should not have entry like “GEN_USR_ORA_INST_NAME@SERVERNAME” and also should have “ora.dbname.instancename.inst” in the crsctl stat res -t output.That was enough of the clue that, though the db was downgraded seemed like it wasn’t removed and added back using the 10gR2 OH srvctl.

wdlab1: (matrix1) /u01/app/grid/11.2.0/bin> ./srvctl remove instance -d matrix_lab -i matrix1 -f
wdlab1: (matrix1) /u01/app/grid/11.2.0/bin> ./srvctl remove instance -d matrix_lab -i matrix2 -f
wdlab1: (matrix1) /u01/app/grid/11.2.0/bin> ./srvctl remove database -d matrix_lab -f
wdlab1: (matrix1) /u01/app/grid/11.2.0/bin>
wdlab1: (matrix1) /u01/app/grid/11.2.0/bin> ./crsctl stat res -t

crsctl stat res -t didn;t show any entry for it. Add the instance and db info using srvctl from 10gR2 home

wdlab1: (matrix1) /u01/app/oracle/product/rdbms/10205/bin> which srvctl
/u01/app/oracle/product/rdbms/10205/bin/srvctl
wdlab1: (matrix1) /u01/app/oracle/product/rdbms/10205/bin> srvctl add database -d matrix_lab -o /u01/app/oracle/product/rdbms/10205
wdlab1: (matrix1) /u01/app/oracle/product/rdbms/10205/bin> srvctl add instance -d matrix_lab -i matrix1 -n wdlab1
wdlab1: (matrix1) /u01/app/oracle/product/rdbms/10205/bin> srvctl add instance -d matrix_lab -i matrix2 -n wdlab2
wdlab1: (matrix1) /home/oracle> srvctl modify database -d matrix_lab -p '/u01/admin/matrix/spfile/spfilematrix.ora' -s open

crsctl stat res -t output showed –

wdlab1: (matrix1) /u01/app/grid/11.2.0/bin> ./crsctl stat res -t
--------------------------------------------------------------------------------
NAME           TARGET  STATE        SERVER                   STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
......................
......................
ora.matrix_lab.db
      1        OFFLINE OFFLINE
ora.matrix_lab.matrix1.inst
      1        OFFLINE OFFLINE
ora.matrix_lab.matrix2.inst
      1        OFFLINE OFFLINE
.....................

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s