After upgrading a 10gR2 ASM instance to 11.2, ASM is able to mount the disk groups however these quickly dismount as soon as v$asm_file is queried:
SUCCESS: diskgroup DATA_DG01 was mounted
SUCCESS: ALTER DISKGROUP DATA_DG01 MOUNT /* asm agent */
Thu Jun 24 15:13:05 2010
NOTE: diskgroup resource ora.DATA_DG01.dg is online
Thu Jun 24 15:18:31 2010
WARNNING: cache read a corrupted block group=DATA_DG01 fn=1 blk=512 from disk 0
NOTE: a corrupted block from group DATA_DG01 was dumped to
/u000/app/grid/diag/asm/+asm/+ASM/trace/+ASM_ora_348464.trc
WARNNING: cache read(retry) a corrupted block group=DATA_DG01 fn=1 blk=512
from disk 0
ERROR: cache failed to read group=DATA_DG01 fn=1 blk=512 from disk(s): 0
DATA_DG01_0000
ORA-15196: invalid ASM block header [kfc.c:23925] [hard_kfbh] [1] [512] [0 !=130]
Solution
-----------
1) Run the following from 11.2 ASM on each disk group a few times (say 3 times):
SQL> ALTER DISKGROUP DATA_DG_1 CHECK ALL REPAIR;
On the first time they execute this command, the ASM alert log will show entries like:
"ERROR: file 1 extent 0: blocks XX to XX are unformatted"
The next runs, the ASM alert logs will show something like:
ERROR: file 1 extent 0: blocks XX to XX are unformatted"
"SUCCESS: file 1 extent 0 repaired"
2) Check if ASM in 11.2 still dismounts the disk group when querying v$asm_file.
3) If the disk group continues to be dismounted with the above errors, then restore the 10gR2 environment (like prior to the ASM upgrade) and then apply Patch 5100163 to the 10.2.0.4 ASM home.
4) After the patch was installed, then run the ALTER DISKGROUP...CHECK ALL REPAIR on all the disk groups. Then retry the ASM upgrade.
