ASM Diskgroup shows USABLE_FILE_MB value in Negative

Today while working on ASM diskgroup i noticed Negative value for USABLE_FILE_MB. I was little surprised as it has been pretty long that i worked on ASM. So i started looking around for blogs and mos docs and found few really nice one around.

A negative value for USABLE_FILE_MB means that you do not have sufficient free space to tolerate a disk failure. If a disk were to fail, the subsequent rebalance would run out of space before full redundancy could be restored to all files.

I would really recommend reading :-

http://prutser.wordpress.com/2013/01/03/demystifying-asm-required_mirror_free_mb-and-usable_file_mb/

The box i was working on was exadata server quarter rack, so it had 3 storage server. Each storage server on an exadata server has 12 cell disk. Grid disk are created within Cell Disks.In a simple configuration, One Grid Disk can be created per Cell Disk and Grid disks are what the storage cell presents to db servers. So basically

GRID DISK = ASM DISK.

When creating disk groups, ASM automatically puts all grid disks from the same storage cell into the same failgroup. The failgroup is then named after the storage cell.


[oracle@test~]$ asmcmd lsdg
State    Type    Rebal  Sector  Block       AU  Total_MB  Free_MB  Req_mir_free_MB  Usable_file_MB  Offline_disks  Voting_files  Name
MOUNTED  NORMAL  N         512   4096  4194304  40697856  8464936         13565952        -2550508              0             N  DATA1/
MOUNTED  NORMAL  N         512   4096  4194304    415296   367220           138432          114394              0             Y  DBFS_DG/
MOUNTED  NORMAL  N         512   4096  4194304  10176480  9018276          3392160         2813058              0             N  RECO1/

compute sum Label total_FG of total_mb on FAILGROUP
compute sum Label total of total_mb on report
col diskgroup for a20
col failgroup for a30
col name for a30
select g.name diskgroup, d.failgroup,  d.name, d.total_mb from v$asm_disk d, v$asm_diskgroup g where g.name = 'DATA1' and d.GROUP_NUMBER = g.GROUP_NUMBER order by g.name, d.failgroup;

DISKGROUP            FAILGROUP                      NAME                                   TOTAL_MB
-------------------- ------------------------------ ------------------------------ ----------------
DATA1               CELL01                         DATA1_CD_00_CELL01             2260992
DATA1                                              DATA1_CD_05_CELL01             2260992
DATA1                                              DATA1_CD_03_CELL01             2260992
DATA1                                              DATA1_CD_04_CELL01             2260992
DATA1                                              DATA1_CD_01_CELL01             2260992
DATA1                                              DATA1_CD_02_CELL01             2260992
                     ******************************                                ----------------
                     total_FG                                                              13565952
DATA1               CELL02                         DATA1_CD_01_CELL02             2260992
DATA1                                              DATA1_CD_05_CELL02             2260992
DATA1                                              DATA1_CD_02_CELL02             2260992
DATA1                                              DATA1_CD_03_CELL02             2260992
DATA1                                              DATA1_CD_00_CELL02             2260992
DATA1                                              DATA1_CD_04_CELL02             2260992
                     ******************************                                ----------------
                     total_FG                                                              13565952
DATA1               CELL03                         DATA1_CD_02_CELL03             2260992
DATA1                                              DATA1_CD_05_CELL03             2260992
DATA1                                              DATA1_CD_01_CELL03             2260992
DATA1                                              DATA1_CD_04_CELL03             2260992
DATA1                                              DATA1_CD_03_CELL03             2260992
DATA1                                              DATA1_CD_00_CELL03             2260992
                     ******************************                                ----------------
                     total_FG                                                              13565952
                                                                                   ----------------
total                                                                                      40697856

For DATA1 diskgroup the USABLE_FILE_MB shows value in Negative (-2550508 MB).

SQL> select name, state, type, total_mb, free_mb, required_mirror_free_mb req_free,  usable_file_mb use_mb from v$asm_diskgroup where name = 'DATA1';

NAME                      STATE       TYPE     TOTAL_MB    FREE_MB   REQ_FREE     USE_MB
------------------------- ----------- ------ ---------- ---------- ---------- ----------
DATA1                      MOUNTED     NORMAL   40697856    8464936   13565952   -2550508
                                                                                                              ----------
total                                                                                                           40697856

TOTAL_MB:- Refers to total capacity of the diskgroup
FREE_MB :- Refers to raw free space available in diskgroup in MB.

FREE_MB = (TOTAL_MB – (HOT_USED_MB + COLD_USED_MB))

REQUIRED_MIRROR_FREE_MB :- Indicates how much free space is required in an ASM disk group to restore redundancy after the failure of an ASM disk or ASM failure group.In exadata it is the disk capacity of one failure group.

USABLE_FILE_MB :- Indicates how much space is available in an ASM disk group considering the redundancy level of the disk group.

Its calculated as :-

USABLE_FILE_MB=(FREE_MB – REQUIRED_MIRROR_FREE_MB ) / 2 –> For Normal Redundancy
USABLE_FILE_MB=(FREE_MB – REQUIRED_MIRROR_FREE_MB ) / 3 –> For High Redundancy

Also to note here is ASM diskgroup do not set aside the space based on reuqired_mirror_free_mb. Its merely calculated and used to derive usable_file_mb.

While reading Mos Doc Id 1551288.1 i came across some interesting terms and script which i wanted to share to everyone (atleast some of you who might not have been familiar)

Failure coverage refers to the amount of space in a disk group that will be used to re-mirror data in the event of some storage failure.

1. Disk Failure Coverage :- Refers to having enough free space to allow data to be re-mirrored (rebalanced) after a single disk failure in Normal redundancy.

2. Cell Failure Coverage :- Refers to having enough free space to allow data to be re-mirrored after loss of One entire Cell Disk.

Reserving space in the disk group means that you monitor the disk group to ensure that FREE_MB never goes below minimum amount needed for disk or cell failure coverage.

I ran the script provided in Mos Docid 1551288 and below was the output :-

Description of Derived Values:
One Cell Required Mirror Free MB : Required Mirror Free MB to permit successful rebalance after losing largest CELL regardless of redundancy type
Disk Required Mirror Free MB     : Space needed to rebalance after loss of single or double disk failure (for normal or high redundancy)
Disk Usable File MB              : Usable space available after reserving space for disk failure and accounting for mirroring
Cell Usable File MB              : Usable space available after reserving space for SINGLE cell failure and accounting for mirroring
.  .  .
ASM Version: 11.2.0.4
.  .  .
----------------------------------------------------------------------------------------------------------------------------------------------------
|          |         |     |          |            |            |            |Cell Req'd  |Disk Req'd  |            |            |    |    |       |
|          |DG       |Num  |Disk Size |DG Total    |DG Used     |DG Free     |Mirror Free |Mirror Free |Disk Usable |Cell Usable |    |    |PCT    |
|DG Name   |Type     |Disks|MB        |MB          |MB          |MB          |MB          |MB          |File MB     |File MB     |DFC |CFC |Util   |
----------------------------------------------------------------------------------------------------------------------------------------------------
|DATA1    |NORMAL   |   18| 2,260,992|  40,697,856|  32,233,944|   8,463,912|  14,922,547|   2,761,008|   2,851,452|  -3,229,318|PASS|FAIL|  79.2%|
|DBFS_DG  |NORMAL   |   12|    34,608|     415,296|      48,076|     367,220|     152,275|      59,425|     153,898|     107,472|PASS|PASS|  11.6%|
|RECO1    |NORMAL   |   18|   565,360|  10,176,480|   1,171,220|   9,005,260|   3,731,376|     703,460|   4,150,900|   2,636,942|PASS|PASS|  11.5%|
----------------------------------------------------------------------------------------------------------------------------------------------------
Cell Failure Coverage Freespace Failures Detected. Warning Message Follows.
Enough Free Space to Rebalance after loss of ONE cell: WARNING (However, cell failure is very rare)
.  .  .
Script completed.

So here i am good with one disk failure but not with One celldisk failure. Basically i need to either add disk to the disk group or free up some space.

This post is more of a note for myself to refer back. Hope it useful for some of you too 🙂

Advertisements

2 thoughts on “ASM Diskgroup shows USABLE_FILE_MB value in Negative

  1. Hi Anand,

    Can u try this
    asmcmd -p lsdg | awk ‘BEGIN { used_per=0; print “\n\t\t ASM – Disk Used Percentage”; print “\n\t\t ============================ \n”; printf “%-15s%-15s%-15s\n”,”Name”,”Total_Mb”,”Free_Mb”,”Used_percent”} NR > 1 { used_per=$8-$9; used_per=(used_per/$8)*100; printf “%-15s%-15d%-15d%-8.3f\n”, $13,$8,$9,used_per; }’

    Does that works in sync with your query.

    1. asmcmd -p lsdg | awk ‘BEGIN { used_per=0; print “\n\t\t ASM . Disk Used Percentage”; print “\n\t\t ============================ \n”;
      printf “%-15s%-15s%-15s\n”,”Name”,”Total_Mb”,”Free_Mb”,”Used_percent”} NR > 1 { used_per=$8-$9; used_per=(used_per/$8)*100;
      printf “%-15s%-15d%-15d%-8.3f\n”, $13,$8,$9,used_per; }’

      ASM . Disk Used Percentage

      ============================

      Name Total_Mb Free_Mb
      DATA1/ 8246176 13565952 -64.512
      DBFS_DG/ 367220 138432 62.303
      RECO1/ 9078588 3392160 62.636

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s