ORA-15032: Not All Alterations Performed - Fix ASM Rebalance

ORA-15032: Not All Alterations Performed

Error Overview

Error Text: ORA-15032: not all alterations performed

The ORA-15032 error is a wrapper error that signals an ALTER DISKGROUP statement completed only partially. Oracle ASM reports this error when one or more sub-operations within a compound diskgroup alteration could not be applied — most commonly during disk add/drop sequences, attribute changes, or rebalance operations that encounter a disk or space problem mid-flight. The error is almost always accompanied by a secondary ORA- error (such as ORA-15040, ORA-15041, or ORA-15042) that identifies the root cause.

Common Causes

1. Partial Disk Add Failures

• One disk in a multi-disk ADD statement cannot be opened (ORA-15025)
• The new disk header is unreadable or already belongs to another diskgroup
• Insufficient space in the diskgroup to absorb the rebalance triggered by the ADD
• Disk device path not visible on all cluster nodes simultaneously

2. Partial Disk Drop Failures

• Dropping a disk would leave the diskgroup without enough redundancy
• Rebalance triggered by DROP cannot complete because free space is exhausted (ORA-15041)
• A disk being dropped is in OFFLINE state and repair timer has not expired
• Concurrent rebalance from a previous operation is still running (ORA-15032 on overlap)

3. Rebalance Operation Aborted Mid-Flight

• Oracle instance crashed or was shutdown while rebalance was active
• A disk became inaccessible during rebalance (ORA-15040 / ORA-15042)
• Rebalance power set too low, causing timeout on RAC rolling patch window
• Disk I/O errors detected during extent migration causing operation abort

4. Diskgroup Attribute Changes Rejected

• Compatibility attribute downgrade attempted (ASM never allows version downgrade)
• ALTER DISKGROUP ... ATTRIBUTE applied to a subset of disks in a failure group
• Invalid attribute value supplied for AU_SIZE, SECTOR_SIZE, or COMPATIBLE.ASM

5. Concurrent DDL Conflicts

• Two sessions simultaneously altering the same diskgroup
• A running rebalance blocking a new ALTER DISKGROUP DROP DISK statement
• RAC node joining or leaving the cluster while a diskgroup alteration is in progress

6. Insufficient Failure Group Redundancy

• HIGH REDUNDANCY diskgroup would drop below 3 failure groups after operation
• NORMAL REDUNDANCY diskgroup would drop below 2 failure groups after operation
• EXTERNAL REDUNDANCY diskgroup disk drop leaving it with no data copies

Diagnostic Queries

Check the Current Diskgroup State

-- Full diskgroup summary including redundancy and disk counts
SELECT
  group_number,
  name,
  state,
  type,
  total_mb,
  free_mb,
  usable_file_mb,
  ROUND((1 - free_mb/NULLIF(total_mb,0))*100, 1) AS pct_used,
  offline_disks,
  voting_files
FROM v$asm_diskgroup
ORDER BY name;

-- Detailed disk status — identify any disks in abnormal states
SELECT
  dg.name          AS diskgroup,
  d.disk_number,
  d.name           AS disk_name,
  d.path,
  d.header_status,
  d.mount_status,
  d.mode_status,
  d.state,
  d.repair_timer,
  d.failgroup,
  ROUND(d.total_mb/1024, 2) AS total_gb,
  ROUND(d.free_mb/1024, 2)  AS free_gb,
  d.read_errs,
  d.write_errs
FROM v$asm_disk d
LEFT JOIN v$asm_diskgroup dg ON d.group_number = dg.group_number
ORDER BY dg.name, d.disk_number;

Monitor Active Rebalance Operations

-- Check whether a rebalance is running and its progress
SELECT
  dg.name                   AS diskgroup,
  op.operation,
  op.state,
  op.power,
  op.actual,
  op.sofar,
  op.est_work,
  op.est_rate,
  op.est_minutes,
  ROUND(op.sofar/NULLIF(op.est_work,0)*100, 1) AS pct_done
FROM v$asm_operation op
JOIN v$asm_diskgroup dg ON op.group_number = dg.group_number
ORDER BY dg.name;

-- Historical rebalance operations (requires AWR license)
SELECT
  begin_time,
  end_time,
  group_number,
  operation,
  state,
  power,
  est_minutes
FROM v$asm_operation
ORDER BY begin_time DESC;

Identify the Root Cause Secondary Error

-- Query ASM disk errors that may have triggered ORA-15032
SELECT
  d.name,
  d.path,
  d.mount_status,
  d.mode_status,
  d.state,
  d.read_errs,
  d.write_errs,
  d.repair_timer
FROM v$asm_disk d
WHERE d.read_errs + d.write_errs > 0
   OR d.mode_status != 'ONLINE'
   OR d.state NOT IN ('NORMAL', 'UNKNOWN')
ORDER BY d.read_errs + d.write_errs DESC;

-- Check for disks in DROPPING state (stuck drop operation)
SELECT
  dg.name AS diskgroup,
  d.name  AS disk_name,
  d.path,
  d.mode_status,
  d.state,
  d.repair_timer
FROM v$asm_disk d
JOIN v$asm_diskgroup dg ON d.group_number = dg.group_number
WHERE d.state = 'DROPPING'
   OR d.mode_status = 'OFFLINE';

Examine Diskgroup Attributes

-- View all diskgroup attributes to verify compatibility levels
SELECT
  dg.name AS diskgroup,
  a.name  AS attribute_name,
  a.value
FROM v$asm_attribute a
JOIN v$asm_diskgroup dg ON a.group_number = dg.group_number
WHERE a.name IN (
  'compatible.asm',
  'compatible.rdbms',
  'au_size',
  'sector_size',
  'disk_repair_time',
  'content.type',
  'redundancy.type'
)
ORDER BY dg.name, a.name;

-- Failure group summary — check minimum redundancy thresholds
SELECT
  dg.name       AS diskgroup,
  d.failgroup,
  COUNT(*)      AS disk_count,
  SUM(d.total_mb) AS total_mb
FROM v$asm_disk d
JOIN v$asm_diskgroup dg ON d.group_number = dg.group_number
WHERE d.mount_status = 'CACHED'
GROUP BY dg.name, d.failgroup
ORDER BY dg.name, d.failgroup;

Step-by-Step Resolution

Step 1: Read the Full Error Stack

ORA-15032 is always accompanied by a secondary error. Check the alert log immediately after the failure:

# Locate the ASM alert log
find $ORACLE_BASE/diag/asm -name "alert_+ASM*.log" 2>/dev/null | head -5

# Extract the error context around ORA-15032
grep -B5 -A10 "ORA-15032\|15032" \
  $ORACLE_BASE/diag/asm/+asm/+ASM1/trace/alert_+ASM1.log | tail -60

The secondary error determines the correct resolution path:

• ORA-15025 — disk cannot be opened: see ORA-15025
• ORA-15040 — diskgroup incomplete: see ORA-15040
• ORA-15041 — insufficient space: see ORA-15041
• ORA-15042 — disk missing: see ORA-15042
• ORA-15063 — too few disks: see ORA-15063

Step 2: Assess Current State of the Diskgroup

-- Is the diskgroup mounted?
SELECT name, state, offline_disks FROM v$asm_diskgroup;

-- Are there disks in unexpected states?
SELECT name, path, mode_status, state, repair_timer
FROM v$asm_disk
WHERE mode_status != 'ONLINE' OR state != 'NORMAL';

Step 3: Cancel a Stuck Rebalance (If Applicable)

-- Determine the group_number of the stuck diskgroup
SELECT group_number, name FROM v$asm_diskgroup WHERE name = 'DATA';

-- Cancel the rebalance by setting power to 0
ALTER DISKGROUP data REBALANCE POWER 0;

-- Confirm the operation stopped
SELECT operation, state FROM v$asm_operation;

Step 4: Resolve Redundancy Shortfall Before Dropping Disks

If the drop failed because it would violate minimum redundancy:

-- Check how many failure groups exist
SELECT failgroup, COUNT(*) disk_count
FROM v$asm_disk
WHERE group_number = (SELECT group_number FROM v$asm_diskgroup WHERE name = 'DATA')
  AND mount_status = 'CACHED'
GROUP BY failgroup;

-- Add a replacement disk before retrying the drop
ALTER DISKGROUP data ADD DISK '/dev/mapper/asm_new01' NAME NEW_DISK_01
  REBALANCE POWER 8;

-- After rebalance completes, drop the target disk
ALTER DISKGROUP data DROP DISK OLD_DISK_01 REBALANCE POWER 8;

Step 5: Retry the Alteration with Higher Rebalance Power

-- Retry an ADD operation with explicit rebalance power
ALTER DISKGROUP data
  ADD DISK '/dev/mapper/asm_data05' NAME DATA_0005,
           '/dev/mapper/asm_data06' NAME DATA_0006
  REBALANCE POWER 11;

-- Retry a DROP with explicit power
ALTER DISKGROUP data DROP DISK DATA_0002 REBALANCE POWER 11;

Step 6: Online Offline Disks Before Retrying

-- If a disk went OFFLINE and repair timer is still valid, bring it online
ALTER DISKGROUP data ONLINE DISK DATA_0003;

-- If multiple disks are offline
ALTER DISKGROUP data ONLINE ALL;

-- Verify before retrying the alteration
SELECT name, mode_status, state, repair_timer
FROM v$asm_disk
WHERE group_number = (SELECT group_number FROM v$asm_diskgroup WHERE name = 'DATA');

Step 7: Tune Rebalance Power for Large Operations

-- Increase rebalance power system-wide (1–1024, default is 1)
ALTER SYSTEM SET asm_power_limit = 11 SCOPE=BOTH;

-- Or set at the diskgroup level during an operation
ALTER DISKGROUP data REBALANCE POWER 8;

-- Monitor until completion
SELECT operation, state, sofar, est_work, est_minutes
FROM v$asm_operation;

Prevention Strategies

1. Pre-Check Space Before Disk Drops

-- Estimate whether a drop is safe before executing it
-- usable_file_mb accounts for mirroring overhead
SELECT
  name,
  type,
  total_mb,
  free_mb,
  usable_file_mb,
  -- Rough estimate: if dropping a disk reduces usable space below 15%, caution
  ROUND(usable_file_mb / NULLIF(total_mb, 0) * 100, 1) AS usable_pct
FROM v$asm_diskgroup
WHERE name = 'DATA';

2. Monitor All Running Operations

-- Scheduled job to detect stuck rebalance
CREATE OR REPLACE PROCEDURE check_asm_rebalance AS
  v_stuck NUMBER;
BEGIN
  SELECT COUNT(*)
  INTO   v_stuck
  FROM   v$asm_operation
  WHERE  state = 'RUN'
    AND  est_minutes > 240  -- Running more than 4 hours
    AND  est_rate   < 1;    -- Essentially stalled

  IF v_stuck > 0 THEN
    RAISE_APPLICATION_ERROR(-20002,
      'ASM rebalance appears stalled. Check V$ASM_OPERATION immediately.');
  END IF;
END;
/

BEGIN
  DBMS_SCHEDULER.CREATE_JOB(
    job_name        => 'CHECK_ASM_REBALANCE',
    job_type        => 'STORED_PROCEDURE',
    job_action      => 'check_asm_rebalance',
    repeat_interval => 'FREQ=MINUTELY;INTERVAL=15',
    enabled         => TRUE
  );
END;
/

3. Use NOFORCE Drops with Explicit Power

Always specify REBALANCE POWER explicitly for large disk drops to avoid the default power of 1 (extremely slow) stalling during maintenance windows:

-- Template for a safe disk drop with monitoring
ALTER DISKGROUP data DROP DISK DATA_0002 REBALANCE POWER 8 WAIT;

-- The WAIT clause causes the ALTER DISKGROUP to block until complete,
-- making scripted operations easier to validate

4. Best Practices

• Before any multi-disk ADD or DROP, verify usable_file_mb shows at least 20% headroom
• Always check v$asm_operation for existing operations before issuing a new alteration
• In RAC environments, ensure all nodes can see all disks before adding them to a diskgroup
• Set disk_repair_time attribute appropriately for your SAN — default 3.6 hours may be too short for some storage failovers
• Keep a record of all disks in each diskgroup, including their device paths and failure group assignments, as a recovery reference

Related Errors

• ORA-15001 — Diskgroup does not exist or is not mounted
• ORA-15017 — Diskgroup cannot be mounted
• ORA-15025 — Could not open disk (disk access failure)
• ORA-15040 — Diskgroup is incomplete (common secondary error)
• ORA-15041 — Diskgroup space not sufficient (common secondary error)
• ORA-15042 — ASM disk is missing from diskgroup
• ORA-15063 — ASM insufficient number of disks in diskgroup

Emergency Response

Quick Assessment

1. Read the secondary error — ORA-15032 is never the root cause; it is always paired with another ORA- code

   grep -A3 "ORA-15032" $ORACLE_BASE/diag/asm/+asm/+ASM1/trace/alert_+ASM1.log | tail -20

2. Check diskgroup state immediately

   SELECT name, state, offline_disks, free_mb, usable_file_mb
   FROM v$asm_diskgroup;

3. Cancel any stuck rebalance

   ALTER DISKGROUP data REBALANCE POWER 0;

Post-Resolution Verification

-- Confirm all disks are healthy after resolution
SELECT
  dg.name AS diskgroup,
  d.name  AS disk_name,
  d.mode_status,
  d.state,
  d.read_errs,
  d.write_errs
FROM v$asm_disk d
JOIN v$asm_diskgroup dg ON d.group_number = dg.group_number
ORDER BY dg.name, d.disk_number;

-- Confirm no operations are still running
SELECT * FROM v$asm_operation;

-- Final space check
SELECT name, state, free_mb, usable_file_mb,
  ROUND(usable_file_mb/NULLIF(total_mb,0)*100,1) AS usable_pct
FROM v$asm_diskgroup;