ORA-1115: I/O ERROR READING BLOCK [ID 1013621.6]

"Checked for relevance on 17-Jul-2012"

Problem Description: 
==================== 

An ORA-01115 is issued whenever Oracle is unable to read from an open 
datafile because of an I/O error: 

 ORA-01115: "IO error reading block from file %s (block # %s)" 
     Cause: Device on which the file resides is probably offline 
    Action: Restore access to the device 

ORA-01115 errors are usually followed by: 

- an ORA-01110 error
- an operating system level Oracle error message such as ORA-0737X 
- an operating system error (e.g., error# 5 in Unix) 

Solution Description: 
===================== 

Because most ORA-01115s are caused by hardware problems, the solution 
consists in first isolating those, and then addressing the problem at 
the database level, if necessary. 

PERFORMING HARDWARE CHECKS IS ESSENTIAL.  If hardware problems are not 
fixed, trying to solve the problem at the database level will be useless. 
Run operating system level utilities and diagnostic tools that check 
for the sanity of disks, controllers, and the I/O subsystem.  Pay special 
attention to the disk where the datafile referenced in the ORA-01115 resides. 
Your system administrator should be able to assist you in this task. 
Such diagnostics should be done in parallel with the steps 
recommended here, if feasible, or as soon as possible thereafter. 

Determining the exact cause of an ORA-01115 is not always trivial. 
Approaches differ according to whether you know the cause of the problem 
or not. 

I. STEPS FOR SOLVING THE PROBLEM WHEN THE CAUSE IS NOT KNOWN 
------------------------------------------------------------ 

1. Try to assess the cause and extent of the problem. 

        Examine the alert.log file for this instance.  Scan the last few 
        days' entries for other occurrences of ORA-01115. If you find them, 

        A) Do they reference files in different disks? 
                If so, it is likely that there you have controller problems. 
                Move on to Scenario II.A below. 
        B) Do they reference different files in the same disk? 
                If so, it is likely that there are problems with that disk. 
                Move on to Scenario II.B below. 
        C) Do they always reference the same datafile? 
                If so, it is likely that the datafile contains bad blocks. 
                Move on to Scenario II.C below.  If the file is bigger 
                than 2GB and you are running 7.1.4 or lower on a Solaris 
                platform, see Scenario II.D below. 
        D) If none of the above applies, move on to Step 2. 

2. If the datafile is in the SYSTEM tablespace, or the database is in 
   NOARCHIVELOG mode, shut the database down. Move on to Step 4. 

        If shutdown immediate fails, do a shutdown abort. 

3. If the database is in ARCHIVELOG mode, you should still shut the 
   database down.  If the database cannot be shut down, offline the 
   datafile. 

        ALTER DATABASE DATAFILE '<full_path_file_name>' OFFLINE; 

4. Try to copy the datafile to another disk (managed by a different 
   controller, if possible). 

5. If the copy fails, even after you retry, the datafile must be considered 
   lost at this point.  The next action depends on the tablespace to 
   which the lost file belongs. See the following Solution References to 
   PR entries, according to the different types of tablespaces, 
   for instructions on how to proceed. 

   IMPORTANT: While going through the references below, keep in mind that 
   if you restore the datafile from backup, you need to place it in another 
   disk, preferably under a different controller, and rename it inside Oracle 
   (see Note:115424.1 for details).  If you recreate any tablespace, make 
   sure its datafiles are created in another disk, preferably under a 
   different controller. 

   Reference for Lost datafile scenarios : Note:198640.1

6. If the database is down, mount it. 

7. Rename the datafile that you succeeded in copying inside Oracle. 

        ALTER DATABASE RENAME FILE '<old_full_path_file_name>' 
        TO '<new_full_path_file_name>'; 

8. If the database is mounted, open it.  If you offlined the datafile, 
   perform media recovery on it, and then bring it online. 

        RECOVER DATAFILE '<full_path_file_name>'; 

        ALTER DATABASE DATAFILE '<full_path_file_name>' ONLINE; 

II. STEPS FOR SOLVING THE PROBLEM WHEN THE CAUSE IS KNOWN 
--------------------------------------------------------- 

II.A  CONTROLLER PROBLEMS 
------------------------- 

These are typically intermittent.  Usually, there is no damage to the 
datafiles.   Unless you can quickly fix the controller and restore 
access to the datafile, follow these steps: 

1. Find out which datafiles are under the bad controller. 

        Query V$DATAFILE to obtain the names of all datafiles in the 
        database. You may need the help of the system administrator 
        to determine which datafiles reside in disks managed by this 
        controller. 

2. If any of the datafiles under the bad controller belongs to the SYSTEM 
   tablespace, or if the database is in NOARCHIVELOG mode, shut the database 
   down.  Move on to step 4. 

        If shutdown immediate fails, do a shutdown abort. 

3. If the database is in ARCHIVELOG mode and none of the datafiles under 
   the bad controller are in the SYSTEM tablespace, you should shut the 
   database down.  If the database cannot be shut down, 
   offline all the datafiles under the bad controller. 

        ALTER DATABASE DATAFILE '<full_path_file_name>' OFFLINE; 

4. Try to copy all the datafiles under the bad controller to disks 
   managed by different controllers. 

5. If the database is down, mount it. 

6. Rename all the files that you succeeded in copying inside Oracle. 

        ALTER DATABASE RENAME FILE '<old_full_path_file_name>' 
        TO '<new_full_path_file_name>'; 

7. If the copy fails for one or more of the datafiles, even after you retry 
   copying them, those datafiles have to be considered lost at this point. 
   See the following Solution References to PR entries, according to the 
   tablespaces to which the lost datafiles belong, for instructions on 
   how to proceed. 

   IMPORTANT: While going through the references below, keep in mind that if 
   you restore datafiles from backup, you need to place them in disks under 
   other controllers and rename them inside Oracle (see Note:115424.1 for 
   details). If you recreate any tablespace, make sure its datafiles are 
   created under other controllers. 

   Reference for Lost datafile scenarios : Note:198640.1 

8. If the database is mounted, open it.  If any of the moved datafiles 
   is offline, apply media recovery to it, and then online it: 

        RECOVER DATAFILE '<full_path_file_name>'; 

        ALTER DATABASE DATAFILE '<full_path_file_name>' ONLINE; 

II.B  DISK PROBLEMS 
------------------- 

If you know for a fact that a disk has bad blocks or is malfunctioning, 
you should focus on moving the datafiles in it to a different disk, if at 
all possible.  If not, you must consider the files lost and address the 
issue according to the tablespaces to which they belong, while working 
in parallel on fixing the disk.  The steps to follow in this scenario 
are analogous to those in Scenario II.A above. 

II.C  DATA BLOCK CORRUPTION 
--------------------------- 

If you are sure that the datafile has bad blocks, it should be considered LOST 
if it belongs to the SYSTEM tablespace or to a ROLLBACK or READ-ONLY tablespace.
See the following References, depending on the tablespace to which the datafile 
belongs. 

   IMPORTANT: While going through the references below, keep in mind that if 
   you restore datafiles from backup, you need to place them in different disks
   (preferably under other controllers) and rename them inside Oracle (see the 
   Note:115424.1 for details). If you recreate any tablespace, make sure its
   datafiles are created on different disks (preferably under other 
   controllers). 

   Reference for Lost datafile scenarios : Note:198640.1

If the datafile belongs to a user or index tablespace,  you may also 
address the problem as an object recreation issue if the ORA-01115 
occurs consistently against the same objects (tables, indexes, etc.). 
The following query returns the object in which the bad block is: 

        SELECT SEGMENT_NAME, SEGMENT_TYPE FROM DBA_EXTENTS 
        WHERE FILE_ID = <file_number> and <block_number> BETWEEN BLOCK_ID 
        AND BLOCK_ID + BLOCKS - 1; 

where <file_number> and <block_number> are those listed in the ORA-01115. 
If this query consistently points to a table or index, you may try 
recreating them, if possible in a different tablespace. 

II.D  VERY LARGE DATAFILE PROBLEMS ON SOLARIS 
--------------------------------------------- 

If you are running Oracle 7.1.4 or lower on a Solaris platform, and 
you get an ORA-07371 with the ORA-01115, and the file is 
bigger than 2GB, you are very likely running into bug:233569. 
This bug is fixed in 7.1.6 and patches are available for 7.1.3 
(bug:233569) and 7.1.4 (bug:281904).

Explanation: 
============ 

What causes ORA-01115 error? 
---------------------------- 

Oracle hands over read from file requests to the underlying operating system 
(except if raw devices are being used).  A read request specifies a 
datafile and a block number to be accessed.  If a low-level I/O error 
prevents the read from completing successfully, Oracle signals an 
ORA-01115. 

The main causes for an ORA-01115 are: 

1. HARDWARE PROBLEMS 

- Disk controller problems: the most common, and usually intermittent. 

- Disk problems: these include bad blocks, disk malfunctioning, etc. 

2. DATA BLOCK CORRUPTION (AT THE PHYSICAL LEVEL) 

Usually caused by previous hardware problems. 

3. PROBLEMS HANDLING VERY LARGE DATAFILES 

In Oracle 7.1.4 and lower on Sun Solaris, bug:233569 causes ORA-01115 
and ORA-07371 when handling datafiles bigger than 2GB. 

Typical scenarios where ORA-01115 can happen include: 

- On execution of DML statements 

- During exports or imports 

- At startup or shutdown

References:
===========

Bug:233569

Note: 184327.1  Common Causes and Solutions on ORA-1157 Error Found in 
                  Backup & Recovery

Note:115424.1   HOW TO RENAME OR MOVE DATAFILES AND LOGFILES

Search Words:
=============

ORA-1115 ORA-7371

References

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s