Oracle Storage Subsystems
RAID Configurations

General Information
Note: The following, unedited, was posted by Cary Milsap of Method-R to on 3-Aug-2004. It says much of what needs to be said ... and very succinctly. Only new technology such as Apple's Xserve RAID changes what has been said below. Before you "trust" your system and storage admins, the sales team that is tryin desparately to convince them to purchase their product d'jour ... keep this in mind as you, the DBA, will pay the price if they are wrong.
My argument is this:

1. Mirroring is excellent for resilience. Striping is excellent for performance (as long as the striping is done with large enough granularity as not to introduce a concurrency problem). Combining the two technologies makes for excellent resilience and excellent performance.

2. But mirroring is expensive per byte of storage. Hence RAID levels 2, 3, 4, 5, and 6 were proposed. Their design goals were to lessen the expense-per-byte of storage of RAID level 1 (mirroring). For example, with G=5 RAID level 5, the price of resilience per byte of storage is 5/4 of a 4-disk array instead of 8/4.

3. But the additional parity operations required to run RAID [23456] produce a performance penalty. The performance penalty is so bad (4:1 for many common Oracle operations) that it begged for a solution: namely, the introduction of non-volatile cache and a complicated kernel (more lines of source code than Oracle 7.1) to run it all. So, as long as the cache is big enough to stay ahead of your sustained throughput, a RAID level 5 array can perform almost as well as a well-configured array that uses levels 0 and 1 combined.

4. But the RAID level 5 array is deficient in a number of important ways (performance degradation during partial outage as Howard mentioned, worse availability performance, worse read performance [contrary to popular belief], inability to take online backups with a simple re-silver operation, etc.). And the cache and microcode make the RAID level 5 array more expensive than the 0+1 or 1+0 array, which defeats the whole purpose of inventing RAID level 5 in the first place.

I'm not telling you not to use RAID level 5 (we use it for low-throughput systems with flexible availability requirements). Just don't assume that it's cheaper than using RAID levels 0 and 1 for high-throughput systems. The insidious thing is that your initial procurement cost is probably cheaper if you use RAID level 5, but if you don't figure out in your initial capacity plan that "disk sizing" means counting I/O calls, not counting bytes of storage, then you're in for a lot more expense than you thought, if you go the RAID level 5 route.

By the way, did you notice in the original Chen/Katz/Gibson/Patterson/Lee paper that block-striped RAID level 1 is mathematically identical to RAID level 5 if you set G=2? So, in a sense, I'm a big proponent of RAID level 5. I just don't like the idea, for high throughput systems, of using parity group sizes bigger than 2 or buying the cache and microkernel.

Cary Millsap
RAID Level Description
0 Striping: The easy way to remember that is that RAID 0 adds no redundancy. In fact, if you stripe disks to improve performance, you will have worse availability than if you just use disks without RAID. That's because if you stripe, the loss of any disks in the stripe will cause the entire stripe to fail.
1 Mirroring: All of the data on one disk is copied exactly onto a second disk. Neither disk is the master or primary; the disks are peers. For writes to be deemed complete, they must make it to both disks. If one disk fails, its partner keeps right on running, without interruption. The good news with RAID-1 is that it's very easy to manage and it does not require significant levels of CPU for normal operations or for recovery. The downside to RAID-1 is the expense: For every gigabyte of disk you wish to protect, you need a second, matching gigabyte. In other words, RAID-1requires twice as much disk space as unprotected disks.
2 Hamming code error correction: RAID-2 uses the same hamming encoding method for checking the correctness of disk data that error correcting code memory (ECC) uses. I have never seen or heard of a single commercial implementation of RAID-2. I include it here only for completeness and because if I left it out, someone would write to ask me about it.
3 Virtual disk blocks: In RAID-3, every write is split (striped) across all of the disks (usually four or more) in the RAID array. Since every write touches every disk, the array can only be writing one block of data at a time that can cause poor performance from the RAID. RAID-3 performance varies based on the nature of the writes: Small writes scattered all over the disks will show very poor performance. Larger sequential writes will result in better performance.
4 Dedicated parity disk: In a RAID-4 array, there is a set of data disks, usually 4 or 5 (although there could be more, at a significant performance penalty), plus one extra disk that is dedicated to managing the parity for the data on the other disks. Since all writes must go through the parity disk, that disk becomes a performance bottleneck slowing down all write activity to the entire array.
5 Striped parity: RAID-5 is virtually identical to RAID-4 except that instead of all of the parity being concentrated on a single disk, the parity is divided up, with a share being given to each disk in the array. This sharing will balance and reduce the performance impact that is evident in RAID-4 implementations. In software implementations of RAID-5, which are fairly common, performance will often become unacceptably slow if writes make up any more than about 15% of disk activity.
6 Dual parity: RAID-6 takes the striped parity region from RAID-5 and duplicates it. Each disk in a RAID-6 stripe has two parity regions, each calculated separately. The advantage of RAID-6 is that even if two disks fail, the RAID stripe would still have a complete set of data and be able to recover the failed disks. The biggest disadvantage is in performance. The performance impact of RAID-5 is roughly doubled as each set of parities is calculated separately. Where RAID-5 requires one extra disk for parity RAID-6 requires two, increasing the cost of the implementation.
7 RAID-7 is a trademarked term, owned by Storage Computer Corporation. They appear to have the only commercial implementation that involves proprietary disk array hardware with an internal CPU and a combination of striping and a RAID-5-like model.
S EMC Symmetrix disk arrays offer an alternate, proprietary method for parity RAID that they call RAID-S. While detailed documentation on RAID-S is proprietary, it appears to be similar to RAID-5 with some performance enhancements as well as the enhancements that come from having a high-speed disk cache on the disk array.
5+0 By building a series of RAID-5 groups and then striping them in RAID-0 fashion, you can improve RAID-5 performance without reducing data protection.
5+3 It would be more accurate to call this RAID 0+3 but it has come to be called RAID 5+3 and it's defined by striping (in RAID-0 style) RAID-3's virtual disk blocks.
10 Combining RAID-0 and RAID-1 is often referred to as RAID-10, and it comes in two different forms. In RAID-0+1, the data is organized as stripes across multiple disks, and then the striped disk sets are mirrored. In RAID-1+0, the data is mirrored and the mirrors are striped.
Drive # RAID Configuration
1 2 3 4 JBOD RAID 0 RAID 1 RAID 0+1 RAID 1+0 RAID 5
O O O O online online online online online online
Single Drive Failure
O O O X fail fail online online online online
O O X O fail fail online online online online
O X O O fail fail online online online online
X O O O fail fail online online online online
Two Drive Failures
X X O O fail fail online online fail fail
X O X O fail fail online fail online fail
X O O X fail fail online fail online fail
O X X O fail fail online fail online fail
O X O X fail fail online fail online fail
O O X X fail fail online online fail fail
Three Drive Failures
X X X O fail fail online fail fail fail
X X O X fail fail online fail fail fail
X O X X fail fail online fail fail fail
O X X X fail fail online fail fail fail
Four Drive Failures
X X X X fail fail fail fail fail fail
X' means the drive failed, while 'O' means the drive is online.
In RAID 0+1, drives in pairs (1,2) and (3,4) are striped, and then the pairs are mirrored.
In RAID 1+0 or 10, drives in pairs (1,2) and (3,4) are mirrored, and then the pairs are striped.

JBOD The simplest way to configure a disk drive array is to combine multiple disks together so that they look like a single logical drive to the system. This technique can be used where disk space is at a premium and there are many drives of varying capacities to be used. The total capacity is the sum of all the drives, and the performance is equal to the performance of the drive that the controller is accessing. If a single disk fails the entire array goes offline. The data becomes unrecoverable even though data files may be intact on the remaining drives. This most basic RAID configuration is called Just a Bunch of Disks (JBOD), and is supported by virtually every RAID controller including the FastTrak SX4.

A way to increase performance is to configure the controller so that it stores data in stripes across two or more drives. With data split up across the drives, reads and writes can occur faster because the controller can read from and write to multiple drives at once. Under certain ideal conditions, the reading and writing can occur at a speed proportional to the number of drives in the array. For example, with two drives, one can achieve twice the performance of a single drive under certain conditions. With a striped array, the total capacity is the capacity of the smallest drive times the number of drives. For example, two striped 80GB drives form a single, faster 160GB drive. A major drawback of striping is that a single drive failure causes total data loss. The remaining drives only contain pieces of files, and there is no software with the controller to recover even the pieces. The striped drive configuration is called RAID-0 (pronounced raid zero).

One way to increase reliability is to configure the controller so that it mirrors data to two or more drives to make a single logical drive with data redundancy. If one physical drive fails, a perfect mirror of the data will be on the surviving physical drives, and you can continue using the data without interruption. After a replacement is drive is installed, the array can be rebuilt by the controller, which means copying the data from the surviving drives onto the replacement drive. Because data is written to and read from synchronously, the performance is no faster than the slowest drive in the array. The total amount of storage available in a mirrored array will be limited to the capacity of the smallest drive, regardless of how many drives are mirrored. Mirrored drives are said to be in a RAID-1 (pronounced raid one) configuration.
Hard drives are mechanical devices and susceptible to many kinds of failure.
  • manufacturing defects
  • mishandling during shipping or installation
  • EMI, magnetic fields, static electricity
  • operating environment - power fluctuations, temperature, vibration, shocks, bad software
  • wear and tear - seeking, spinning, on/off cycling
Striping drives significantly decreases reliability, more so than commonly expected. Striped arrays should not be used to store important information because there is a very high probability of failure. On the other hand, mirroring drives significantly increases reliability, and a two drive mirrored array will tend to survive a long time. Precise calculations of reliability in multi-drive arrays is complex enough that teams at IBM research it, and an alternative would be a policy of preparing for the worst and then deriving reliability information from experience in the particular environment at hand.

For more information, you can read the Hard Disk Reliability Article and try their online Calculator. In the example at the end of the article, the three disk RAID-0 array is 10,000 times more likely to fail in a given month than the three disk RAID-1 array. In fact, the odds of the three disk RAID-0 array failing in a given month are an incredible one in twenty. Remember that other parts of the system can fail besides just the drives, including the controllers and the cables themselves.
Both RAID-0 and RAID-1 require the synchronization of disk drive heads. Seek time of a disk drive is the time it takes for the read/write head to position itself over the right track on the disk, and for the disk to spin to the position of the data. If disks are of different speeds, then seek time in an array will be lowered to that of the slowest drive. Seek time is important for applications that require reading and writing to many random locations, such as file servers. Neither RAID-0 nor RAID-1 have any effect on seek time. Seek time can be improved by faster spinning disks or disks with greater data density.

Another aspect of drive performance is sustained transfer rate. This is the amount of contiguous data that can be read or written over a certain period of time. Since it is usually determined for large blocks of data, it indicates performance for applications that manipulate large data files, such as image and video files. Striping can multiply sustained transfer rate by the number of drives in the striped array, whereas mirroring has no effect on transfer rate.
Morgan's Library Page Footer
This site is maintained by Dan Morgan. Last Updated: This site is protected by copyright and trademark laws under U.S. and International law. © 1998-2014 Daniel A. Morgan All Rights Reserved