Software RAID - Penguin Magazine

Software RAID

Introduction to RAID
Written by: Scott R. Haven

RAID is defined as a "Redundant Array of Inexpensive Disks." Simply put, it is the process of combining two or more partitions to create a single logical device. This logical device will then be used in place of the combined partitions.

While RAID can be used in many different configurations, known as "modes", we will be covering only the most popular configurations in this article.

RAID 0 (striping)

Explanation:
Two or more partitions are combined to form one logical unit, which will then be the sum of its physical partitions. Data written to the logical unit will be dispersed among the physical partitions.
Example:

/dev/sda1 ( a 1 gig partition )
+
/dev/sdb1 ( a 2 gig partition )

/dev/md1 (a 3 gig logical RAID device)
Advantages:

High performance. Reading and writing to the RAID device is done in parallel to all the partitions participating in the RAID.

No wasted disk space. As this mode provides no data redundancy or parity, all available disk space remains usable to the filesystem.

Large volumes. By combining many disks into a single RAID stripe, you can create a volume, which is larger than any single disk available.

Disadvantages:

Decreased reliability.. If any one partition in the RAID should fail, the entire RAID device will become unusable.

Common pitfalls:

Using disks of varying speeds or sizes, may decrease overall performance.

RAID 1 (mirroring)

Explanation:
Two or more equally sized partitions are combined into one logical device. Data is written to all of the partitions simultaneously, and read from the partitions in parallel. Each partition is a duplicate, or "mirror," of the others.
Example:

/dev/sda1 ( a 1 gig partition )
+
/dev/sdb1 ( a 1 gig partition )

/dev/md1 (a 1 gig logical RAID device)
Advantages:

Reliability. The RAID device will not fail so long as any one of the partitions remains usable.

Availability. The performance of the RAID device does not decrease substantially when operating in a degraded mode.

Low hardware cost. RAID mirrors typically consist of only two disk drives. This makes RAID 1 a good choice where limitations on the number of disks exist. (such as IDE)

Simplicity. Mirrors are the easiest RAID configuration to setup and maintain.

Disadvantages:

Lower capacity. Since each partition is a mirror of the others, that disk space cannot be used to store additional data.

Common pitfalls:

RAID 1 is best used only when the number of disks is limited. RAID 5 is a more efficient use of disk space.

RAID 5 (striping with parity)

Explanation:
Several equally sized partitions are combined to form one logical device. Data and parity information is dispersed among the available partitions. Parity data is divided equally among the partitions. Should one of the partitions be come unusable, the missing data will be reconstructed from the remaining partitions.
Example:

/dev/sda1 ( a 1 gig partition )
+
/dev/sdb1 ( a 1 gig partition )
+
/dev/sdc1 ( a 1 gig partition )
+
/dev/sdd1 ( a 1 gig partition )

/dev/md1 (a 3 gig logical RAID device)
1 gig of disk space is used to store the parity information.

Advantages:

Reliability. The RAID device will remain operational should any one disk fail.

Efficient use of disk space. Unlike RAID 1, the amount of unusable disk space does not increase with the number of disks.

Large volumes. Combining many disks allows you to create a very large volume, larger than any single disk available.

Disadvantages:

Complexity. RAID 5 is perhaps the most difficult mode to understand, and also the most difficult to configure.

Lower availability. Should a disk fail, the performance of the RAID 5 device will diminish, as it reconstructs missing data from the parity information. When the failed disk is replaced, the system will rebuild the data on the new partition, a time consuming process.

Common pitfalls:

RAID 5 is best used with a large number of disks. Many small disks provide better performance than a few large disks, while leaving more usable disk space.

Setting Up RAID 0

WARNING: The instructions below involve creating partitions, and if done incorrectly may cause serious data loss or cause your system to become unusable.

These instructions are only for systems with kernel 2.2 or latter, as previous kernels used different tools and did not have the ability to autodetect raid partitions.

In the following example we will be using two disks, /dev/sdb and /dev/sdc. Neither of these disks contain any data we wish to preserve. For your system you will need to substitute "/dev/sdb" and "/dev/sdc" with the appropriate device names for your system in the instructions below.

Login as root.

Create Partitions

Now we will create the partitions for our RAID stripe using fdisk.

fdisk /dev/sdb

At the prompt:

n ( new partition )
p ( primary )
1 ( partition number 1 )
enter ( first cylinder, default is 1 )
enter ( last cylinder )
enter ( first cylinder, default is 1 )
t ( to change a partition type )
1 ( change type for partition 1 )
fd ( change type to "fd" Linux raid autodetect )
w ( Write partition table and exit )

fdisk /dev/sdc

At the prompt:

/etc/raidtab

In order to create a RAID device, we must first set up the raidtab file. On most systems this file does not already exist, so we will need to create it manually.

Open /etc/raidtab in a text editor.

Insert the following:

raiddev /dev/md1
raid-level 0
nr-raid-disks 2
nr-spare-disks 0
chunk-size 4
persistent-superblock 1
device /dev/sdb1
raid-disk 0
device /dev/sdc1
raid-disk 1

Create RAID device

Now that our RAID device(s) are defined, we will use the mkraid command to create them.

mkraid /dev/md1

Create filesystem

Create a filesystem on the RAID device.

mkfs /dev/md1

Mount filesystem

If you do not already have a mount point, create one now.

mkdir /stripe

and mount the filesystem.

mount /dev/md1 /stripe

See also:

man 5 raidtab

mkraid

Mkraid will complain if any of the partitions defined in your /etc/raidtab file already contain a filesystem. If this is the case, it will instruct you how to force it to create the RAID device.

See also:

man mkraid

Making Filesystems

In this example, we are creating an ext2 filesystem. You could create any filesystem your system supports.

To check the device for bad blocks while creating the filesystem try:

mkfs -c /dev/md1

Mounting Filesystems

See also:

Setting Up RAID 1

WARNING: The instructions below invole creating partitions, and if done incorrectly may cause serious data loss or cause your system to become unusable.

These instructions are only for systems with kernel 2.2 or latter, as previous kernels used different tools and did not have the ability to autodetect raid partitions.

In the following example we will be using three disks, /dev/sdb, /dev/sdc and /dev/sdd as a spare disk. None of these disks contain any data we wish to preserve. For your system you will need to subtititue "/dev/sdb" , "/dev/sdc" and "/dev/sdd" with the appropriate device names for your system in the instructions below.

Login as root.

Create Partitions

Now we will create the partitions for our RAID stripe using fdisk.

fdisk /dev/sdb

At the prompt:

fdisk /dev/sdc

At the prompt:

fdisk /dev/sdd

At the prompt:

/etc/raidtab

In order to create a RAID device, we must first set up the raidtab file. On most systems this file does not already exist, so we will need to create it manually.

Open /etc/raidtab in a text editor.

Insert the following:

raiddev /dev/md1
raid-level 1
nr-raid-disks 2
nr-spare-disks 1
chunk-size 4
persistent-superblock 1
device /dev/sdb1
raid-disk 0
device /dev/sdc1
raid-disk 1
device /dev/sdd1
spare-disk 0

Create RAID device

Now that our RAID device(s) are defined, we will use the mkraid command to create them.

mkraid /dev/md1

Create filesystem

Create a filesystem on the RAID device.

mkfs /dev/md1

Mount filesystem

If you do not already have a mount point, create one now.

mkdir /mirror

and mount the filesystem.

mount /dev/md1 /mirror

See also:

man 5 raidtab

mkraid

Mkraid will complain if any of the partitions defined in your /etc/raidtab file already contain a filesystem. If this is the case, it will instruct you how to force it to create the RAID device.

See also:

man mkraid

Making Filesystems

In this example, we are creating an ext2 filesystem. You could create any filesystem your system supports.

To check the device for bad blocks while creating the filesystem try:

mkfs -c /dev/md1

Mounting Filesystems

See also:

Setting Up RAID 5

WARNING: The instructions below invole creating partitions, and if done incorrectly may cause serious data loss or cause your system to become unusable.

These instructions are only for systems with kernel 2.2 or latter, as previous kernels used different tools and did not have the ability to autodetect raid partitions.

In the following example we will be using three disks, /dev/sdb, /dev/sdc and /dev/sdd. None of these disks contain any data we wish to preserve. For your system you will need to subtititue "/dev/sdb" , "/dev/sdc" and "/dev/sdd" with the appropriate device names for your system in the instructions below.

Login as root.

Create Partitions

Now we will create the partitions for our RAID stripe using fdisk.

fdisk /dev/sdb

At the prompt:

fdisk /dev/sdc

At the prompt:

fdisk /dev/sdd

At the prompt:

/etc/raidtab

In order to create a RAID device, we must first set up the raidtab file. On most systems this file does not already exist, so we will need to create it manually.

Open /etc/raidtab in a text editor.

Insert the following:

raiddev /dev/md1
raid-level 5
nr-raid-disks 3
nr-spare-disks 0
chunk-size 4
persistent-superblock 1
parity-algorithm left-symmetric
device /dev/sdb1
raid-disk 0
device /dev/sdc1
raid-disk 1
device /dev/sdd1
raid-disk 2

Create RAID device

Now that our RAID device(s) are defined, we will use the mkraid command to create them.

mkraid /dev/md1

Create filesystem

Create a filesystem on the RAID device.

mkfs /dev/md1

Mount filesystem

If you do not already have a mount point, create one now.

mkdir /raid5

and mount the filesystem.

mount /dev/md1 /raid5

See also:

man 5 raidtab

mkraid

Mkraid will complain if any of the partitions defined in your /etc/raidtab file already contain a filesystem. If this is the case, it will instruct you how to force it to create the RAID device.

See also:

man mkraid

Making Filesystems

In this example, we are creating an ext2 filesystem. You could create any filesystem your system supports.

To check the device for bad blocks while creating the filesystem try:

mkfs -c /dev/md1

Mounting Filesystems

See also:

RAID Administration

Now that we have our RAID devices configured, we will look into the administration of the RAID system. The following is a guide to the RAID utilites which will allow you to safely operate and troubleshoot your RAID devices.

Status

The RAID subsystem maintains a record of the it's current state in a plain text file.

To check the status of all RAID devices:

cat /proc/mdstat

For a properly working RAID device, the output will look similar to:

Personalities : [raid1] [raid5]
read_ahead 1024 sectors
md1 : active raid1 sdc1[1] sdb1[0] 255424 blocks [2/2] [UU]
unused devices: <none>

The important things to notice in the output above are the phrases "active" and "[2/2]." This means that the RAID device is active and using both disks.

Should a disk become unusable, the output will look like:

Personalities : [raid1]
read_ahead 1024 sectors
md1 : active raid1 sdb1[0] 255424 blocks [2/1] [U_]
unused devices: <none>

You'll notice in this case that while the RAID system is still active, it is only using one disk (/dev/sdb1), noted by the fraction "[2/1]."

Replacing a Failed Disk

The process for using a replacement disk is not automatic. After the drive has been replaced you will need add it back into the existing RAID system.

Once the failed drive has been replaced:

Create the partition(s) on the new disk.

Add it to the RAID device with the raidhotadd command:

raidhotadd /dev/md1 /dev/sdc1

Checking the Filesystem

If you need to use fsck to check the filesystem on a RAID device, always use the RAID device name. Under normal circumstances, you should never fsck the individual partitions.

Unmount the filesystem you wish to check.

umount /dev/md1

fsck /dev/md1

Stopping and Starting RAID Devices

RAID devices are automaticly started when the system boots, and they are automaticly stopped when the system is shutdown.

Should you need to start a RAID device manually:

raidstart /dev/md1

To stop a RAID device:

Unmount the filesystem if nessesary.

umount /dev/md1

Then stop the device:

raidstop /dev/md1