UCSD T2 Linux Software RAID FAQ

Contents

Introduction

This FAQ covers answers some UCSD T2 specific topics related to the use of Linux software RAID

What is a quick command for creating a raid level 5 array from 4 disk partitions?

 mdadm --create /dev/md0 --level=5 --raid-devices=4 /dev/sda4 /dev/sdb1 /dev/sdc1 /dev/sdd1

How do you increase the speed of the RAID rebuild?

With todays very large disks the default raid rebuild speed in linux is excruciatingly slow taken sometimes several days to complete. The max rebuild speed can be controlled however.

Raid rebuild speed measured in KB/s is controlled by the /sys device file

/proc/sys/dev/raid/speed_limit_min

This number in this file is expressed in KB/s

To increase the speed from the default 1000KB/s (1MB/s) to 50,000KB/s (50MB/s) execute the following command

echo 50000 >/proc/sys/dev/raid/speed_limit_min 

With new drives and controllers it may be possible to increase this limit even further to speed up recovery of very large arrays.

How do you increase the speed of the RAID rebuild?

cat /proc/mdstat

to see this file in a way that constantly updates run

watch cat /proc/mdstat

How do you build an array from disk images?

The first thing needed is a set of disk images (assuming that they are images of some formatted filesystem), for this example let's use:

# ls -alh
total 392M
drwxr-xr-x  2 root root 4.0K Jan 28 15:04 .
drwxr-xr-x 26 root root 4.0K Jan 27 13:23 ..
-rw-r--r--  1 root root  98M Jan 28 14:55 sda1.img
-rw-r--r--  1 root root  98M Jan 28 14:53 sdb1.img
-rw-r--r--  1 root root  98M Jan 28 14:53 sdc1.img
-rw-r--r--  1 root root  98M Jan 28 14:53 sdd1.img

So with images in hand, we must now mount these to /dev somehow to build an array. If the /dev/loop* devices are already available the following step may be skipped, otherwise they need to be created using the MAKEDEV package:

# /sbin/MAKEDEV loop0
# /sbin/MAKEDEV loop1
# /sbin/MAKEDEV loop2
# /sbin/MAKEDEV loop3

The loop devices are now available and we then have to associate them with the disk images so that mdadm will recognize them as normal block devices. For this purpose we use losetup. Depending on the version the syntax may be slightly different, to see available options look at the man for losetup and see what available options there are.

If losetup -f is available (it will automatically search out open loop devices), then simply run the commands:

# losetup -f sda1.img 
# losetup -f sdb1.img 
# losetup -f sdc1.img 
# losetup -f sdd1.img

To verify that the image is now attached run:

# losetup /dev/loop0
/dev/loop0: [fd00]:3921036 (/temp/sda.img)

If losetup -f is not available, then the syntax should be:

# losetup /dev/loop0  sda1.img
# losetup /dev/loop1  sdb1.img
# losetup /dev/loop2  sdc1.img
# losetup /dev/loop3  sdd1.img

# losetup /dev/loop0
/dev/loop0: [0900]:13 (sda1.img)

At this point we now have block devices that mdadm can be friendly with, so we can then create an array:

# mdadm --create /dev/md0 --level=0 --raid-devices=4 /dev/loop[0-3]
mdadm: array /dev/md0 started.

mdadm --assemble --force /dev/md1 /dev/loop[0-3] 

Access keys: S = Save, Q = Quiet save, K = Save and Continue, P = Preview, C = Cancel

What other information is there about Linux software RAID?

• http://www.sanitarium.net/golug/Linux_Software_RAID.html

-- TerrenceMartin - 08 Nov 2006