Software RAID with mdadm

From MyWiki

---

Revision as of 14:26, 2 April 2010

I had a RedHat server that I managed to screw up, while testing different things, to the extent that an re-built was in order. The machine had no RAID controller and I was using software RAID, but couldn't find previous notes on how did I configure it, hence this page that I put together while rebuilding the system.

The server has 6 146G drives and runs RedHat EL 5.3. OS installed via kickstart and the first two drives used in RAID1. I had the rest 4 to play with. The server is going to be an NFS server with moderate load, so I decided to go with RAID5 in the end, to get more space. But initially I was thinking to build RAID10, so the notes have both set of commands.

First, I partitioned disk 3 for use with md.

fdisk /dev/sdc

Choose p to see if the disk has any partitions and delete it (hope you had backups of all the data). Mine had none, so I moved to creating new partition and changed the ID of the newly created partition to fd (Linux raid auto). Wrote changes and quit fdisk.

Now, I need to do the same on the rest of the drives. You can do it in one go, instead of getting into fdisk every time:

for d in d e f ; do sfdisk -d /dev/sdc | sfdisk /dev/sd${d} ; done

Check that we have it the way we wanted:

fdisk -ul /dev/sd{c,d,e,f}
Disk /dev/sdc: 146.8 GB, 146814976000 bytes
255 heads, 63 sectors/track, 17849 cylinders, total 286748000 sectors
Units = sectors of 1 * 512 = 512 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1              63   286744184   143372061   fd  Linux raid autodetect

Disk /dev/sdd: 146.8 GB, 146814976000 bytes
255 heads, 63 sectors/track, 17849 cylinders, total 286748000 sectors
Units = sectors of 1 * 512 = 512 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdd1              63   286744184   143372061   fd  Linux raid autodetect

Disk /dev/sde: 146.8 GB, 146814976000 bytes
255 heads, 63 sectors/track, 17849 cylinders, total 286748000 sectors
Units = sectors of 1 * 512 = 512 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sde1              63   286744184   143372061   fd  Linux raid autodetect

Disk /dev/sdf: 146.8 GB, 146814976000 bytes
255 heads, 63 sectors/track, 17849 cylinders, total 286748000 sectors
Units = sectors of 1 * 512 = 512 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdf1              63   286744184   143372061   fd  Linux raid autodetect

Time to create the RAID. For RAID10 you do this:

# mdadm --create /dev/md3 --level 10 --raid-devices=4 /dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1
mdadm: /dev/sdc1 appears to be part of a raid array:
    level=raid10 devices=4 ctime=Thu Dec  4 15:15:03 2008
mdadm: /dev/sdd1 appears to be part of a raid array:
    level=raid10 devices=4 ctime=Thu Dec  4 15:15:03 2008
mdadm: /dev/sde1 appears to be part of a raid array:
    level=raid10 devices=4 ctime=Thu Dec  4 15:15:03 2008
mdadm: /dev/sdf1 appears to be part of a raid array:
    level=raid10 devices=4 ctime=Thu Dec  4 15:15:03 2008
Continue creating array? y
mdadm: array /dev/md3 started.

Check that it's building:

# cat /proc/mdstat 
Personalities : [raid1] [raid10] 
md3 : active raid10 sdf1[3] sde1[2] sdd1[1] sdc1[0]
      286743936 blocks 64K chunks 2 near-copies [4/4] [UUUU]
      [>....................]  resync =  0.3% (1005312/286743936) finish=89.9min speed=52911K/sec
      
md0 : active raid1 sdb1[1] sda1[0]
      128384 blocks [2/2] [UU]
      
md1 : active raid1 sdb2[1] sda2[0]
      1052160 blocks [2/2] [UU]
      
md2 : active raid1 sdb3[1] sda3[0]
      142191232 blocks [2/2] [UU]
      
unused devices: <none>

Now, I created physical volume to use with LVM:

# pvcreate -ff /dev/md3
Really INITIALIZE physical volume "/dev/md3" of volume group "datavg" [y/n]? y
  WARNING: Forcing physical volume creation on /dev/md3 of volume group "datavg"
  Physical volume "/dev/md3" successfully created

And created volume group:

# vgcreate datavg /dev/md3
  Volume group "datavg" successfully created

To do RAID5, you would need this commands:

# mdadm --misc --stop /dev/md3
# mdadm --create /dev/md3 --level 5 --raid-devices=4 /dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1

I stopped the freshly built RAID10 and created RAID5 instead. If you're doing it with no existing RAID devices, you can skip stop command.