mdadm

Created Friday 24 May 2019

Create a configuration file

Do this to add appropriate details:

mdadm --detail --scan >> /etc/mdadm/mdadm.conf

This is necessary to make the array active at boot.

Upgrade individual disks

Initial state:

Server has three disks.

sda is the boot/root disk.
sdb and sdc are for RAID array.

Create the array

mdadm --create /dev/md0 --level=5 -n2 /dev/sd[bc] --force

Check the status

cat /proc/mdstat

Set up LVM

pvcreate /dev/md0
vgcreate raid /dev/md0
vgdisplay
lvcreate -L 4G raid -n raid1

Create a filesystem, mount it, check it.

mkfs /dev/raid/raid1 
mount /dev/raid/raid1 /srv
df -h

Add a disk

At this point, physically add a fourth disk to the machine. It manifests as sdd.

Add the disk to the array, make it active.

mdadm --add /dev/md0 /dev/sdd
mdadm --grow /dev/md0 --raid-devices=3

Monitor for the reshaping to complete.

cat /proc/mdstat

Adjust the pv to accept the new array size. Confirm it is reflected in the vg.

pvresize /dev/md0
vgdisplay

Add another, now larger, disk to the array, the first of three to upgrade.

At this point, physically add a larger disk to the machine. It manifests as sde.

Add the disk to the array, make it active by removing one of the existing ones.

mdadm --add /dev/md0 /dev/sde
mdadm /dev/md0 --fail /dev/sdb --remove /dev/sdb

Remove the old disk, replace it with another new one

Physically remove the disk that we just failed out from sdb, and replace it with a larger one. It manifests as sdb because sdb was freed up when we pulled out the old one.

Monitor for the reshaping to complete.

cat /proc/mdstat

Add the new disk to the array, make it active by removing one of the existing ones.


mdadm --add /dev/md0 /dev/sdb
mdadm /dev/md0 --fail /dev/sdc --remove /dev/sdc

Remove the old disk, replace it with another new one

Physically remove the disk that we just failed out from sdb, and replace it with a larger one. It manifests as sdc because sdc was freed up when we pulled out the old one.

Monitor for the reshaping to complete.

cat /proc/mdstat

Add the new disk to the array, make it active by removing the last of the existing ones.


mdadm --add /dev/md0 /dev/sdc
mdadm /dev/md0 --fail /dev/sdd --remove /dev/sdd

Remove the old disk.

Physically remove sdd.

Monitor for the reshaping to complete.

cat /proc/mdstat

Grow the physical volume.

pvresize /dev/md0

Confirm it.

vgdisplay

Shut down all the things

umount /srv/
lvremove /dev/raid/raid1 
vgremove raid
pvremove /dev/md0 
mdadm --stop /dev/md0 
mdadm --remove /dev/md0

Convert SLED to RAID

Initial state:

Server has two disks:

sda is boot/root disk
sdb is the data disk.

sdb is a pv

vg 'sled' has sdb in it.

lv 'data' is 7GB in 'sled' vg.

sled-data is formatted ext4 and has data
sled-data is mounted at /mnt/sled.

Add sdc

Physically add a disk to the server.

Create a RAID1 array with a missing disk.

mdadm --create /dev/md0 --level raid1 --raid-disks 2 missing /dev/sdc

Create the LVM objects and filesystem

pvcreate /dev/md0
vgcreate raid /dev/md0
lvcreate -L 7G -n data raid
mkfs /dev/raid/data
mount /dev/raid/data /mnt/raid

Copy the data from the SLED disk to the RAID disk.

cp -prv /mnt/sled/* /mnt/raid/

Unmount the SLED disk.

umount /mnt/sled

Destroy the SLED LVM objects.

lvremove /dev/sled/data
vgremove sled
pvremove /dev/sdb

Add that disk to the array. Monitor until complete.

mdadm /dev/md0 --add /dev/sdb
watch cat /proc/mdstat

End state:

Server has three disks:

sda is boot/root disk
sdb and sdc are the data disks.

These are a RAID1 array md0

md0 is a pv

vg 'raid' has md0 in it.

lv 'data' is 7GB in 'raid' vg.

raid-data is formatted ext4 and has data
raid-data is mounted at /mnt/raid.

Convert RAID1 to RAID5

Initial state:

Server has three disks:

sda is boot/root disk
sdb and sdc are the data disks.

These are a RAID1 array md0

md0 is a pv

vg 'raid' has md0 in it.

lv 'data' is 7GB in 'raid' vg.

raid-data is formatted ext4 and has data
raid-data is mounted at /mnt/raid.

Add sdd

Physically connect a disk to the server.

Add the disk to the array

mdadm /dev/md0 --add /dev/sdd

Change the RAID level to 5

mdadm --grow /dev/md0 -l5

Change the number of active disks in the array

mdadm --grow /dev/md0 -n3

Monitor until complete

watch cat /proc/mdstat

End state:

Server has four disks:

sda is boot/root disk
sdb, sdc and sdd are the data disks.

These are a RAID5 array md0

md0 is a pv

vg 'raid' has md0 in it.

lv 'data' is 7GB in 'raid' vg.

raid-data is formatted ext4 and has data
raid-data is mounted at /mnt/raid.