An mdadm software RAID array on Rocky Linux survives a dead disk only if three files around it are right as well. You need /etc/mdadm.conf, a safe mount entry and a monitor that tells you a disk died. Skip mdadm.conf and the array can come back as /dev/md127 in read-only mode, while mdmonitor refuses to start, so no failure email ever leaves the box. We hit both on a test server.
You’ll build software RAID 1, 5 or 10, mount it with XFS, pin its name, wire up email alerts, add a hot spare and then fail a disk on purpose to time the rebuild. Every command was run on Rocky Linux 10.2 (kernel 6.12, mdadm 4.4) with SELinux enforcing in October 2026. AlmaLinux 10 ships the identical mdadm-4.4-3.el10_2.2 build, so the steps carry over to AlmaLinux and RHEL 10.
Prerequisites
You need a Rocky Linux 10, AlmaLinux 10 or RHEL 10 server with sudo access and a set of empty disks that the operating system does not live on. Everything on those disks is destroyed in the partition step, so check twice which ones they are:
- 2 disks for RAID 1, 3 for RAID 5, 4 for RAID 10
- one extra disk if you want a hot spare
- disks of the same size, ideally the same model
- the
mdadm,parted,postfixands-nailpackages from BaseOS and AppStream
Before you buy anything, work out two numbers from the disk size. Those are usable capacity and rebuild time. mdadm uses the smallest member’s size for every member, so a 4 TB disk mixed with 8 TB disks wastes the difference. Rebuild time grows with capacity, because md copies the whole member, not just your data. If you are still choosing drives, our NAS hard drive comparison covers CMR versus SMR and price per TB, and SMR drives are the ones to avoid under RAID because their rewrite speed collapses during a rebuild. A server that has run out of bays can take more disks through a JBOD enclosure, as long as it presents each disk individually rather than doing its own RAID.
Our lab VM had 2 vCPU, 2 GB RAM and four 10 GB virtual disks. That is a floor for following along, not a production recommendation: md itself needs little CPU or RAM, but the disks and the controller set the rebuild speed, so size by disk count and capacity instead.
Pick the RAID level
RAID 1 is the right default for two disks, RAID 10 for four or more when you care about write speed, and RAID 5 when capacity matters more than rebuild risk. The table shows what each level gave us on 10 GB members, including how long the initial sync took with md’s default speed cap:
| Level | Disks | Usable space (10 GB members) | Survives | Initial sync in our lab |
|---|---|---|---|---|
| RAID 1 (mirror) | 2 | 9.99 GiB | 1 disk | 58 s |
| RAID 5 (striping with parity) | 3 | 19.98 GiB | 1 disk | 84 s |
| RAID 10 (striped mirrors, near=2) | 4 | 19.98 GiB | 1 disk per mirror pair | 109 s |
RAID 10 took longest because md resyncs the full 20 GiB array, while RAID 1 only mirrors one 10 GiB member. RAID 1 and RAID 10 averaged about 185 and 197 MB/s, close to the kernel’s default ceiling of 200000 KB/s that we read from /proc/sys/dev/raid/speed_limit_max. RAID 5 averaged only about 128 MB/s, even though its first progress line showed 300 MB/s. Treat that ceiling as the best case you plan around: even at a full 200 MB/s, a single 10 TB member needs about 14 hours, and RAID 5 runs that long with no redundancy left. That is why we’d pick RAID 10 or RAID 6 over RAID 5 once disks pass a few TB. If you’re still weighing md against LVM mirroring or ZFS, the RAID vs LVM vs ZFS comparison covers the trade-offs.
Install mdadm and identify the disks
Our Rocky Linux image already had mdadm installed, but installing it explicitly costs nothing and makes sure parted is there for the next step.
sudo dnf install -y mdadm partedConfirm which version you got. The output is a single line.
mdadm --versionThe BaseOS build reports itself like this.
mdadm - v4.4 - 2024-11-07 - 3Now find the disks. Do not trust the sdX letters alone, because the kernel hands them out in probe order and that order changes between boots. List each disk with its model and serial so you can tie a name to a physical drive:
lsblk -d -o NAME,SIZE,TYPE,MODEL,SERIAL,ROTAOur four data disks are the 10G entries with no partitions, and sda holds the operating system:
NAME SIZE TYPE MODEL SERIAL ROTA
sda 10G disk QEMU HARDDISK drive-scsi0 1
sdb 10G disk QEMU HARDDISK drive-scsi3 1
sdc 10G disk QEMU HARDDISK drive-scsi4 1
sdd 10G disk QEMU HARDDISK drive-scsi2 1
sde 10G disk QEMU HARDDISK drive-scsi1 1
sr0 4M rom QEMU DVD-ROM QM00003 1Notice that sdb is the disk in slot 3 and sde is in slot 1, so the letters already do not follow the hardware. After later reboots on the same VM, the operating system disk came up as sdb and one data disk as sda. md itself does not care, because it finds members by the UUID in their superblock. You should care whenever you type a device name into a destructive command, so run lsblk again right before each one. The stable names live in /dev/disk/by-id/, and on real hardware they carry the drive model and serial number printed on its label.
Partition the disks for RAID
mdadm can use whole disks, but a single partition per disk is safer. Other tools then see a partition table instead of a disk that looks empty, and the raid flag tells anyone reading it what the partition is for. Starting at 1 MiB keeps the partition aligned to the physical sectors of modern disks. This loop writes a GPT label, one full-size partition and the raid flag on each data disk. Edit the list to match your own lsblk output before you run it:
for disk in /dev/sdb /dev/sdc /dev/sdd /dev/sde; do
sudo parted --script "$disk" mklabel gpt mkpart primary 1MiB 100% set 1 raid on
doneCheck one of them before you go further, because a typo in the loop shows up here.
sudo parted --script /dev/sdb printPartition 1 should span the disk and carry the raid flag:
Model: QEMU QEMU HARDDISK (scsi)
Disk /dev/sdb: 10.7GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
Number Start End Size File system Name Flags
1 1049kB 10.7GB 10.7GB primary raidIf the flag column is empty or the size is wrong, rerun the loop for that disk before you create the array, because mdadm takes whatever partition you hand it.
Create the mdadm RAID array
Pick one of the three commands below. Each one creates /dev/md0 with version 1.2 metadata and an internal write-intent bitmap. Pass --bitmap=internal yourself, because current mdadm otherwise stops to ask about it first, and the bitmap is what makes a briefly disconnected disk resync in seconds instead of minutes (measured further down). The array is usable as soon as the command returns, while the initial sync runs in the background.
Create RAID 1 with two disks
A mirror keeps a full copy on each disk, so you get the capacity of one disk and can lose either.
sudo mdadm --create /dev/md0 --level=1 --raid-devices=2 --bitmap=internal /dev/sdb1 /dev/sdc1RAID 1 is the only level that asks a question here. The warning is about booting from the array, which this data array does not need, so answer y. We piped the answer in for this capture, so the y doesn’t appear after the prompt:
mdadm: Note: this array has metadata at the start and
may not be suitable as a boot device. If you plan to
store '/boot' on this device please ensure that
your boot-loader understands md/v1.x metadata, or use
--metadata=0.90
Continue creating array [y/N]? mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md0 started.If you plan to boot from a mirror later, that’s a separate design with its own bootloader steps, so keep this array for data.
Create RAID 5 with three disks
RAID 5 stripes data with rotating parity, so three disks give you two disks of space and survive one failure. This is the array the rest of this guide continues with, keeping sde1 free for the hot spare:
sudo mdadm --create /dev/md0 --level=5 --raid-devices=3 --bitmap=internal /dev/sdb1 /dev/sdc1 /dev/sdd1RAID 5 creates without a prompt and returns the same two lines about version 1.2 metadata and the array starting.
Create RAID 10 with four disks
RAID 10 stripes across mirrored pairs. You get half the raw space, it writes no parity, so random writes don’t pay the RAID 5 penalty, and it survives one failure in each pair:
sudo mdadm --create /dev/md0 --level=10 --raid-devices=4 --bitmap=internal /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1For RAID 10, the member list in mdadm --detail shows how the default near=2 layout paired the disks. The mirror pairs are disks 0 and 1, and disks 2 and 3. set-A (disks 0 and 2) and set-B (disks 1 and 3) each hold one complete copy of the data. The array survives losing every disk in one set, because that is one disk per mirror pair. Losing one disk from each set can take out both halves of a pair, and then the array is gone:
Layout : near=2
Chunk Size : 512K
Consistency Policy : bitmap
Name : raid01:0 (local to host raid01)
UUID : 4143c99c:1922afb3:f990c271:72deb20f
Events : 21
Number Major Minor RaidDevice State
0 8 17 0 active sync set-A /dev/sdb1
1 8 33 1 active sync set-B /dev/sdc1
2 8 49 2 active sync set-A /dev/sdd1
3 8 65 3 active sync set-B /dev/sde1The Name : raid01:0 (local to host raid01) line matters later. mdadm stamped the server’s hostname into the superblock, and that stamp decides what the array is called at the next boot.
Watch the initial sync
The kernel’s view of every md array lives in /proc/mdstat, and it is the first file to read whenever you’re unsure what an array is doing:
cat /proc/mdstatRight after we created the RAID 5 array, it looked like this. You’ll see a progress bar and an estimated finish time.
Personalities : [raid1] [raid4] [raid5] [raid6]
md0 : active raid5 sdd1[3] sdc1[1] sdb1[0]
20948992 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/2] [UU_]
[=>...................] recovery = 5.7% (600960/10474496) finish=0.5min speed=300480K/sec
bitmap: 1/1 pages [4KB], 65536KB chunk
unused devices: <none>That [3/2] [UU_] and the word recovery look alarming on a brand new array, but they are normal for RAID 5. md builds it degraded and computes the third member from the other two. That is faster than a full resync. RAID 1 and RAID 10 show resync with [UU] or [UUUU] instead. When the progress line disappears and every slot reads U, the array has full redundancy. You can format and use it during the sync, but a disk that fails before it finishes takes data with it, so don’t load anything you can’t replace until then.
Create the XFS filesystem and mount it
XFS is the Rocky Linux default. On an md device, mkfs.xfs reads the chunk size and data disk count and aligns itself to them. ext4 works too if you prefer it, and our ext4 vs XFS comparison covers where each one wins. Format the array:
sudo mkfs.xfs /dev/md0Look for the sunit and swidth line. With a 512K chunk, sunit=128 blocks of 4 KiB is one chunk, and swidth=256 is two chunks, the two data disks of a three-disk RAID 5:
meta-data=/dev/md0 isize=512 agcount=16, agsize=327296 blks
= sectsz=512 attr=2, projid32bit=1
= crc=1 finobt=1, sparse=1, rmapbt=1
= reflink=1 bigtime=1 inobtcount=1 nrext64=1
= exchange=0 metadir=0
data = bsize=4096 blocks=5236736, imaxpct=25
= sunit=128 swidth=256 blks
naming =version 2 bsize=4096 ascii-ci=0, ftype=1, parent=0
log =internal log bsize=4096 blocks=16384, version=2
= sectsz=512 sunit=8 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
= rgcount=0 rgsize=0 extents
= zoned=0 start=0 reserved=0Create the mount point and mount it by hand once. You’ll add the permanent fstab entry after mdadm.conf is in place.
sudo mkdir -p /srv/raid
sudo mount /dev/md0 /srv/raidWith SELinux enforcing, check the label on the new mount point before any service writes to it. A fresh XFS root directory has no label at all:
ls -ldZ /srv/raidOurs came back unlabeled_t, which confined services such as Samba, nginx or a database are denied access to:
drwxr-xr-x. 2 root root system_u:object_r:unlabeled_t:s0 6 Oct 16 13:23 /srv/raidApply the policy’s default label for that path. You don’t need a custom rule for /srv, because the targeted policy already defines one.
sudo restorecon -Rv /srv/raidrestorecon reports each change it makes. No output would mean the label was already correct.
Relabeled /srv/raid from system_u:object_r:unlabeled_t:s0 to system_u:object_r:var_t:s0var_t is the generic label for /srv. When a specific service will own the data, give the directory that service’s type with semanage fcontext and run restorecon again rather than turning SELinux off. If you want to carve the array into several resizable volumes instead of one filesystem, put LVM on top of the RAID device before formatting.
Save mdadm.conf so the array keeps its name
Without /etc/mdadm.conf, udev still assembles the array at boot, but the name it gets depends on the hostname stamped in the superblock. On the same host, the array came back as md0. To see what happens when the disks move to another server, or when you rename the host after building the array, we changed the hostname to nas02 and rebooted:
Personalities : [raid4] [raid5] [raid6]
md127 : active (auto-read-only) raid5 sdb1[0] sdc1[3] sde1[1]
20948992 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU]
bitmap: 0/1 pages [0KB], 65536KB chunk
unused devices: <none>The array became md127, the device link turned into /dev/md/raid01:0, and it sat in auto-read-only until the first write. Anything that referenced /dev/md0 broke. On that same boot the members also swapped letters (they were sdb1, sdc1 and sdd1 when we built it), which proves again why names in commands need checking. The fix is an ARRAY line in mdadm.conf that names the device and identifies the array by UUID. On the host that built the array, generate it from the running array:
sudo mdadm --detail --scan | sudo tee /etc/mdadm.confThe file now holds one ARRAY line. Your UUID will differ, and the scan may print /dev/md/0 instead of /dev/md0, which works the same way. If the array is already showing up as md127, the scan prints ARRAY /dev/md/raid01:0 instead, and you must edit that name to /dev/md0 before saving, which is what we did on nas02.
ARRAY /dev/md0 metadata=1.2 UUID=887e0290:5db58f14:ff7e0c94:7875f51dWith that line in place, we rebooted again while the host was still called nas02, and the array assembled as md0. We also ran sudo dracut -f, and sudo lsinitrd | grep mdadm.conf came back empty: our initramfs carries the mdraid module but not mdadm.conf. A data array is assembled after the real root is mounted, from the mdadm.conf on disk. Run the same grep on your server, though, because dracut can copy the file in. If it shows up there, run sudo dracut -f after every mdadm.conf change so the two copies don’t disagree at boot. It matters most when / or /boot itself sits on md. The file has a second job, too. mdmonitor does not start without it, which the alerting section below runs into.
Mount the array at boot with fstab
Mount by filesystem UUID rather than /dev/md0, so a renamed array still mounts. Add nofail as well, because without it an array that fails to assemble drops the whole server into emergency mode, and you cannot reach it over SSH to fix it. This one line reads the UUID and appends the entry:
echo "UUID=$(sudo blkid -s UUID -o value /dev/md0) /srv/raid xfs defaults,nofail 0 0" | sudo tee -a /etc/fstabThe appended line looks like this, with your own UUID.
UUID=50fb9497-3220-4ecf-91a3-b6bc1fd399c9 /srv/raid xfs defaults,nofail 0 0Reload systemd so it picks up the new mount unit, then test the entry without a reboot. Unmount first, because mount -a skips a filesystem that is already mounted, and with nofail a wrong UUID would otherwise pass silently:
sudo systemctl daemon-reload
sudo umount /srv/raid
sudo mount -a
findmnt /srv/raidA working entry shows /dev/md0 as the source. The mount options also show XFS picked up the stripe geometry (sunit and swidth are in 512-byte sectors here):
TARGET SOURCE FSTYPE OPTIONS
/srv/raid /dev/md0 xfs rw,relatime,seclabel,attr2,inode64,logbufs=8,logbsize=32k,sunit=1024,swidth=2048,noquotaIf mount -a prints an error or findmnt shows nothing, fix the line now. A typo in fstab is far easier to correct from a working shell than from the emergency console. findmnt can also lint the whole file:
sudo findmnt --verifyA clean fstab prints Success, no errors or warnings detected. One caveat comes with nofail: if the array doesn’t assemble, the server boots with /srv/raid as an empty directory on the root filesystem, and a service that writes there fills your root disk instead. Give any service that uses the array a RequiresMountsFor=/srv/raid drop-in. For an NFS export of the array, open a drop-in for the NFS server:
sudo systemctl edit nfs-server.serviceAdd these two lines in the editor that opens, then save and quit:
[Unit]
RequiresMountsFor=/srv/raidsystemd turns that into Requires= and After= on srv-raid.mount. When the array fails to assemble, the mount unit fails and the service stays down, instead of writing into the root filesystem.
Enable mdmonitor email alerts
A degraded RAID array keeps running without a sound, which is the whole problem: the first failure costs nothing, the second costs everything, and you only hear about the second one. Rocky Linux ships mdmonitor.service enabled, but that doesn’t help on its own. To show why, we moved mdadm.conf aside, rebooted and read the unit’s journal:
sudo journalctl -b -u mdmonitorsystemd never started the monitor, and systemctl is-active mdmonitor printed inactive:
Oct 16 13:55:21 raid01 systemd[1]: mdmonitor.service - Software RAID monitoring and management was skipped because of an unmet condition check (ConditionPathExists=/etc/mdadm.conf).The unit has ConditionPathExists=/etc/mdadm.conf, so with the file from the previous section in place it starts, but it still needs somewhere to send mail. The simplest setup that always works is local delivery to root through Postfix, which listens only on localhost by default and needs no relay. Install Postfix and the mail client:
sudo dnf install -y postfix s-nail
sudo systemctl enable --now postfixTell mdadm where alerts go by adding a MAILADDR line to mdadm.conf.
echo "MAILADDR root" | sudo tee -a /etc/mdadm.confRestart mdmonitor, don’t just enable it. Ours was already running from the reboot, and a running monitor keeps the config it read at start, so enable --now would have left it with no address:
sudo systemctl enable mdmonitor
sudo systemctl restart mdmonitorSend a test alert for every array in mdadm.conf. You should never trust an alert path you haven’t seen work.
sudo mdadm --monitor --scan --test --oneshotmdadm confirms it raised the event. That line alone doesn’t prove the mail left, so check the mailbox next.
mdadm: TestMessage event detected on md device /dev/md0Read root’s mailbox to confirm you got the message delivered, not just generated.
sudo mail -H -f /var/spool/mail/rootOne message from “mdadm monitoring” means the whole chain works. You can open it with sudo mail -f /var/spool/mail/root, then type p and press Enter to print it.
▸N 1 mdadm monitoring 2026-10-16 13:30 23/880 "TestMessage event on "The body includes a copy of /proc/mdstat at the moment of the event, which helps when you read it on a phone. Local mail only works if somebody reads it. On a real server, forward root to an address you watch with an alias in /etc/aliases, or point Postfix at a relay such as the one in our Rocky Linux mail server guide. Two details from the man page shape what you’ll receive:
- mail goes out only for Fail, FailSpare, DegradedArray, SparesMissing and TestMessage
- every other event (SpareActive, RebuildStarted, RebuildFinished) reaches the journal only, because the unit runs with
--syslog - the polling delay defaults to 60 seconds, tunable with
MONITORDELAYin mdadm.conf
The man page says the delay applies when no event happened, and mdadm also waits on /proc/mdstat for changes, so treat our timing as one observation, not a rule. In our failure test below, run right after a monitor restart, the kernel logged the failed disk at 13:30:54 and the Fail email went out at 13:31:52, 58 seconds later.
Add a hot spare
A hot spare is an idle member that md pulls in the moment another member fails, so the rebuild starts before you’ve even read the alert. On a box nobody visits every day, that’s the best use of one extra disk. Add it by its /dev/disk/by-id path rather than its sdX letter, because by this point in our lab the reboots had already shuffled the names and the free disk was sdc. List the partition links and pick the one that points at a partition not listed in /proc/mdstat:
ls -l /dev/disk/by-id/ | grep part1On our VM, sdb1 was the 2 MB boot partition on the operating system disk and sda1, sdd1 and sde1 were array members, which left drive-scsi1 as the free disk. On real hardware these names carry the model and serial number:
lrwxrwxrwx. 1 root root 10 Oct 16 13:27 scsi-0QEMU_QEMU_HARDDISK_drive-scsi0-part1 -> ../../sdb1
lrwxrwxrwx. 1 root root 10 Oct 16 13:27 scsi-0QEMU_QEMU_HARDDISK_drive-scsi1-part1 -> ../../sdc1
lrwxrwxrwx. 1 root root 10 Oct 16 13:27 scsi-0QEMU_QEMU_HARDDISK_drive-scsi2-part1 -> ../../sdd1
lrwxrwxrwx. 1 root root 10 Oct 16 13:27 scsi-0QEMU_QEMU_HARDDISK_drive-scsi3-part1 -> ../../sde1
lrwxrwxrwx. 1 root root 10 Oct 16 13:27 scsi-0QEMU_QEMU_HARDDISK_drive-scsi4-part1 -> ../../sda1Confirm where the chosen link points before you use it:
readlink -f /dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi1-part1It resolved to the unused partition, /dev/sdc1, so add it:
sudo mdadm --manage /dev/md0 --add /dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi1-part1On a healthy, complete array an added disk becomes a spare, marked (S) in mdstat and spare in the detail output:
Number Major Minor RaidDevice State
0 8 65 0 active sync /dev/sde1
1 8 1 1 active sync /dev/sda1
3 8 49 2 active sync /dev/sdd1
4 8 33 - spare /dev/sdc1Regenerate mdadm.conf now, because the scan output gains spares=1. With that recorded, mdmonitor knows the array is meant to have a spare and mails a SparesMissing alert when it doesn’t, which is how you find out the spare was used up. Rewrite the file with both lines and restart the monitor:
{ sudo mdadm --detail --scan; echo "MAILADDR root"; } | sudo tee /etc/mdadm.conf
sudo systemctl restart mdmonitorThe file should now read as follows, and you can diff it against yours.
ARRAY /dev/md0 metadata=1.2 spares=1 UUID=887e0290:5db58f14:ff7e0c94:7875f51d
MAILADDR rootRepeat this regeneration whenever you add or remove a spare, so the count mdmonitor checks matches what you actually run.
Replace a failed disk in mdadm
Practise this on a test array before you need it for real, because a disk swap under pressure is where people remove the wrong disk. Before failing anything, we wrote a 2 GiB file of random data to the array and saved its checksum, so we could prove afterwards that nothing was lost. Then we marked one active member as failed, the same state md enters on its own when a disk throws I/O errors:
sudo mdadm --manage /dev/md0 --fail /dev/sda1Within five seconds the spare had taken the failed slot and was rebuilding. In mdstat, the failed member carries (F), the array shows [3/2] [U_U], and the recovery line gives an estimate:
The 10 GiB rebuild ran from 13:30:54 to 13:31:48, so 54 seconds. The speed fell from 250 MB/s to about 130 MB/s partway through. mdmonitor then logged the Fail and SpareActive events and mailed the Fail alert. The checksum still matched once the rebuild finished, which you can see in the last line.
The failed disk stays listed as faulty until you remove it, and that’s the first step of the actual replacement. Before pulling hardware, match the faulty name to a serial with ls -l /dev/disk/by-id/, so you pull the right drive. Remove the failed member from the array:
sudo mdadm --manage /dev/md0 --remove /dev/sda1Swap the disk, then give the new one the same partition layout as the others. Check its name with lsblk first, since a hot-plugged disk can take any free letter:
sudo parted --script /dev/sda mklabel gpt mkpart primary 1MiB 100% set 1 raid onIf you’re reusing a disk that was once in an array, including the one you just removed, wipe its old md superblock before adding it. With the old superblock still there, mdadm may treat the disk as a returning member and re-add it from the bitmap, or refuse it and suggest --zero-superblock. Neither is what you want for a replacement, so start it clean:
sudo mdadm --zero-superblock /dev/sda1Add the new partition to the array. You can watch the result in /proc/mdstat straight away.
sudo mdadm --manage /dev/md0 --add /dev/sda1Because the spare already rebuilt the array, the new disk becomes the next spare, and the array is back to the layout mdadm.conf describes. Without a spare, the same --add starts the rebuild straight onto the new disk. Our RAID 5 lab stopped at the fail step, so the failed member stayed attached. We timed the remove, zero and add sequence on the RAID 10 test array instead: a full rebuild of one 10 GiB member took 71 seconds by the kernel log.
The write-intent bitmap changes that number a lot when a disk comes back rather than being replaced, say after a loose cable or a controller reset. We failed and removed a RAID 10 member, wrote 512 MiB to the array while it was out, then returned it with --re-add instead of --add:
sudo mdadm --manage /dev/md0 --re-add /dev/sdd1md copied only the regions the bitmap had marked dirty. The kernel log shows the recovery finishing six seconds after it started, against 71 seconds for the full rebuild of the same member:
Oct 16 13:22:25 raid01 kernel: md/raid10:md0: Disk failure on sdd1, disabling device.
md/raid10:md0: Operation continuing on 3 devices.
Oct 16 13:22:35 raid01 kernel: md: recover of RAID array md0
Oct 16 13:22:41 raid01 kernel: md: md0: recover done.Use --re-add only for the same disk coming back with its superblock intact. A disk that actually failed gets replaced, partitioned and added with --add, never trusted again.
Troubleshoot mdadm assembly and alert errors
We hit each of these on the lab server while writing this guide, so the messages below are verbatim.
md: kicking non-fresh sdb1 from array!
We saw this at the first boot after the failure test. The failed member was still physically attached with its old superblock, its event counter read 26 while the array was at 54, so md left it out and assembled the array from the three current members. That’s md protecting you from stale data, not a new failure. Compare the counters with sudo mdadm --examine /dev/sdb1 | grep Events and sudo mdadm --detail /dev/md0 | grep Events, then either remove the disk or wipe it with --zero-superblock and add it back as a fresh member. The same boot also triggered the SparesMissing email, because mdadm.conf said spares=1 and the spare was now an active member.
md127 : active (auto-read-only)
The array was assembled without an ARRAY line in mdadm.conf, and the hostname in its superblock doesn’t match the current host, so md treated it as foreign. It happens after moving disks to a new server, renaming the host, or booting a rescue system. Generate the ARRAY line with mdadm --detail --scan as shown earlier, change the device to /dev/md0 if the scan printed /dev/md/raid01:0, and reboot. The read-only flag clears on the first write to the array.
mdmonitor.service was skipped because of an unmet condition check
There is no /etc/mdadm.conf, so systemd skips the monitor and you get no alerts. Create the file, add MAILADDR and restart the service. The kernel also printed md: async del_gendisk mode will be removed in future, please upgrade to mdadm-4.5+ at every boot. That’s a deprecation notice about mdadm’s userspace version, and our arrays assembled normally through six reboots with it showing.
Schedule RAID scrubs and keep the array healthy
A RAID 5 rebuild reads every sector of every surviving disk, and a RAID 1 or RAID 10 rebuild reads every sector of the failed disk’s mirror partner. Either way, that’s the worst moment to find a bad sector nobody has read in a year. A scrub reads the whole array on a schedule and catches those sectors while redundancy still exists. Rocky Linux already ships raid-check.timer enabled, set to run every Sunday at 01:00, with its settings in /etc/sysconfig/raid-check (CHECK=check, low priority). Confirm it’s scheduled:
systemctl list-timers raid-check.timerThe next run shows in the first column. On our fresh install, LAST showed the first boot time, not a completed scrub.
NEXT LEFT LAST PASSED UNIT ACTIVATES
Sun 2026-10-18 01:00:00 UTC 1 day 11h Fri 2026-10-16 13:13:17 UTC - raid-check.timer raid-check.serviceRun one by hand after building the array, so you know how long it takes on your disks before the weekly run starts competing with real traffic:
echo check | sudo tee /sys/block/md0/md/sync_actionProgress shows in /proc/mdstat as check. Ours finished in 53 seconds for 10 GiB members. When it’s done, read the mismatch counter:
cat /sys/block/md0/md/mismatch_cntZero is what you want. A non-zero count on RAID 5 means parity disagrees with data somewhere, so check the disks with smartctl before you run a repair. On RAID 1 and RAID 10, small non-zero counts can come from swap or other writes that were in flight, and they aren’t automatically a sign of a failing disk.
The rest of the routine is short, and you can do all of it from the commands already in this guide:
- a spare in the array, or at least one on the shelf
- a higher
speed_limit_minduring a rebuild when finishing matters more than latency - a test alert every few months with
mdadm --monitor --scan --test --oneshot
The spare matters most, because the 14 hours a large disk needs to rebuild is a bad time to wait for a delivery. The test alert catches the quieter failure: a relay password that changed last spring is how a degraded array goes unnoticed for months. For anything the md layer can’t fix, the kernel md documentation and Red Hat’s RHEL 10 RAID chapter are the references we checked these steps against. Done this way, a dead disk means an email, a swap and a minute of rebuild per 10 GB, not a restore.
