Three unRAID Disks Failed at Once. It Was a SATA Power Plug.

In one night, three of the four drives in my unRAID array threw errors. One got disabled, one dropped out in the middle of an rsync, and one faulted when the array started. Every one of them passed SMART, and the fault turned out to be a single SATA power plug.

TL;DR: Device Fault and ABRT errors plus SATA link resets on several drives, with no UNC errors, point at power, not media. Read smartctl -l error on each drive and look for the pattern. Then change one thing, the power path, and prove it with a load that touches every drive at once. After that, rebuild the disabled disk onto the same drive.

The setup that matters

The cache SSD takes the NUC's onboard SATA port, so the array hangs off an ASM1166 SATA controller on an NVMe-to-SATA M.2 adapter. The drives sit in a 4-bay enclosure. A Gigabyte GP-P450B ATX PSU, jumpered on, powers the enclosure. One SATA power strand with four plugs feeds the cache SSD and two plugs on the enclosure backplane.

All four are 4 TB WD Purples.

What it looked like

On 2026-08-26 the failures arrived one at a time, each on a different drive:

• disk1 got disabled at 00:38.
• disk2 dropped while an rsync was running.
• disk3 faulted when the array auto-started.

Three drives in one night looks like a bad batch or a dying controller. The logs said something else. These were the kernel errors:

What was missing matters as much: no UNC. A disk with bad media reports uncorrectable read errors. These drives were not complaining about sectors. They were faulting and disappearing off the bus.

The signature

This is the rule I now keep in my notes:

DF/ABRT + link down across multiple drives with no UNC = power, not media.

The reasoning is simple once you see it. Media failures are per disk and random. Three disks don't grow bad sectors on the same night. What several drives share is the controller, the data cables, and the power. A drive that browns out mid-command does exactly this: it trips its fault bit, aborts what it was doing, and the PHY drops. When power comes back, the link retrains. That is the PHYRdyChg.

SMART agreed. Every drive was clean. The drive-side error log is where the pattern shows, and it is per drive, so read each one:

for d in /dev/sdc /dev/sdd /dev/sde /dev/sdf; do
  echo "== $d"; smartctl -l error "$d"
done

The kernel side shows the same thing from the host's view. My array ports are ata6 to ata9, so:

dmesg | grep -E "ata[6-9]" | grep -E "DF ERR|ABRT|PHYRdyChg|link down"

If that grep lights up on more than one port and SMART stays clean, stop looking at the disks.

Ruling out ASPM

The other usual suspect on these cards is PCIe power management. ASPM on an ASM1166 can drop links, so it had to be ruled out. pcie_aspm=off was already on the syslinux append line in /boot/syslinux/syslinux.cfg. The trap: this BIOS turns ASPM back on anyway, so the kernel flag alone does not mean it is off. The per-boot fix clears the ASPM bits in the Link Control register of the controller and its upstream port:

for dev in 6d:00.0 00:1d.0; do
  v=$(setpci -s $dev CAP_EXP+10.w)
  setpci -s $dev CAP_EXP+10.w=$(printf '%04x' $((0x$v & ~3)))
done

ASPM wasn't the cause. I keep it off as a precaution, but it didn't change the picture.

The plug

That left power. The failures clustered around load: an rsync, an array start with every drive spinning up. That fits a supply path that holds at idle and sags when several drives pull current at once.

It was plug 4 on that one strand. Under multi-drive load its voltage sagged, and the drives on it browned out. I rotated the plugs, retired plug 4, and now run on plugs 1 to 3. I also ordered spares so the next fix doesn't wait on a delivery: a SATA 1-to-2 splitter, a Molex-to-2xSATA adapter for a second strand, spare SATA data cables, and a 24-pin ATX jumper.

Proving it, not hoping

Swapping a plug and waiting a week to see if it happens again is not a test. The fault only showed under load on several drives, so the proof had to be that load: a parallel dd read on all four drives at the same time, 48 GB of it. Then the same dmesg grep.

Zero errors. No DF, no ABRT, no link resets. The same kind of load that had knocked drives off the bus now ran clean on every port at once. That was enough for me to trust the array with a rebuild.

Rebuilding onto the same drive

disk1 was disabled but healthy. unRAID disables a disk when a write to it fails, and after that it trusts parity over the disk's contents. Nothing was wrong with the drive, so there was no reason to buy a new one. You just have to convince unRAID that the same drive is a replacement:

1. Stop the array and unassign disk1.
2. Start the array, so unRAID records the slot as empty.
3. Stop the array again.
4. Reassign the same drive to disk1.
5. Start. unRAID sees a "new" disk and rebuilds it from parity.

The rebuild took 8 h 15 min with 0 errors. That run was a second proof in itself: hours of every drive reading at once, and the power path held.

While it runs, this is the one line I watch:

grep -E "mdResyncAction|mdResyncPos|mdResyncSize|rdevStatus|rdevNumErrors|sbSyncErrs" /proc/mdstat

rdevNumErrors per disk and sbSyncErrs should stay at zero. After a restart, the md: import diskN lines in syslog show which drive went into which slot, which is how I confirm the reassignment took.

Don't order three drives. Read the error types before the error count. UNC means the disk. DF and ABRT with link drops across several drives mean something they share, and power is the cheapest thing to check and the easiest to miss. It is also the one thing SMART can't see.