Shrinking an unRAID Array from 3 Data Disks to 2 and Keeping the Old Disk as a Spare

The usual answer to a tired disk is to buy a new one. Disk prices are high right now, so I did the opposite: I took a disk out of my unRAID array and kept the array at two data disks plus parity.

The old disk didn't go in a drawer either. It sits in the fourth bay, unassigned, as a spare I can copy to, with a sealed new drive on the shelf behind it.

TL;DR: empty the disk with unBALANCE, run New Config with preserve all, unassign the disk, and let parity rebuild. Budget a night for the rebuild, and remember the array has no protection until it finishes. Then clean up what the move leaves behind: root-owned files, cache files the mover will never touch, and a ZFS dataset that won't die.

Why this disk, and why out

A few days earlier a bad SATA power plug had knocked three drives off the bus. That one was power, not the disks, and I wrote it up in Three unRAID Disks Failed at Once. It Was a SATA Power Plug.. But it made me look hard at every drive, and disk3 stood out:

The other three are WD40PURX. Eight uncorrectable sectors is not a disk that dies tomorrow. It's also not a disk I want holding one third of my data with only one parity drive behind it. The array gets smaller, and the old disk keeps a useful job as a backup target where a bad sector costs me a copy, not my library.

Step 1: empty the disk with unBALANCE

The unBALANCE plugin has two modes. Gather pulls one share's scattered folders onto a single disk. Scatter pushes a disk's contents out onto other disks, which is the one you want here. I scattered disk3's media to disk2 with the move operation, not copy, so disk3 ended up empty and nothing existed twice.

Moving between array disks goes disk to disk under /mnt/diskN, and the user share at /mnt/user shows the same paths before and after.

Step 2: New Config, preserve all, unassign

The array layout is the part that scares people, so be precise:

1. Stop the array.
2. Tools > New Config, with "preserve current assignments" set to all. That keeps parity, disk1 and disk2 in their slots, so there is no chance of assigning a data disk to the parity slot by mistake.
3. Unassign disk3.
4. Start the array. Do not tick "parity is already valid".

That last point is the whole mechanism. Parity is XOR across every data disk. Take one out and the parity bits no longer match what remains, unless the removed disk was all zeros. So parity has to be rebuilt from the two data disks that are left.

The rebuild took 8 h 50 min with 0 sync errors. For those hours there was no redundancy at all, and that is the real cost of this method. It is also why I did it only after the power fix was proven under load and disk1 was rebuilt. To watch it:

grep -E "mdResyncAction|mdResyncPos|mdResyncSize|rdevStatus|rdevNumErrors|sbSyncErrs" /proc/mdstat

Where the drives are now

If an array disk fails, the sealed drive goes in. The old disk3 stays for copies I can afford to lose.

Gotcha 1: files owned by root

Some of my media files were root:root. A container running as root had written them, and the rest of my stack runs as unRAID's standard nobody:users and couldn't manage them. The fix was to match the unRAID default: owner nobody:users, directories 777, files 666. It's loose, but it's the standard unRAID share layout, and every container that runs as nobody can work with it again. The real fix is to stop running containers as root when they write into shares.

Gotcha 2: the mover skips shares set to "no"

I set the media share to shareUseCache="no", since it lives on the array now. What I didn't expect: the mover ignores a share with the cache set to "no". Files that were already on the cache before the change stay there for good, and /mnt/user keeps showing them, so nothing looks wrong.

Four files, 15 GB, were stuck like that. Moving them by hand is three steps, and the middle one is the one people skip:

rsync -a /mnt/cache/media/ /mnt/disk2/media/
rsync -ai --checksum --dry-run /mnt/cache/media/ /mnt/disk2/media/

The first copies the files. The second compares every file by content, and -i makes it print one line per file that differs. A dry run that prints nothing means the copies are identical. Without -i or -v a dry run is silent either way, so it proves nothing. Only then do I delete the cache side. To check later that nothing is left:

find /mnt/cache/media -type f

A missing path or an empty list means the share is fully on disk2.

Gotcha 3: "dataset is busy"

My cache pool is ZFS, and each cache share is its own dataset (zfs list -r cache). So rm -rf on the empty share folder fails with "busy". It's a mount point, not a folder. The right tool is zfs destroy cache/media.

That also said "dataset is busy", with no holds on it. The cause: a container that bind-mounts /mnt/user still had the old mount in its own mount namespace. The host no longer uses the dataset, but that namespace still does. The mount table of each process shows who:

grep -l media /proc/[0-9]*/mounts

That gave me a PID. The PID belonged to homepage, my dashboard container. I restarted it, and zfs destroy cache/media went through.

The array is one disk smaller and runs clean, with two spares in the right places. I didn't buy anything to get there.