My Glacier Cleanup Deleted Nothing for 49 Runs: rclone One-Way Checks and Deep Archive Orphans

My Deep Archive backup has a cleanup step that deletes files I no longer have locally. It ran 49 times. It deleted nothing, every single time, and its own status message said so: Cleaned: 0.

Nothing was wrong with the delete. The list of files to delete was always empty, because I asked rclone for a list it never writes.

TL;DR: rclone check --one-way --missing-on-src can never fill that file, because --one-way means "only look from the source side". I replaced it with rclone lsf on both sides and comm -13, guarded by an orphan cap, a 180 day age check on the S3 upload time, and a skip whenever the copy failed. The run cadence is now a 66 h marker file instead of a cron schedule.

I set this pipeline up in an earlier post: rclone crypt into S3 Glacier Deep Archive, encrypted names and content. This post is about the part that post did not cover. What happens to files on the remote after they disappear at home.

Why the cleanup exists at all

The upload is rclone copy, per directory. It only adds new or changed files and never deletes. That is deliberate for an archive, but it means the remote grows forever. A photo I delete, a folder I move, a directory I start excluding: all of it stays in Deep Archive and bills monthly.

So after each copy, a second step finds orphans. An orphan is a file on the remote that the copy would not upload today, either because it is gone locally or because it is now excluded.

The v2 bug

v2 built the orphan list like this (simplified):

rclone check /source "$remote" --one-way --missing-on-src orphans.txt

It looks right. --missing-on-src is exactly "on the remote, not local". But --one-way tells check to only verify that source files exist and match on the destination. It never walks the destination looking for extras. The flag that would fill orphans.txt is the one direction I had switched off.

Nothing errored. check exits 1 on differences, and the script correctly treated 1 as normal and 2 and up as a real error. The delete step got an empty file, deleted nothing, and reported success.

An audit of all my unRAID user scripts found it on 2026-09-03. I ran that audit as a 14-agent workflow: one auditor per group of scripts, then a skeptic per group that tried to refute each finding. The same pass found a second dead step: v2's multipart cleanup pointed at a bucket that no longer existed, so it failed on every run too.

The v3 orphan logic

Now each side gets a plain listing, and comm does the diff:

rclone lsf -R --files-only /source  "${RCLONE_EXCLUDES[@]}" > local.txt
rclone lsf -R --files-only "$remote"                          > remote.txt
LC_ALL=C sort -o local.txt local.txt
LC_ALL=C sort -o remote.txt remote.txt
LC_ALL=C comm -13 local.txt remote.txt > orphans.txt

A few details that matter:

• The local listing uses the same excludes as the copy. That is what makes "now excluded" count as an orphan. Add node_modules/** to the excludes and those files eventually leave Deep Archive too.
• The remote is the crypt remote, so lsf returns decrypted paths. Both lists are in the same namespace.
• comm needs both inputs sorted in the same collation. LC_ALL=C on the sorts and on comm itself removes any locale surprise.
• comm -13 suppresses lines only in the first file and lines in both. What is left is "only on the remote".

Guards before anything is deleted

A diff between two listings is only as good as the local listing. A missing disk, an unmounted share, or a half-read directory would make everything look like an orphan. So the script refuses in several places:

1. The copy failed: skip the orphan step for that directory (continue). Never delete remote files for a directory the script just failed to read.
2. The local listing is empty but the directory is not: count it as a failed copy and skip.
3. More than ORPHAN_MAX orphans (20000) in one directory: log a WARN and skip the delete. That many at once means a broken listing, not a cleanup.
4. Any listing command failed: skip and count a cleanup warning.

Then the delete itself:

rclone delete "$remote" \
  --files-from-raw /cleanup/orphans_$tag.txt \
  --min-age 180d \
  --use-server-modtime

--files-from-raw reads the list as literal paths, without the comment and whitespace handling of --files-from, which is what you want for machine-generated file lists.

The 180 day window, and the trap in it

Deep Archive has a 180 day minimum billing period, so deleting a file earlier saves nothing. --min-age 180d keeps anything younger. An orphan that is too young stays, shows up as an orphan again next run, and is deleted once it passes 180 days.

The trap is which age. rclone crypt preserves the original file's modification time. A photo from years ago, uploaded last week, has an mtime from years ago. Without --use-server-modtime, --min-age would see an old file and delete it right away. With it, the age is the S3 upload timestamp, which is the one the billing clock uses.

The 128 KB floor

Small files make the math worse. Deep Archive bills a 128 KB minimum per object.

That became real when my Thunderbird mail sync started landing a maildir in one of the backed-up folders: 30 GB, 324,539 .eml files, averaging about 92 KB each. Every file under the floor bills as 128 KB, so the 30 GB bills as about 41 GB, plus a one-off PUT charge for the first upload.

It also changes how you think about orphans. A mail client that moves or rewrites messages turns into a stream of tiny remote objects, each billed at the floor, each held for 180 days before the cleanup may touch it. If that ever matters, the fix is to tar the maildir before it reaches Glacier, so it is one object instead of 324k.

A run marker instead of a schedule

The pass should run every third day. Cron fires daily at 06:00, and the script decides whether today is the day:

MIN_AGE_MIN=4000   # 66 h: day 1 and day 2 skip, day 3 runs
[ -n "$(find "$OK_MARKER" -mmin -"$MIN_AGE_MIN")" ] && exit 0

At start it touches glacier-backup.last-ok.pending. Only a fully successful run moves that file over the marker, so the marker's mtime is the start time of the last good run. 24 h and 48 h later it is too young; at 72 h it is not.

The point is what happens to a bad run. A run killed by a reboot or an outage never moves the marker, so the next morning it simply runs again, not three days later. A hand run in the afternoon shifts the cadence by one day, which I can live with.

That is exactly the failure it exists for. On 2026-09-03 the 06:00 pass was killed by a 10:09 reboot: Failed: 6, and no status push at all.

Other small fixes in the same rewrite: flock on a lock file that is never deleted (a skipped run used to delete the running run's lock), docker exit 125 aborting the run with one down push instead of failing every directory in a second, and logs older than 90 days pruned at start. Multipart cleanup now runs as rclone cleanup on the crypt remote, which hands it to the S3 backend.

The status line still ends with Cleaned: N. This time I know what zero means.