How to check files against a manifest on Mac

You have a list: a few hundred lines of digest and path, exported when the archive was still warm, or downloaded from a publisher who put a SHA256SUMS next to their files. And you have the folder itself, some months older and possibly none the better for it. Lining the two up is mechanical. Reading the result properly takes a little more.

Start by deciding which question you are asking, because they are not the same question and they do not have the same answer:

1. Has a file in the list changed? Today’s digest for it will not be the digest on its line.
2. Is a file in the list gone? Then there is no digest for it today at all, and nothing to compare.
3. Has anything been added to the folder since? Nothing inside the list can tell you. A file that did not exist has no line.

This page covers all three: the single-line check that takes seconds, the whole-folder run that produces today’s list, and how to read the two lists against each other. If you have not made a list yet, exporting a checksum manifest is the other end of the same workflow, and what a SHA256SUMS file contains takes one line apart if the format is new to you.

What a check can and cannot see

The mechanism is worth holding in your head, because it explains every surprise that follows. A manifest is a set of claims about content, one claim per line. Checking it means producing today’s digest for the same file and seeing whether the two strings agree. Nothing in the list describes the folder as a whole: not how many files belong in it, not their dates, not their sizes.

The third row is the one that bites. A clean result against an old list says “everything I wrote down is still as I wrote it” — which is a genuinely useful sentence, and not at all the same as “this folder is unchanged”.

Checking a single line

When one file is the reason you are here — a download, a delivered master, the one file somebody has raised a question about — the folder does not come into it. Copy the digest out of its line in the manifest, open the Verify tool in Rocket Hash, leave the segmented control on Against a Checksum, paste the digest where it asks for one and drop the file in. The answer is a green seal and a sentence rather than two strings to compare by eye.

The algorithm comes from the digest you pasted rather than from anything you set, so a 40-character line is read as SHA-1 and a 64-character one as SHA-256 without your having to know which list you are working from. The file is read once, end to end, at whatever rate the disk can feed it — a few seconds for a document, a couple of minutes for a disk image. Verifying a SHA-256 checksum walks that route slowly, including where published digests hide.

Check a folder against a manifest

For the list as a whole, the move is not to check it line by line but to produce today’s list and compare the two. Four steps, one of which is waiting.

1. Read one line of the list

   Open the manifest in any text editor and look at a single line. The first column is the digest, and its length tells you which algorithm made it: 64 hexadecimal characters is SHA-256, 128 is SHA-512, 96 is SHA-384, 40 is SHA-1, 32 is MD5, 8 is CRC32. You need that before anything else, because today’s run has to answer the same question the list answered. Identifying which algorithm a checksum uses covers the two lengths that are ambiguous.

2. Re-hash the whole folder

   Drag the top of the tree into Files — a folder brings everything underneath it — and put the Algorithm control, the one marked with a #, on whatever the digest length told you. Set it to something else and today’s list disagrees with the manifest on every single line while nothing at all is wrong. A hundred thousand files is an ordinary size for this rather than an extreme one, and memory stays flat at roughly 16 MB however many there are, because each file is streamed rather than loaded.

3. Compare the counts before the digests

   Wait for the two halves of the status bar to agree: the summary on the left counts what is loaded, the progress on the right counts what is hashed. Then put that file count against the number of lines in the manifest, before you read a single digest. More files than lines means something arrived after the list was written, which is the one finding the list itself could never produce. Expect a modest surplus regardless — .DS_Store files are real files with real bytes, and they get rows too.

4. Export today’s list and set the two side by side

   Export the finished run and open it next to the old manifest: two text files of the same shape, one per moment. From here the job is reading, and the next section is what to read for. Keep both files afterwards — today’s list is also next year’s manifest, and a dated pair is the only thing that can ever tell you when a file changed.

Reading the two lists side by side

Match lines on the path, never on the position. Export order follows the run rather than the alphabet, so the same file can sit on line 40 of one list and line 11 of the other without anything at all having happened to it. Once you are comparing by path, four shapes appear and each has exactly one meaning.

9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08  raw/DSC_0041.NEF
2c26b46b68ffc68ff99b453c1d30413413422d706483bfa0f98a5e886266e7ae  raw/DSC_0042.NEF
fcde2b2edba56bf408601fb721fe9b5c338d10ee429ea04fae5511b68fbf8fb9  exports/contact-sheet.pdf

Say today’s export repeats the first of those lines exactly, carries a different digest against the second path, holds no line at all for the third, and adds one for a file that did not exist in March. That is one of each shape:

• Same path, same digest. Untouched, byte for byte. This is what you want on every line, and it is the only shape that proves anything.
• Same path, a different digest. The contents changed. Dates, tags and permissions cannot do this; only the bytes can.
• A path in the old list and nowhere in the new one. Deleted, renamed, or moved out of the tree.
• A path only in the new list. Added since — the case the old manifest was structurally incapable of reporting.

A fifth shape is worth recognizing: the same digest on two different paths. That is not damage, it is a duplicate — the same bytes living in two places, which is information rather than a fault, and the whole basis of finding duplicate files by hash. And when a digest disappears from one path and turns up on another in the same comparison, nothing was damaged at all: a file was renamed or moved, and its bytes came through it intact.

When the folder has moved

This is the most common practical obstacle and it has nothing to do with hashing. The digests in your manifest are still perfectly valid. The paths are not, because the archive now lives on a different volume, or macOS has mounted the same drive as Archive 1 after an untidy eject.

Nothing needs rewriting. You give the app the files — drag the folder in from wherever it is now, or drop one file into Verify — so nothing ever tries to resolve the path the list recorded. Compare the digests and ignore the prefix: the first column is the claim, and the second is a note about where the file was standing when the claim was made.

It is still worth noticing which kind of list you were handed, because it decides how much of the path you can usefully compare. Lines reading raw/DSC_0041.NEF line up against a fresh run of the same tree on any machine and any volume. Lines reading /Volumes/Archive/raw/DSC_0041.NEF will differ from today’s export in their prefix on every line, while the digests and the tail of each path still match exactly — so read the tail, and do not let a changed mount point look like a changed archive.

What a clean check does not prove

Three honest limits, in descending order of how often they matter.

It does not prove the folder is complete: a file created since the list was written has no line in it to contradict. It does not prove anything about a file’s metadata: a digest covers content, so a photo whose tags, permissions or modification date changed passes cleanly, and a file restored from a backup arrives with new dates and the same digest. And it does not prove the manifest is honest. If you downloaded the list from the same place as the files, a clean result establishes internal consistency and nothing about origin — which is why distributions sign their manifests, and why detecting tampering is a different page from this one.

Troubleshooting

Every digest is different

Nearly always the Algorithm control does not match the list. Count the characters in one line’s first column and set the control to the same algorithm; a SHA-256 run compared against a SHA-512 manifest disagrees everywhere and means nothing. The other cause is that you are hashing a different copy of the tree than the one the list was made from — material that was re-exported or re-saved rather than copied is new bytes, and every line of it will differ honestly.

A file in the list is not in the folder

It was renamed, moved, deleted, or it sits behind a permission macOS has not granted yet. Before you go looking for a backup, check whether its digest appears in today’s list on some other path: that is the difference between a missing file and a missing name, and renaming is far more common than loss. If you are deliberately checking a subset — one download out of a published list of fourteen — the absent lines are not findings at all.

Today’s run has more files than the list has lines

Usually not a mystery. .DS_Store appears in every folder you have ever opened in the Finder, ._ companion files appear on drives formatted for Windows, and an app or a bundle you dropped in counts as every file inside it. They are real files with real bytes, so they get rows and lines. When the gap is that kind of gap, compare the named files you care about rather than the totals.

The run is taking hours

It is reading every byte of every file, so the floor is however long it takes to read the whole archive once — the arithmetic is not the bottleneck, the disk is. An external drive over USB, a spinning disk or a network share can each be an order of magnitude slower than an internal SSD, and the throughput figure on the right of the status bar tells you which you are dealing with within a few seconds. Why hashing is slow covers what to do about it; a long run can also be paused and picked up from the same byte, so it does not have to be one sitting.

Permission denied partway through

macOS keeps Desktop, Documents, Downloads and external volumes behind explicit permission, handed out a category at a time the first time something reaches into one. A run that stops partway has usually crossed into a folder nobody has approved yet, which is also why the same archive can run cleanly under one account and stop under another. Permission problems when hashing a folder has the System Settings switches that matter.