What Is SHA256 Hashing?
SHA256 belongs to the SHA-2 family of hash functions designed by the National Security Agency (NSA) and published by NIST in 2001. The algorithm processes input data in 512-bit blocks through 64 rounds of mathematical transformations, producing a 64-character hexadecimal string as output.
A hash function is a one-way operation. You can compute the hash of any file instantly, but you cannot reconstruct the original file from its hash. This property makes SHA256 useful for verifying file integrity. If two files produce the same SHA256 hash, their content is identical. If their hashes differ by even one character, the files are different.
For duplicate detection, SHA256 is ideal because it is fast enough to hash thousands of files per second on modern hardware, and its output is compact enough to compare millions of hashes in memory. Apple's CryptoKit framework provides hardware-accelerated SHA256 on Apple Silicon Macs, making the computation even faster.
Why Is SHA256 Better Than Filename Comparison?
Filename-based duplicate detection has two fundamental problems. First, it misses renamed duplicates. A photo saved as "IMG_4523.jpg" and later renamed to "Vacation Sunset.jpg" is the same file with the same content, but a filename comparison would never identify them as duplicates.
Second, filename comparison produces false positives. Two files named "document.pdf" in different folders could contain entirely different content. A filename-based tool would incorrectly flag them as duplicates, potentially leading to data loss if one is deleted.
SHA256 hashing solves both problems by ignoring metadata entirely. The hash is computed solely from the file's binary content. Two files with different names but identical content produce the same hash. Two files with the same name but different content produce different hashes. This content-based approach is the only reliable method for accurate duplicate detection.
How Does DupScan Use Two-Pass SHA256 Hashing?
The two-pass approach is a significant performance optimization. On a typical Mac with 500,000 files, most files are unique. Computing a full SHA256 hash for every file would require reading every byte of every file from disk. The partial hash pass eliminates the vast majority of files by reading only 4 KB from each one.
Files are first grouped by size because files with different sizes cannot be duplicates. This grouping eliminates most files before any hashing occurs. Within each size group, the 4 KB partial hash further reduces the candidate set. Only files that match on both size and partial hash proceed to the full hash computation.
This cascading filter approach means DupScan reads the full content of only a small fraction of files on disk. A scan that would take minutes with full-file hashing completes in seconds with the two-pass method. The results are identical in accuracy because every potential match is verified with a full SHA256 hash before being reported as a duplicate.
DupScan's features pageexplains the full technical architecture, including how CryptoKit's hardware-accelerated SHA256 implementation further accelerates hash computation on Apple Silicon Macs.
Is SHA256 Duplicate Detection 100% Accurate?
A SHA256 hash collision would require two different files to produce an identical 64-character hexadecimal string. The number of possible SHA256 outputs is 2^256, which is approximately 1.16 x 10^77. For comparison, the estimated number of atoms in the observable universe is approximately 10^80.
No SHA256 collision has ever been demonstrated in practice. Unlike its predecessor SHA-1, which was shown to be vulnerable to collision attacks in 2017, SHA256 remains cryptographically secure with no known weaknesses. Every major security protocol on the internet relies on SHA256 for integrity verification, from TLS certificates to code signing.
For duplicate file detection, the practical accuracy is absolute. If DupScan reports two files as duplicates based on matching SHA256 hashes, those files contain identical content. There is no realistic scenario in which a false positive would occur. This is why SHA256 is the standard hash function used by duplicate detection tools, package managers, and file integrity verification systems worldwide.