Skip to main content
Be Bitwise

Common Hash Functions: MD5, SHA & Beyond

Hashing Fundamentals · Data & Hashing track

6 min read

In this module

The Hash Function Zoo

Decades of hash functions have been designed, broken, and replaced. Knowing which ones are still in service and why the old guard got retired is the kind of practical knowledge that pays off in real systems.

MD5 (Message Digest 5)

Output size: 128 bits (32 hex characters)

Ron Rivest designed MD5 in 1991, and for years it was the workhorse hash. It chews through input in 512-bit blocks across four rounds of bitwise logic, modular addition, and rotations. Each round applies a different non-linear function to mix the block into the running state. Once every block has been processed, what's left in the state is the 128-bit hash.

text
MD5("Hello") = "8b1a9953c4611296a827abf8c47804d7"

Status: Broken for cryptographic use. Researchers demonstrated practical collision attacks in 2004. By 2008, attackers were forging fake SSL certificates using MD5 collisions. MD5 should never touch a security-sensitive system today.

It still has a job in non-security checksums — verifying file downloads against accidental corruption — because it's fast and random corruption almost never produces a collision. But for anything with an adversary, MD5 is dead. The attacks have only gotten cheaper. On modern hardware, an MD5 collision falls out in seconds.

SHA-1 (Secure Hash Algorithm 1)

Output size: 160 bits (40 hex characters)

SHA-1 was designed by the NSA and published in 1995. For more than a decade it was the standard cryptographic hash, baked into SSL certificates, Git, and digital signatures. Under the hood, it processes 512-bit blocks through 80 rounds of mixing built from rotations, XOR, and modular addition.

text
SHA-1("Hello") = "f7ff9e8b7bb2e09b70935a5d785e0cc5d9d0abf0"

Status: Broken. In 2017, Google and CWI Amsterdam pulled off the first practical SHA-1 collision (the "SHAttered" attack), producing two different PDF files with the same SHA-1 hash. It took roughly 2^63 computations — enormous, well within reach of any well-funded adversary. SHA-1 has no business in new security applications.

Git still uses SHA-1 to identify objects in almost every repository. SHA-256 repositories exist (git init --object-format=sha256), and Git 3.0 is slated to make them the default for new repositories, but the switch has no release date yet and older clients and many forges can't read them. The collision attack means that, in theory, someone could engineer two different repository states with the same commit hash. Deeply unsettling for a version control system.

Key Concept

Both MD5 and SHA-1 have fallen to practical collision attacks. Fine for non-adversarial checksums, useless for security-sensitive work — use SHA-256 or SHA-3 there. The lesson: hash functions once considered unbreakable can buckle under advances in mathematics and computing power.

SHA-2 Family (SHA-256, SHA-512)

Output sizes: 224, 256, 384, or 512 bits

SHA-2 came out of the NSA in 2001. SHA-256 is the variant you'll meet most often, producing a 256-bit hash. It runs on 512-bit blocks through 64 rounds of compression. Each round applies logical functions (Ch, Maj, Sigma) alongside round constants pulled from the fractional parts of the cube roots of the first 64 primes — yes, really.

text
SHA-256("Hello") = "185f8db32271fe25f561a6fc938b2e264306ec304eda518007d1764826381969"

Status: Secure. No practical attacks are known. SHA-256 powers TLS certificates, Bitcoin's proof-of-work, digital signatures, and a long tail of other applications. The best known attack is still plain brute force — 2^128 operations to find a collision, a number so large no foreseeable hardware reaches it.

SHA-512 is the same shape, just bigger: 1024-bit blocks, 80 rounds, 512-bit output. On 64-bit processors it actually outpaces SHA-256, because it chews through 128-byte blocks per pass and uses 64-bit arithmetic natively.

SHA-3 (Keccak)

Output sizes: 224, 256, 384, or 512 bits

SHA-3 came out of a public competition (2007-2012), commissioned as an insurance policy in case SHA-2 ever broke. Internally, it's a completely different beast: the sponge construction. Where MD5, SHA-1, and SHA-2 all use the Merkle-Damgard structure, the sponge "absorbs" input data into a large internal state and then "squeezes" out the hash. An attack that one day cracks SHA-2's design wouldn't automatically carry over, and length-extension attacks, which work against MD5, SHA-1 and SHA-256, don't apply to SHA-3 at all.

text
SHA-3-256("Hello") = "8ca66ee6b2fe4bb928a8e3cd2f508de4119c0895f22e011117e22cf9b13de7ef"

Here are the two structures next to each other. In Merkle-Damgard, a compression function f folds each message block into a fixed-size chaining value, and the final chaining value is the hash. In the sponge, each block is XORed into part of a larger state (the rate), the whole state goes through a permutation f, and at the end only part of the state is read out. The rest (the capacity) never leaves.

Each f on the left also takes the next message block (the last one carrying padding and the message length). That last detail has a consequence. Because a Merkle-Damgard hash hands out its entire final state, anyone who knows hash(secret + message) and the total length can carry on from that state, feeding in more blocks and computing hash(secret + message + padding + extra) without ever learning the secret. That's a length-extension attack, and it's why hash(key + message) is a broken way to authenticate messages with SHA-256. The sponge never reveals its capacity, so there's no state to resume from, and SHA-3 isn't affected.

Status: Secure. SHA-3 sits on a different mathematical foundation from SHA-2. Having both means a breakthrough against one family doesn't sink everything at once. The diversity is deliberate engineering — if the assumptions underlying SHA-2 ever fall, the world can migrate to SHA-3 without starting from scratch.

Non-Cryptographic Hash Functions

For hash tables and data structures, speed matters more than cryptographic security:

MurmurHash3: Very fast, good distribution, widely deployed in hash tables. Not cryptographic — if an attacker controls the input, collisions are trivial.

xxHash: Extremely fast — chews through data at memory-bandwidth speeds. Common in databases and file systems for checksumming.

CityHash / FarmHash (Google): Tuned for short strings. Used inside Google for hash table indexing.

SipHash: Built to resist hash-flooding DoS attacks while staying fast. Default for dictionary hashing in Python, Rust, and Ruby. SipHash is a keyed hash function — it takes a secret key, so attackers can't predict hash values and engineer adversarial inputs.

These are optimised for throughput and distribution quality, not for shrugging off deliberate collision attacks from a sophisticated adversary.

Choosing the Right Hash

The choice depends entirely on your threat model:

  • File integrity checking (no adversary): MD5 or SHA-256 both work. MD5 is faster.
  • Digital signatures, certificates: SHA-256 minimum.
  • Password storage: Not a general hash - use bcrypt, scrypt, or Argon2 (these are deliberately slow).
  • Hash table indexing: MurmurHash, xxHash, or SipHash.
  • Git-style content addressing: SHA-256 (Git supports it and plans to make it the default, but most repositories still run on SHA-1).
  • Blockchain: SHA-256 (Bitcoin), Keccak/SHA-3 (Ethereum).

Knowing this landscape prevents two common mistakes: using a broken hash where security matters, and using a slow cryptographic hash where speed matters. The right hash for the job depends on whether your adversary is random noise or a determined attacker.