Rainbow Tables
whoami
●
Andrew Kramer
●
Dakota State University
●
Grad Student in Cyber Operations
●
SFS Scholarship Recipient
●
[email protected]
INDEX
●
Hash Functions
●
Cracking Hashes
●
●
–
CPU/GPU Bruteforce
–
Precomputed Dictionaries
Rainbow Tables
–
Reduction Functions
–
Table Generation
–
Hash Lookup
–
Potential Problems
Related Information
INDEX
●
Hash Functions <<<
●
Cracking Hashes
●
●
–
CPU/GPU Bruteforce
–
Precomputed Dictionaries
Rainbow Tables
–
Reduction Functions
–
Table Generation
–
Hash Lookup
–
Potential Problems
Related Information
What Is a Hash Function?
●
Different than “encryption”
●
One-way
●
Irreversible (sort of)
●
Arbitrary length in, fixed length out
●
Pseudo-random
●
Few collisions
Example Hash Function
●
Convert all characters to numbers, i.e...
–
A=1
–
B=2
–
Etc...
●
Add all the numbers
●
Divide the sum mod 26
●
Represent as two digits (zero padded)
Example Hash Function
●
●
“foobar”
–
6 + 15 + 15 + 2 + 1 + 18 = 57
–
57 mod 26 = 5
–
Therefore: MYHASH(“foobar”) = 05
“catdog”
–
3 + 1 + 20 + 4 + 15 + 7 = 50
–
50 mod 26 = 24
–
Therefore: MYHASH(“catdog”) = 24
Real Hash Functions
●
●
MD5
–
128 bits (16 bytes)
–
Commonly represented as 32 hex characters
–
md5(“foobar”) = 3858f62230ac3c915f300c664312c63f
SHA1
–
160 bits (20 bytes)
–
Commonly represented as 40 hex characters
–
sha1(“foobar”) =
8843d7f92416211de9ebb963ff4ce28125932878
Why hash passwords?
●
[email protected]:i<3computers
[email protected]:Password1!
[email protected]:kittiesrule
Etc...
●
[email protected]:5f306463f...
[email protected]:ff4c22878...
[email protected]:a97fe3c...
Etc...
INDEX
●
Hash Functions
●
Cracking Hashes <<<
●
●
–
CPU/GPU Bruteforce
–
Precomputed Dictionaries
Rainbow Tables
–
Reduction Functions
–
Table Generation
–
Hash Lookup
–
Potential Problems
Related Information
What's a Poor Hacker To Do?
●
Two simple ways to recover plaintext
–
–
On-the-fly bruteforce
●
Hashcat (CPU or GPU)
●
John the Ripper (CPU)
●
Wordlists (CPU or GPU)
Precomputed lookup tables
●
aaaaaa:0b4e7a0e5fe84ad35fb5f95b9ceeac79
●
aaaaab:9dcf6acc37500e699f572645df6e87fc
●
aaaaac:52a0a42bc3e1675eccb123b56ea5e3c8
●
Etc... for gigs... and gigs... and gigs...
INDEX
●
Hash Functions
●
Cracking Hashes
●
●
–
CPU/GPU Bruteforce <<<
–
Precomputed Dictionaries
Rainbow Tables
–
Reduction Functions
–
Table Generation
–
Hash Lookup
–
Potential Problems
Related Information
Raw Bruteforce?
●
Sooooo slooow
–
4GHz 8-core CPU = ~74Million MD5s / sec
–
[a-zA-Z0-9] ^ 10 = 83,929,936,586,800,000
–
~36 years :(
●
Duplicate work for every hash
●
The power bill...
INDEX
●
Hash Functions
●
Cracking Hashes
●
●
–
CPU/GPU Bruteforce
–
Precomputed Dictionaries <<<
Rainbow Tables
–
Reduction Functions
–
Table Generation
–
Hash Lookup
–
Potential Problems
Related Information
Precomputed Lookup Lists?
●
Requires storage space in the...
–
Terabytes?
–
Petabytes??
–
Exabytes ???
–
Forget it
●
●
[10 chars] + [16 bytes] + [1 bytes '\n'] over [a-zA-Z0-9]
(10 + 16 + 1) * (26 + 26 + 10)^10 =
2,266,108,287,840,000,000 bytes =
~ 2.266 exabytes :(
INDEX
●
Hash Functions
●
Cracking Hashes
●
●
–
CPU/GPU Bruteforce
–
Precomputed Dictionaries
Rainbow Tables <<<
–
Reduction Functions
–
Table Generation
–
Hash Lookup
–
Potential Problems
Related Information
Rainbow Tables
●
Best of both worlds
●
Time-memory trade off
●
A few hundred gigs + a few hours of CPU
–
Same coverage as a hundred years of CPU
–
Same coverage as a petabyte of lookup lists
Or More Visually...
INDEX
●
Hash Functions
●
Cracking Hashes
●
●
–
CPU/GPU Bruteforce
–
Precomputed Dictionaries
Rainbow Tables
–
Reduction Functions <<<
–
Table Generation
–
Hash Lookup
–
Potential Problems
Related Information
Reduction Function
●
“Opposite” or a hash function
●
Input hash... output a plaintext
–
NOT THE SAME PLAINTEXT
●
Fixed length in... arbitrary length out
●
Ideally also pseudo-random
Example Reduction Function
●
For charset=[0-9], length=1-10
–
Find all digits in the hash
–
First digit represents length (0 = 10)
–
Other digits (up to length) represent plaintext
–
MYREDUCE(“52a0a42bc3e1675eccb123b56ea5e3c8”)
= 20423
–
MYREDUCE(“9dcf6acc37500e699f572645df6e87fc”)
= 63700699
–
MYREDUCE(“0b4e7a0e5fe84ad35fb5f95b9ceeac79”)
= 4705843559
INDEX
●
Hash Functions
●
Cracking Hashes
●
●
–
CPU/GPU Bruteforce
–
Precomputed Dictionaries
Rainbow Tables
–
Reduction Functions
–
Table Generation <<<
–
Hash Lookup
–
Potential Problems
Related Information
We Can Chain Hash/Reduce!
●
plaintext -hash()-> HASH -reduce()-> plaintext -hash()-> HASH
●
Do this 1,000,000 times
●
Only store the FIRST plaintext and LAST hash
●
●
●
–
17829: ...lots of iterations... :52a0a42bc3e1675eccb123b56ea5e3c8
–
15186: ...lots of iterations... :9dcf6acc37500e699f572645df6e87fc
–
123037: ...lots of iterations... :0b4e7a0e5fe84ad35fb5f95b9ceeac79
Each of these chains actually contains 1,000,000
password->hash transitions
Now do THAT 1,000,000 times
Congratulations! You stored 1,000,000,000,000
combinations in the space of 1,000,000
INDEX
●
Hash Functions
●
Cracking Hashes
●
●
–
CPU/GPU Bruteforce
–
Precomputed Dictionaries
Rainbow Tables
–
Reduction Functions
–
Table Generation
–
Hash Lookup <<<
–
Potential Problems
Related Information
Looking Up a Password
●
Hash the password
–
●
Reduce and rehash
–
●
●
●
If one of your chains ends in the hash, you know the password
was the second to last iteration of that chain.
Reduce and rehash
–
●
If one of your chains ends in the hash, you know the password
was the last iteration of that chain.
If one of your chains ends in the hash,... third to last
Etc... 1,000,000 times
Once you find a matching hash, simply walk that chain
by hashing and reducing until you hit your password
Congratulations! You just covered 1,000,000,000,000
combinations with only 1,000,000 hash cycles
INDEX
●
Hash Functions
●
Cracking Hashes
●
●
–
CPU/GPU Bruteforce
–
Precomputed Dictionaries
Rainbow Tables
–
Reduction Functions
–
Table Generation
–
Hash Lookup
–
Potential Problems <<<
Related Information
Problem #1: Collisions
●
●
●
When you go from a small plaintext to a large
hash, it's unlikely that two plaintexts will
produce the same hash.
When you go from a large hash to a small
plaintext, it's quite likely that two hashes will
generate the same plaintext.
Our chains will be constantly colliding (merging)
Why Are Collisions Bad?
●
LOTS of duplicate work
●
Chains will have to be very small
●
Hard to detect
●
End up throwing out lots of hard work
Varied Reduction Functions
●
Reduce1(), reduce2(), reduce3(), repeat...
●
Use them in sequence
●
Lowers the chances of collision
●
And if you represent each function with a color...
●
That's why they call it a rainbow table :)
Problem #2: Detecting Collisions
●
●
●
Unfortunately, a few collisions WILL still
occur
We need to throw out those chains to
achieve maximum efficiency
How do we know if there has been a
collision without wasting lots of CPU
work?
Collision Detection Solution
●
●
●
Stop each chain at a predefined point
For example, when the first 8 characters
are “0”
Every time a chain ends, search the table
for a match
–
●
●
If you find one, throw out the shorter chain
Slightly more work = way better efficiency
Remember: your chains are different
lengths, so you need to search up to the
specific chain length
Rainbow Table Drawback #1
●
You must do extra work up front
–
–
You will inevitably hash more than the
keyspace
●
Collisions
●
Duplicates
However, once the rainbow table has been
generated, the work pays off
Rainbow Table Drawback #2
●
You will never cover 100% of your
keyspace
–
Good design allows in the >=99.9% range
–
You WILL miss passwords here and there
–
The exact numbers and probability can be
estimated.
Rainbow Table Downside #3
●
Doesn't work for salted hashes :(
–
Because, two of the same password will
generate different hashes
●
Breaks our predictable keyspace
●
Breaks our chains
●
Breaks everything
●
PSA: If you store other people's
passwords, please make sure they are
hashed AND salted!
INDEX
●
Hash Functions
●
Cracking Hashes
●
●
–
CPU/GPU Bruteforce
–
Precomputed Dictionaries
Rainbow Tables
–
Reduction Functions
–
Table Generation
–
Hash Lookup
–
Potential Problems
Related Information <<<
Other Resources
●
https://freerainbowtables.com/
–
Distributed rainbow table generation
●
Help out! Donate some CPU cycles!
–
rcracki_mt: Multi-threaded rainbow table lookup
–
Lots of tables available for free download
–
●
MD5 / SHA1 / LM / NTLM
●
Bittorrent downloads are lightning fast
Will ship you the same tables on HDD (for $)
●
Total of about 12 TB
●
$1,200
Other Great Information
●
●
●
http://kestas.kuliukas.com/RainbowTables/
http://stichintime.wordpress.com/2009/04/09/rai
nbow-tables-part-1-introduction/
https://www.freerainbowtables.com/en/faq/
EOF
●
Thanks!
●
Questions?
●
Comments?
●
Andrew Kramer – Dakota State University
●
[email protected]