Fighting Unicode-Obfuscated Spam

1
004F 1:041E:О 1:004F:O 1:039F:Ο 1:FF2F:O 0.9525223:1ECC:Ọ 0.9119318:0398:Θ 0.8885794:FF31:Q
0050 1:0050:P 1:FF30:P 1:0420:Р 1:03A1:Ρ
0051 1:0051:Q 1:FF31:Q 0.8885794:041E:О 0.8885794:0398:Θ 0.8885794:039F:Ο
0052 1:0052:R
1:FF32:R 0.9553072:1E5A:Ṛ 0.9369863:0156:Ŗ 0.912:1E5E:Ṟ
0053 1:FF33:S 1:0053:S 1:0405:Ѕ 0.9522388:1E62:Ṣ 0.9327486:015E:Ş
0054 1:0422:Т 1:0054:T 1:03A4:Τ 1:FF34:T 0.9139785:1E6C:Ṭ 0.9071038:01AC:Ƭ 0.880829:0162:Ţ
0055 1:FF35:U 1:0055:U 0.943662:1EE4:Ụ 0.8933333:1E72:Ṳ 0.8729641:1E76:Ṷ
0056 1:0056:V 1:FF36:V 1:2164:Ⅴ 0.9438596:1E7E:Ṿ 0.9191176:0474:Ѵ 0.9107807:01B2:Ʋ
0021 1:FF01:! 1:01C3:ǃ 1:0021:! 0.8706896:FF4C:l 0.8706896:006C:l 0.8706896:0406:І 0.8706896:0049:I
0022 1:0022:" 1:FF02:"
Fighting
Unicode-Obfuscated Spam
0023 1:0023:# 1:FF03:# 0024 1:0024:$ 1:FF04:$
0025 1:0025:% 1:FF05:% 0.815562:2105:℅
0026 1:FF06:& 1:0026:&
0027 1:FF07:' 1:0027:'
0028 1:0028:( 1:FF08:(
0029 1:FF09:) 1:0029:)
Changwei Liu
Sid Stamm
002A 1:002A:* 1:FF0A:* 0.9673913:2217:∗
<[email protected]>
<[email protected]>
Indiana University
002B 1:002B:+ 1:FF0B:+
003C 1:003C:< 1:FF1C:< 0.856383:22D6:⋖
003D 1:003D:= 1:FF1D:=
003E 1:FF1E:> 1:003E:> 0.856383:22D7:⋗
003F 1:FF1F:? 1:003F:?
0040 1:0040:@ 1:FF20:@
0042 1:0392:Β 1:FF22:B 1:0042:B 1:0412:В
2
Evolution of SPAM
Blacklists/Whitelists
Bot Networks
Bayesian Filtering
Image Spam (removing detail)
Column Reordering
OCR
... what looks like real text but
is confusing to the filters?
3
SPAM Detection
Address-based channeling
([email protected] v. [email protected])
Duplicate Detection (FDD)
Keyword Filtering
Bayesian Filtering
(e.g., CRM114, SpamAssassin)
4
Partial-Word Unicode
Transliteration (PWUT)
which
whiϲһ
4
Partial-Word Unicode
Transliteration (PWUT)
which
whiϲһ
5
PWUT do you mean?
1,300,925,111,156,286,160,896 ways to spell “viagra”†
V
I
A
1 l | ï
ì : Ì Î
Í Ï ⅼ IІ
A А ẠA
40
30
G
R
A
A А ẠA
ᾼḀ Ά À
ᾼḀ Ά À
lⅠ l ǀ Ӏ Á Â Ã Ä G G Ǥ Ģ R ® R Ṛ Á Â Ã Ä
V Ⅴ Ṿ Ѵ
Å /\ a @
Å /\ a @
Ι ‫ ׀‬Ɩ ί ἰ ἱ
6gg ɡ 9 Ŗ Ṟ rrг
á à â ã
á à â ã
v \/
ŀ ḷ !¡
ṛ ɾ ŗ
ä å æ аa 9ǥϱ qq
ä å æ аa
ǃ ! ſ і ⅰ
ạ ḁ ą ล
ạ ḁ ą ล
i ị iί ļ
ɐ
ɐ
İ í
5
14
12
30
† http://cockeyed.com/lessons/viagra/viagra.html
5
PWUT do you mean?
1,300,925,111,156,286,160,896 ways to spell “viagra”†
V1AGRА
Ⅴ|ẠǤRA
V
ⅤïAĢṚᾼ
ѴÎΆgrÀ
ѴÍÀɡrÁ
vÏÁ9гÂ
vⅼÂ9ṛÃ
VIÃǥɾÄ
VⅠÄϱŗÅ
Ⅴ l Å q R /\
ṾΙ@GṚá
Ѵ‫׀‬áǤŖà
ѴƖàĢṞâ
vίâ6rã
vἰãgrä
Vἱägгå
Ⅴ!а9ŗa
I
A
1 l | ï
ì : Ì Î
Í Ï ⅼ IІ
A А ẠA
G
R
A
A А ẠA
ᾼḀ Ά À
ᾼḀ Ά À
lⅠ l ǀ Ӏ Á Â Ã Ä G G Ǥ Ģ R ® R Ṛ Á Â Ã Ä
V Ⅴ Ṿ Ѵ
Å /\ a @
Å /\ a @
Ι ‫ ׀‬Ɩ ί ἰ ἱ
6gg ɡ 9 Ŗ Ṟ rrг
á à â ã
á à â ã
v \/
ŀ ḷ !¡
ṛ ɾ ŗ
ä å æ аa 9ǥϱ qq
ä å æ аa
ǃ ! ſ і ⅰ
ạ ḁ ą ล
ạ ḁ ą ล
i ị iί ļ
ɐ
ɐ
İ í
30,240,000
5
40
30
14
12
30
Ѵ|ḁGгą
ѴïąGṛล
vÌɐĢŗA
VÎA6RА
ⅤÏẠgRA
ⅤⅼAɡṚᾼ
ѴlΆǥrÀ
ѴǀÀϱrÁ
vӀÁqгÂ
vΙÂqṛÃ
V‫׀‬ÃGɾÄ
VƖÄGŗÅ
Ⅴ ί Å Ǥ R /\
ѴḷágŖà
Ѵ!àɡṞâ
Vịäǥгå
ṾïaGRạ
ѴÌḁǤRą
† http://cockeyed.com/lessons/viagra/viagra.html
6
Experiment
Corpus:
1
7
Experiment
2
A
SCORE
B
SCORE
SCORE
C
8
Experiment
A
B
3
SCORE
Obfuscator
SCORE
SCORE
C
9
…
A top team of British scientists and
medical doctors have worked to
develop the state-of-the-art Pen1s
Enlargment Patch delivery system
which automatically increases penls
size up to 3-4 full inches.
…
…
Sim
List
A top teаm οf British sϲientists аnd
medicaǀ doсtorѕ һаve worked to
develοp tһe ѕtate-ofˍtһe-аrt Рenlѕ
EnΙarɡment Раtϲһ deӀiverу system
whiϲһ autοmaticalǀy inсreaѕeѕ pen1ѕ
ѕize up tο 3-4 fulӀ inсһeѕ.
…
Here's the link to check out!
Here's tһe ǀink to cһeck out!
http://www.all-love-pillzz.net/pt/?
46&wnwug
http://www.all-Ӏove-piІlzz.net/pt/?
46&wnwuɡ
The Obfuscator
9
…
…
A top team of British scientists and
medical doctors have worked to
develop the state-of-the-art Pen1s
Enlargment Patch delivery system
which automatically increases penls
size up to 3-4 full inches.
…
A top teаm οf British sϲientists аnd
medicaǀ doсtorѕ һаve worked to
develοp tһe ѕtate-ofˍtһe-аrt Рenlѕ
EnΙarɡment Раtϲһ deӀiverу system
whiϲһ autοmaticalǀy inсreaѕeѕ pen1ѕ
ѕize up tο 3-4 fulӀ inсһeѕ.
which
Sim
whiϲһ
List
…
Here's the link to check out!
Here's tһe ǀink to cһeck out!
http://www.all-love-pillzz.net/pt/?
46&wnwug
http://www.all-Ӏove-piІlzz.net/pt/?
46&wnwuɡ
The Obfuscator
10
0021 1:FF01:! 1:01C3:ǃ 1:0021:! 0.8706896:FF4C:l 0.8706896:006C:l 0.8706896:0406:І 0.8706896:0049:I
0022 1:0022:" 1:FF02:"
0023 1:0023:# 1:FF03:# 0024 1:0024:$ 1:FF04:$
0025 1:0025:% 1:FF05:% 0.815562:2105:℅
...
0027 1:FF07:' 1:0027:'
...
002A 1:002A:* 1:FF0A:* 0.9673913:2217:∗
...
0042 1:0392:Β 1:FF22:B 1:0042:B 1:0412:В
...
004F 1:041E:О 1:004F:O 1:039F:Ο 1:FF2F:O 0.9525223:1ECC:Ọ 0.9119318:0398:Θ 0.8885794:FF31:Q
0050 1:0050:P 1:FF30:P 1:0420:Р 1:03A1:Ρ
0051 1:0051:Q 1:FF31:Q 0.8885794:041E:О 0.8885794:0398:Θ 0.8885794:039F:Ο
0052 1:0052:R
1:FF32:R 0.9553072:1E5A:Ṛ 0.9369863:0156:Ŗ 0.912:1E5E:Ṟ
0053 1:FF33:S 1:0053:S 1:0405:Ѕ 0.9522388:1E62:Ṣ 0.9327486:015E:Ş
0054 1:0422:Т 1:0054:T 1:03A4:Τ 1:FF34:T 0.9139785:1E6C:Ṭ 0.9071038:01AC:Ƭ 0.880829:0162:Ţ
0055 1:FF35:U 1:0055:U 0.943662:1EE4:Ụ 0.8933333:1E72:Ṳ 0.8729641:1E76:Ṷ
0056 1:0056:V 1:FF36:V 1:2164:Ⅴ 0.9438596:1E7E:Ṿ 0.9191176:0474:Ѵ 0.9107807:01B2:Ʋ
...
Anthony Y. Fu, Xiaotie Deng, Liu Wenyin, Greg Little, “The methodology and
an application to fight against Unicode attacks”, July 2006, ACM Press
Unicode Sim-List
11
Experiment
A
B
4
SCORE
DeObfuscator
SCORE
SCORE
C
12
…
A top teаm οf British sϲientists аnd
medicaǀ doсtorѕ һаve worked to
develοp tһe ѕtate-ofˍtһe-аrt Рenlѕ
EnΙarɡment Раtϲһ deӀiverу system
whiϲһ autοmaticalǀy inсreaѕeѕ pen1ѕ
ѕize up tο 3-4 fulӀ inсһeѕ.
…
…
Sim
List
A top team of British scientists and
medical doctors have worked to
develop the state-of-the-art Penis
Enlargment Patch delivery system
which automatically increases penls
size up to 3-4 full inches.
…
Here's tһe ǀink to cһeck out!
Here's the link to check out!
http://www.all-Ӏove-piІlzz.net/pt/?
46&wnwuɡ
http://www.all-love-pillzz.net/pt/?
46&wnwug
The De-Obfuscator
13
Ultimate Goal
A
B
C
13
Ultimate Goal
A
B
C
13
Ultimate Goal
A
B
C
SCORES
13
Ultimate Goal
A
Obfuscator
B
C
SCORES
13
Ultimate Goal
A
Obfuscator
B
C
SCORES
A’
B’
C’
13
Ultimate Goal
A
Obfuscator
B
C
A’
SCORES
SCORES’
B’
C’
13
Ultimate Goal
A
Obfuscator
B
C
A’
SCORES
SCORES’
B’
C’
DeObfuscator
13
Ultimate Goal
A
Obfuscator
B
C
A’
SCORES
SCORES’
B’
C’
A’’
DeObfuscator
B’’
C’’
13
Ultimate Goal
A
Obfuscator
B
C
A’
SCORES
=
SCORES’
B’
SCORES’’
C’
A’’
DeObfuscator
B’’
C’’
13
Ultimate Goal
A
Obfuscator
B
C
=
SCORES
SCORES
SCORES’’
A’
SCORES’
B’
C’
A’’
DeObfuscator
B’’
C’’
13
Ultimate Goal
A
Obfuscator
B
C
=
SCORES
SCORES
SCORES’’
A’
>
SCORES’
B’
C’
A’’
DeObfuscator
B’’
C’’
14
Results
15
Results
#
1
2
3
4
5
6
7
8
9
10
Raw Score
16.2
21.7
17.2
10.6
7.9
21.7
19.2
12.1
13.1
10.6
Obfuscated*
3.15
5.6
4.38
1.9
5.85
5.0
5.4
2.55
5.94
5.35
De-Obfuscated*
16.2
21.7
17.2
10.6
7.9
21.7
19.2
12.1
13.1
10.6
*Averages over 10 trials
16
Phishing Twist
Selling what the customer SHOULD want
(not what they PROBABLY DON’T want)
Mimic legit messages closely
(MUST look nearly perfect)
17
“Perfect” PWUT
18
Future Work
Support other languages
Optimizations (faster data structures, efficient code)
Dictionary lookups/proximity heuristics
19
004F 1:041E:О 1:004F:O 1:039F:Ο 1:FF2F:O 0.9525223:1ECC:Ọ 0.9119318:0398:Θ 0.8885794:FF31:Q
0050 1:0050:P 1:FF30:P 1:0420:Р 1:03A1:Ρ
0051 1:0051:Q 1:FF31:Q 0.8885794:041E:О 0.8885794:0398:Θ 0.8885794:039F:Ο
0052 1:0052:R
1:FF32:R 0.9553072:1E5A:Ṛ 0.9369863:0156:Ŗ 0.912:1E5E:Ṟ
0053 1:FF33:S 1:0053:S 1:0405:Ѕ 0.9522388:1E62:Ṣ 0.9327486:015E:Ş
0054 1:0422:Т 1:0054:T 1:03A4:Τ 1:FF34:T 0.9139785:1E6C:Ṭ 0.9071038:01AC:Ƭ 0.880829:0162:Ţ
0055 1:FF35:U 1:0055:U 0.943662:1EE4:Ụ 0.8933333:1E72:Ṳ 0.8729641:1E76:Ṷ
0056 1:0056:V 1:FF36:V 1:2164:Ⅴ 0.9438596:1E7E:Ṿ 0.9191176:0474:Ѵ 0.9107807:01B2:Ʋ
0021 1:FF01:! 1:01C3:ǃ 1:0021:! 0.8706896:FF4C:l 0.8706896:006C:l 0.8706896:0406:І 0.8706896:0049:I
0022 1:0022:" 1:FF02:"
Fighting
Unicode-Obfuscated Spam
0023 1:0023:# 1:FF03:# 0024 1:0024:$ 1:FF04:$
0025 1:0025:% 1:FF05:% 0.815562:2105:℅
0026 1:FF06:& 1:0026:&
0027 1:FF07:' 1:0027:'
0028 1:0028:( 1:FF08:(
0029 1:FF09:) 1:0029:)
Changwei Liu
Sid Stamm
002A 1:002A:* 1:FF0A:* 0.9673913:2217:∗
<[email protected]>
<[email protected]>
Indiana University
002B 1:002B:+ 1:FF0B:+
003C 1:003C:< 1:FF1C:< 0.856383:22D6:⋖
003D 1:003D:= 1:FF1D:=
003E 1:FF1E:> 1:003E:> 0.856383:22D7:⋗
003F 1:FF1F:? 1:003F:?
0040 1:0040:@ 1:FF20:@
0042 1:0392:Β 1:FF22:B 1:0042:B 1:0412:В