acma news
Dissecting spam—
The Spam Intelligence Database
The ACMA recently launched a new tool designed to bring
a transformational approach to storing and analysing spam.
The spam intelligence database (SID) works to assist the
ACMA’s anti-spam investigators and compliance officers
enforce the requirements of the Spam Act 2003.
Designed and written entirely in-house by
ACMA staff, SID is a high-performance
system capable of processing hundreds
of thousands of spam email messages
per day. SID currently accepts spam
reports submitted by users of Telstra
BigPond webmail accounts, users of
the SpamMATTERS spam reporting
button, the ACMA’s own spam traps,
and third party spam traps. The
diversity of these sources enables a
more sophisticated analysis of spam,
as spam takes many forms and is
constantly evolving. SID’s flexible system
architecture also provides for scalability
and flexible future development.
The diagram below provides a technical
overview of SID’s processing steps. Once
a spam email message is received, it is
stored, indexed and analysed by SID’s
‘post processors’. These processing
operations yield much more information
than was originally visible in the message,
including an analysis of URL links
embedded within the message, message
attachments, identification of the spam’s
origin and malicious components.
The information indexed and collated
though SID’s processes is substantial
and enables the ACMA’s anti-spam
team to identify commonalities between
otherwise distinct spam messages,
which can lead to increased visibility of
a spammer’s activities. This additional
information also enables identification
of the varying methods used to send
spam. For example, a spam campaign
utilising hundreds of different originating
IP addresses for a set of identical reports
is indicative of a botnet spam campaign
(spam sent from an aggregate of infected
or ‘compromised’ computers), rather
than spam originating from a business or
other source, which will typically originate
from the same IP address.
The ability to associate spam with
botnets is of particular interest to the
ACMA. The Australian Internet Security
Initiative (AISI) is an ACMA program
that provides participating Australian
ISPs with daily reports identifying
compromised IP addresses on their
networks—that is, customer computers
that have been infected with botnetrelated malware. Part of SID’s ongoing
development is to link SID and the
AISI so that each is a source of useful
intelligence to the other. For example,
a future SID system development will
enable spam messages that have
been sent from a ‘bot’ to be analysed,
the ‘sending’ bot-related IP address
extracted, and this data provided to
the AISI. Conversely, spam complaints
received separately by the ACMA can
already be correlated with spam reports
contained in SID to enable a more
comprehensive analysis of a spammer’s
activities.
Designed and written entirely in-house by ACMA staff,
SID is a high-performance system capable of processing
hundreds of thousands of spam email messages per day.
SID is referred to as a spam intelligence
database, rather than just a spam
database, because it generates such
significant and actionable output on
an ever-growing body of spam.
For more information on spam and how
the ACMA is combating it, visit the ACMA
website at www.spam.acma.gov.au.
1st stage
2nd stage
Pre processors
> Standardise
message formats
> Record when
message was
received
Post processors
Message archive
Message
Message
3rd stage
Main processors
> Run post processors
to create additional
information
> Extract all data contained
with email such as email
addresses, URLs,
message headers
Message queue
PP queues
Quarantine folder
SID
Message
Issue #53 June 2010
13