As mentioned on the smtp checks page, FastMail.FM performs many checks at the time an email is received to stop spam. While these tests are very effective, they can't stop all spam, and some still enters the FastMail.FM system.
After FastMail.FM receives the email, we perform a number of checks based on the content of the email to try and determine if the email is spam. These checks are controlled by the settings on the Options -> Spam/Virus protection screen.
Unsolicited Commercial Email ("spam", or UCE) can be a major problem making it difficult to sort out your important emails from junk mail. However, spam can often be identified by looking for telltale signs such as mention of pornography, formatting used by certain bulk-email software, and the presence of known spam servers in the headers. By enabling the spam protection option, FastMail.FM automatically searches for these signs and performs various actions depending how 'spammy' the message is.
The following levels are available:
-
Basic - all users have this basic level of spam protection. If a message comes from a host or network known to be a common source of spam, then the message is rejected.
-
Normal - most users will want this level of protection. It contains a sensible combination of actions that will file probable spam into a folder called 'Junk Mail'. Messages that are just possibly spam are marked with a 'Spam Score' in their subject. A higher score means that the message is more 'spammy'. If email comes from people in your address book, then it will be less likely to be recognised as spam. Probable backscatter is also filed into the 'Junk Mail' folder.
Unfortunately, the spam checking takes a significant amount of CPU time to run, so we can only provide this for users who pay a yearly subscription.
-
Aggressive - a more aggressive level of filtering. Spam checking is a bit more strict than the 'Normal' setting, and mails that have a very high probabitlity of spam or are backscatter are discarded. Useful if you receive a lot of spam in your account.
-
Whitelist Only - an extremely aggressive level of protection. With this level of spam protection enabled, only email from people who are in your address book will reach you. Everything else will be discarded. Use with caution.
-
Custom - whatever you like! Custom spam protection consists of the same elements as Normal, but you are able to configure the behaviour of the individual elements. This includes the spam score thresholds that are used for deleting / filing / marking messages, the folder that messages are to be filed to, the way in which messages are marked, and the special treatment of messages from people in your address book.
When spam protection is enabled, FastMail.FM automatically searches each message for characteristics that are typical of spam and assigns a 'spam score' using the SpamAssassin engine. Research by SpamAssassin shows that 99.5% of messages that score over 5.0 are spam.
The assigned score is placed in the header of incoming emails. FastMail.FM may take some actions based on the value of the spam score, depending on how you have configured the spam protection.
Advanced users can also define additional actions (reject, file, or forward) in the Define Rules screen, by selecting 'Spam Score >=' as the field to Look In, and entering the spam score theshold into the 'For Text Matching' field.
The main types of checks done by SpamAssassin are:
-
Bayes database: compares tokens found in the message with a local database, and computes a probability that this message is spam. The database is updated using high-scoring and low-scoring messages as examples. This method will adapt to evolutions in the structure of spam, but not revolutions.
-
Razor: Creates a checksum of the message and compares it to recently reported spam in a global database; if they match, it's spam!
-
DNSBL: Check all servers in the SMTP path for open relays, open proxies, and blacklisted hosts in many blacklists, each one weighted differently depending on past results
-
Header and body checks: Various checks for common spam phrases like "This is not spam!" and "This is sent in compliance with bill S1618"
-
Structure: Checks whether it's HTML mail, has an embedded form, contains formatting errors, etc
Discard mail without a trace
If you select the discard mail checkbox, then messages that have a spam score greater than or equal to the threshold will be silently discarded, via the sieve "discard" command. You won't see them in any of your folders, or in the trash, and the sender will not receive a bounce or reject message.
It is usually wise to set this threshold larger than the spam score on any real message (i.e. not spam) that you have ever received!
File probable spam to the given folder
If you select the file mail checkbox, then messages that have a spam score greater than or equal to the threshold will be placed in the indicated folder. You can choose whatever folder you like, but it is easier to empty if you file to "Junk Mail".
Change the subject of suspected spam
If you select the change subject checkbox, then messages that have a spam score greater than or equal to the threshold will have their subject modified according to the value you select in the drop down list.
The possible values are:
-
None - the subject is unchanged, but the X-Spam header will be set
-
{SPAM ###} - at the start of the subject the word SPAM will be added, and also a simple graph representing how spammy the message is. The graph ranges from _____ (hardly spammy at all) to ##### (very spammy).
-
{SPAM 99.9} - at the start of the subject the word SPAM will be added, and also the spam score for this message. The score will always have two digits to the left of the decimal point, so if you are using this form then you can sort a folder by subject, and the messages will be in order of spam score.
-
{SPAM 99.9 ###} - at the start of the subject the word SPAM, the spam score for this message, and a graph (as described above) will be added.
-
Custom - you can specify what text you want to be added before and after the original subject, and you can specify the format of the spam graph.
In custom mode, you'll see a new 'Trusted hosts' field.
This is useful if you have email forwarded to your fastmail account via another provider you trust. Basically SpamAssassin will, in quite a few cases, only look at the network "edge" where the email came from to our system because you can't trust headers beyond that. In the case of forwarding services, that means the forwarding service itself, which are legtimate trusted servers (eg: gmail.com, hotmail.com etc..), are checked against RBLs. Thats not really useful and you might end up getting a lot of spam forwarded from them.
By specifying the forwarding services as trusted hosts, we can scan back through them to find out the real source. Internally we have list of trusted hosts (google.com, hotmail.com, runbox.com etc..).
However, one may also add to this list by specifying the domain names in the "Trusted Hosts" field. For instance one may have an email address at their old university and forward all the email that arrives there to fastmail, in which case he/she would want to add the domain of the university to the trusted hosts list. Doing this will allow us to parse back through those forwarding headers to find the true source IP of the message.
Note that being a "trusted" system doesn't mean we don't spam check it, it just means that we parse back through the Received headers to find what server delivered the email to that service, rather than using that services IP.
The FastMail.FM system adds the following headers to spam checked email.
- X-Spam-score - aggregate spam score. A number with 1 decimal place. At "Normal" spam protection level, scores < 5.0 are considered not-spam, scores > 5.0 are considered spam.
-
X-Spam-hits - shows which SpamAssassin rules were triggered by an email and the score of each rule that hit.
So a header like this:
-
X-Spam-hits: BAYES_99 3.5, EXTRA_MPART_TYPE 1.091, HTML_MESSAGE 0.001, SPAMMY_XMAILER 1 X-Spam-score: 5.5
Shows that BAYES_99 had a score of 3.5, EXTRA_MPART_TYPE a score of 1.091, etc. Adding these all up gives the final score of 5.5 (always rounded to 1 decimal place). Note that originally this header didn't have the score of each hit, it just had a list of hit names.
-
- X-Spam-source - information parsed from the Received: headers that shows the calculated source of the message. This may not be the server FastMail.FM received the email from if you have trusted hosts setup.
- X-Spam-charsets - character sets found in the message either in message headers, or in the content-type headers for each section of the message
- X-Spam - legacy header present and set to 'spam' if a message has a score greater than the threshold score, or 'high' if a message has a score twice the threshold score. Do not use.
Note that not all headers may be added on each message. No X-Spam headers are added if the message is not spam checked because:
- The user has only 'Basic' spam protection enabled
- It's great than 1.2M in size (may change in the future)
- A message was detected as containing a virus