AllExperts > Encyclopedia 
Search      
Find out about volunteering to AllExperts

Sender Policy Framework: Encyclopedia BETA


Free Encyclopedia
 Home · Index · Browse A-Z  · Questions and Answers ·
Encyclopedia

Browse A-Z
ABCDEFGHIJKLMNOPQRSTUVWXYZNum


License
Disclaimer

 
 
 
 
Free Online Courses
12 Weeks to Weight Loss
Take Charge of Stress
Learn How to Bake
Budgeting 101
Deeper Faith
DIY Fashion Makeover

       MORE E-COURSES
 
   

A B C D E F G H I J K L M N O P Q R S T U V W X Y Z  Misc

Sender Policy Framework

In computing, Sender Policy Framework (SPF) is an extension to the Simple Mail Transfer Protocol (SMTP). SPF allows software to identify and reject forged addresses in the SMTP MAIL FROM (Return-Path), a typical nuisance in e-mail spam. SPF is defined in RFC 4408.

Principles of Operation

Normal SMTP allows any computer to send e-mail claiming to be from anyone. Thus, it's easy for spammers to send e-mail from forged addresses. This makes itdifficult to trace back to where the spam truly comes from, and easy for spammers to appear to besenders the receiver would ordinarily trust. Manybelieve that the ability for anyone to forge Return-Path addresses is a security flaw in modernSMTP, caused by an undesirable side-effect of thedeprecation of source routes.

Further information: Sender Rewriting Scheme (SRS)

SPF allows the owner of an Internet domain to use special DNS records to specify which machines are authorized to transmit e-mail for that domain. For example, the owner of the example.org domain can designate which machines are authorized to send e-mail whosee-mail address in the Return-Path ends with "@example.org". Receivers checking SPF can then reject any e-mail that claims to come from that domain, but fails in a check against the IPslisted in the sender policy of this domain.

SPF protects the address in the Return-Path, thatis the address to which bounces would be sent ifthe mail is not delivered. While the address inthe Return-Path often matchesother originator addresses in the mail header like "From:" or "Sender:" this is not necessarily thecase, and SPF does not prevent forgeries of theseother addresses.

Spammers can send e-mail with an SPF PASS resultif they have an account in a domain with a senderpolicy, or abuse a compromised system in thisdomain. However, doing so makes the spam easier totrace and prosecute. An SPF PASS result from unknown strangers still guarantees that auto-replieslike error messages (bounces) cannot hit innocent bystanders.

The main benefit of SPF is to people whose e-mail addresses are forged in the Return-Paths. Theyreceive a large mass of unsolicited error messagesand other auto-replies, making it difficult to usee-mail normally. If such people use SPF tospecify their legitimate sending IPs with a FAILresult for all other IPs, then receivers checkingSPF can reject forgeries, already reducing theamount of back-scatter. More important spammersknowing their trade will avoid forging SPF FAILprotected addresses, because they want to reachas many of their primary victims as possible -there are more than enough unprotected addressesfor this abuse.

SPF has potential advantages beyond helping identify unwanted e-mail. In particular, if a sender provides SPF information, then receivers can use SPF PASS results in combination with a white list to identify known reliable senders.Scenarios like compromised systems and shared sending mailers limit this use, but at least auto-replies cannot hit innocent bystanders.

FAIL and forwarding

If a domain publishes an SPF FAIL policy,then legitimate mails sent to receivers forwardingtheir mail to third parties can be rejected andbounced if (1) the forwarder doesn't rewrite theReturn-Path unlike mailing lists, (2) the nexthop doesn't white list the forwarder, and (3) this hop checks SPF. This is an intentional and obvious feature of SPF - checks behind the "border" MTA(MX) of the receiver can't workdirectly.

Publishers of SPF FAIL policies must accept thispotential problem, they cannot expect that all receivers change their forwarding arrangements,i.e. clear at least one of the three criticalconditions.

A technique called Sender Rewriting Scheme(SRS) is a way for mail forwarding services to avoid this problem.

HELO tests

For an empty Return-Path as used in error messagesand other auto-replies SPF checks the HELO-identity. Actually it checks an artificial postmaster@mail.example.org address for aHELO or EHLO mail.example.org.

With a bogus HELO identity the result NONE won't help, but for valid host names SPF also protects the HELO identity. This SPF feature was always supported as option for receivers, and later SPF drafts including the final specification recommendto check the HELO always.

This allows to white list sending mailers based ona HELO PASS, or to reject all mails after a HELO FAIL. It can be also used in reputation systems (any white or black list is a simple case of a reputation system).

Implementation

Implementing SPF has two parts:
* Domains identify the machines authorized to send e-mail on their behalf. Domains do this by adding an additional record to their existing DNS information.
* Receivers can request and use SPF information. They use ordinary DNS queries, which are typically cached to enhance performance. Receivers then interpret the SPF information as specified and act upon the result.

Thus, the key issue in SPF is the specification for the new DNS information that domains set and receivers use. The records are laid out like this(in typical DNS-syntax):

example.org. IN TXT "v=spf1 a mx -all"

"v=" defines the version of SPF used, the following words provide mechanisms to use to determine if a domain is eligible to send mail. The "a" and "mx" specify the systems permitted to send messages for the given domain. The "-all"at the end specifies that, if the previous mechanisms did not match, the message shouldbe rejected.

Mechanisms

Eight mechanisms are defined:{| valign="top" | ALL
Matches always, used for a default result like -all for all IPs not matched by prior mechanisms.
AIf the domain name has an A (or AAAA for IPv6 ) record corresponding to the sender's address, it will match. (That is, the mail comes directly from the domain name.)
IP4If the sender is in a given IPv4 range, match.
IP6If the sender is in a given IPv6 range, match.
MXIf the domain name has an MX record resolving to the sender's address, it will match. (That is, the mail comes from one of the domain's mail servers)
PTRIf the sender reverse-resolves to a name ending in the domain, and that name resolves to the sender's IP address, match.
EXISTSIf the given domain resolves, match (no matter the address it resolves to). This is rarely used, along with the SPF macro language it offers more complex matches like DNSBL-queries.
INCLUDEIf the included (a misnomer) policy passes the test this mechanism matches. This is typically used to include policies of more than one ISP.

Qualifiers

Each mechanism can be combined with one of four qualifiers:
* + for a PASS result, this can be omitted, +mx is the same as mx.
* ? for a NEUTRAL result interpreted like NONE (no policy).
* ~ for SOFTFAIL, a debugging aid between NEUTRAL and FAIL.
* - for FAIL, the mail should be rejected (see below).

Modifiers

The modifiers allow for future extensions ofthe framework. So far only the two modifiersdefined in the RFC are widely deployed.

exp=some.example.com gives the name of adomain with a DNS TXT record,which is interpreted using SPF's macro language toget an explanation for FAIL results, typically anURL, added to the SMTPerror code. This baroque feature is rarely used.

redirect=some.example.com can be used instead of the ALL-mechanism to link to thepolicy record of another domain. This modifieris easier to understand than the somewhat similarINCLUDE-mechanism.

Error handling

As soon as SPF implementations detect syntax errors in a sender policy they must abortthe evaluation with result PERMERROR. Skippingerroneous mechanisms cannot work as expected,therefore include:bad.example andredirect=bad.example also cause aPERMERROR.

Another safety guard is the maximum of tenmechanisms querying DNS, i.e. any mechanismexcept from IP4, IP6, and ALL. Implementations can abort the evaluation with result SOFTERROR when ittakes too long or a DNS query times out, but theymust return PERMERROR if the policy directlyor indirectly needs more than ten queries for mechanisms, any redirect= also counts towards this processing limit.

A typical SPF HELO policy v=spf1 a -allneeds three DNS queries: (1) TXT, (2) SPF, and (3)A or AAAA. This last query counts as the first mechanism towards the limit (10), in thisexample it's also the last, because ALL needs noDNS lookup.

Caveats

SPF normally only validates the domain of the envelope sender (in the Return-Path). Domains that share mail senders (e.g. with virtual hosting) can forge each others' domain. SPF does not validate that a given e-mail actually comes from the claimed user, because it operates at the network level.

SPF FAIL rejection

SPF FAIL policies can be an effective but dangerous tool.Some publishers of SPF policies try to avoid the dangersby using SOFTFAIL (designed for limited testing periods)instead of FAIL.But SOFTFAIL can be even more dangerous than FAIL withreceivers rejecting FAIL and accepting SOFTFAIL taggedas potential junk.

A reject in a forwarding scenario is a clean decision,the forwarder will send an error message to the addressin the Return-Path. Typically the error message (bounce)contains an explanation of the SMTP error and the failing(forwarded to) address. The original sender can thensend his mail again directly to this address bypassingthe forwarder, a crude emulation of the normal SMTPerror code 551 user not local.

However an accepted SOFTFAIL tagged as potential junkcould be deleted by the final recipient. This is a userwho has arranged his forwarding in a way that cannotwork with SPF, this user could be also careless withchecking his potential junk and simply delete it.

The same line of arguments also suggests that receiversshould take the SPF recommendation to reject SPF FAILseriously where possible. Accepting SPF FAIL resultsas potential junk can be more dangerous than simplyrejecting FAILed mails.While senders with an SPF FAIL can be expected to knowwhat it means, the same is obviously not the case for areceiver arrranging his forwarding in a way that cannotwork with SPF.

Checkpoints

Checking SPF behind the "border"MTA(MX) is not impossible,the relevant info is usually noted in a Receivedtimestamp line added by one of the MXs of the receiver.But at this time it's too late to reject SPF FAIL, thechecking entity can essentially only delete FAILing mail.

Experts should be able to get this right, but it's noplug and play situation, therefore the SPF specificationrecommends to check SPF only at the "border" (MX) in theSMTP session, not later.Later SPF checks can also have unexpected results if the publishers of sender policies don't plan modifications of their policy carefully with regard to DNS cache expiration.

DoS attack

A new draft SPF DoS Exploitationdiscusses concerns related to the scale of an SPF answer leading to network exploits as a meansto corrupt DNS. This issue is also covered in the security considerations of the SPF RFC.

History

The SPF concept was presented at YAPC and OSCON (O'Reilly Open Source Convention) in 2003, in a short paper titled "Repudiating Mail-From" written by Paul Vixie in 2002. Other predecessors were"Reverse MX" by Hadmut Danisch, and "Designated Mailer Protocol" by Gordon Fecyk.

In June 2003, Meng Weng Wong merged the RMX and DMP specifications and solicited suggestionsfrom other programmers. Over the next six months,a large number of changes were made and a large community had started working on SPF.

Originally SPF stood for Sender Permitted Fromand was sometimes also called SMTP+SPF, but it waschanged to Sender Policy Framework in February 2004.

In early 2004, the IETF created the MARID working group and tried to use SPF and Microsoft'sCallerID proposal as the basis for what is now known as Sender ID.

After the collapse of MARID the SPF communityreturned to the original "classic" version of SPF.In July 2005 this version of the specification wasapproved by the IESG as anIETF experiment. On April 28th, 2006, the SPF RFC was published as RFC 4408.

Controversy

Steven M. Bellovin has written a [news://news.gmane.org./6.0.1.1.2.20040105081256.03788ec0@ux13.sp.cs.cmu.edu long e-mail dated January 5, 2004], discussing some of hisconcerns with SPF. Some of these include:
* SPF uses TXT records in DNS, which are supposed to be free-form text with no semantics attached. SPF proponents readily acknowledge that it would be better to have records specifically designated for SPF, but this choice was made to enable rapid implementation of SPF. In July 2005, IANA assigned the Resource Record type 99 to SPF. SPF publishers may publish both record types and SPF checkers may check for either types. It will likely take many years before all DNS software fully supports this new record.
* As of the time he wrote his message, there was no consensus that this is the right way to go. Some major e-mail service providers have not bought into this scheme. Unless and until they do, it doesn't help much, either for their customers (who make up a substantial proportion of the user population) or for everyone else (since their addresses could be forged). It's worth noting that since this concern was raised, among others AOL has embraced SPF.
* Bellovin's strongest concerns involve the underlying assumptions of SPF (SPF's "semantic model"). When using SPF, the SPF DNS records determine how a sender is allowed to send. That means that the owner of the domain will control how senders are allowed to send. People who use "portable" e-mail addresses (such as e-mail addresses created by professional organizations) will be required to use the domain owner's SMTP sender, which may not currently even exist. Organizations providing these "portable" addresses could, however, create their own Mail Submission Agents (MSAs) (RFC 4409) or offer VPNs. Besides SPF only ties the SMTP Return-Path to permitted MSAs, users are still free to use their RFC 2822 addresses elsewhere.

Jonathan de Boyne Pollard has written a separate diatribe against SPF that discusses, among other things, SPF's conflict with several major mail RFCs, its ability to force ISPs to force their customers to use mail in particular ways, and the fact that it breaks when used with secondary MX hosts.

The Register has written a brief article mentioning that more spam uses SPF than legitimate mail.

Brad Knowles has a general article on problems with SPF.

John Levine has a short general article on problems with SPF as well.

Deployment

In spite of its limitations, many people have decided that the pros of SPF outweigh its cons and have begun implementing SPF. Anti-spam products such as SpamAssassin version 3.0.0 implement SPF. Many mail transfer agents (MTAs) support SPF directly such as Wildcat, or have patches/plug-ins available that support SPF, including Postfix, Sendmail, Exim, and Qmail. Many prominent domains decided to post SPF data for their domains as of mid-2004, includingAmazon, AOL, EBay, Google, GMX, Hotmail, and W3C.

See also

* E-mail authentication overview
* MARID
* Sender ID
* Sender Rewriting Scheme
* Sender Signing Policy

External links

* SPF homepage and news
* Mailing list archives
* SPF Testing site
* emailbattles: SPF Claws Sender-ID (Aug 2005)
* Canadian recommendation for ISPs (May 2005)
* Interview with co-author W. Schlitt (Mar 2005)
* David Woodhouse discusses flaws in SPF (Jan 2005)
* SpamCop FAQ entry about bogus bounces discusses also SPF (2005)
* M. Wong's Deployment White Paper (PDF, Nov 2004)
* CircleID interview with co-author M. Wong (Jun 2004)
* Jonathan de Boyne Pollard's list of the flaws in SPF (2004)
* Diagram of e-mail flow and SPF's impact (PDF, PNG )
* MIT Spam Conference Handout on SPF (Jan 2004)
* Steve Bellovin expresses doubts (Jan 2004)



Email this page
About Us | Advertise on This Site | User Agreement | Privacy Policy | Kids' Privacy Policy | Help
About and About.com are registered trademarks of About, Inc. The About logo is a trademark of About, Inc. All rights reserved.
This is the "GNU Free Documentation License" reference article from the English Wikipedia. All text is available under the terms of the GNU Free Documentation License. See also our Disclaimer.