PROCMAIL QUICK START
An introduction to email filtering with a focus on procmail

What is Procmail and What You Need to Use this Quick Start

Procmail is free/libre open-source software that is both a mail processor and a mail delivery agent (MDA). It can be used by either a system administrator or a user to automatically process and deliver incoming mail messages. It can also be used to re-process and re-deliver messages that are already in a mailbox.

This Procmail tutorial is aimed at regular users, not system administrators. In order to use these instructions, you need:

1. A basic understanding of Internet mail flow, which you can learn about at:
   • bath.ac.uk: Anatomy of an Email Message
   • Google Help > Gmail Help > Privacy & Security > Learn to read full email headers
   • How UW Email Works -- this article, including its nice diagrams, is relevant to most modern Internet email systems (BTW, if you have ever taken a class at the University of Washington, you can get a complimentary or premium MyUW.net account)
   • Reverse Spam Filtering: Winning Without Fighting, especially in the Email Deflexion Flowchart
   • MUAs, MTAs, and MDAs
   • How Unix Email Works
   • E-Mail Server Administration 101 by Dee Ann LeBlanc and Robert LeBlanc, which describes mail flowing through MUA->MSA->MTA->...->MTA->(MFA)->MDA->MUA
   • Procmail FAQ: Appendix B -- Figuring Out the Mail Flow
   • SPF Mail Flows diagram, which is also available on a T-shirt
   • Internet mail infrastructure by D. J. Bernstein
   • The Internet Protocol Journal: Volume 8, Number 4: Challenges in Anti-Spam Efforts by Dave Crocker
   • Archive.org: Cyrusoft.com: Sieve: Documents: The Sieve Language and a General Model for Delivery and Interoperable Filtering in Internet Mail by Matthew Wall (PDF or HTML)
   • The message Re: Usability. Spamassassin in the SpamAssassin list -- this includes a quiz that anyone who edits SpamAssassin or Procmail configuration files should be able to pass

In addition to understanding Internet mail flow, your mail messages must be delivered to a system that:

2. has procmail installed
3. you have shell access to
4. either
   1. uses procmail as the default local delivery agent (LDA), or
   2. uses a message transfer agent (MTA) that lets you use a .forward file to set procmail as your LDA, or
   3. allows you to set procmail as your LDA some other way. For example, with qmail, use dot-qmail files; with MMDF, use .maildelivery; with getmail, use .getmailrc; and with fetchmail, use .fetchmailrc.

If your system satisfies 4a or 4c, make sure that you skip Steps 8 & 9 below, that is, do not set up a .forward file.

	If you are looking for a provider that has procmail installed, see my list of Free or Reasonably-Priced IMAP Service Providers — most of the providers who give full Unix shell access also let their users write and manage their own procmail recipes.

Terms Used in This Article

Note about the ordering of these terms:  The terms listed earlier are used in the definitions of later terms.

PM  or  pm

In this article I use PM as an abbreviation for Procmail. For example, I use pmlog for the Procmail log file, PMRE for Procmail regular expression (defined below), and PMDIR for the variable that points to the directory that holds your Procmail-related files. Note that there are others who use this abbreviation, for example, Paul Chvostek's Procmail Log Watch awk script is called pmlw and Jari Aalto's Procmail Documentation and Procmail Library are located at pm-doc.sf.net and pm-lib.sf.net.

mail client  or  mail user agent  or  MUA

A program — such as Mulberry, pine, alpine, mutt, Opera Mail (M2), Thunderbird, SeaMonkey Suite, Outlook Express, Outlook, Mail.app, or Eudora — that a user uses to process mail messages. "Processing" includes reading, extracting or launching attachments, moving, printing, deleting, and composing & initiating the sending of messages. For more information, see the definition of MUA at FOLDOC and Email client at . For discussion about offline-, online-, and disconnected-mode IMAP clients, see this 2002-June-01 comp.mail.imap message. For a comparison of Pine, Mutt, Mulberry, SeaMonkey Suite and Thunderbird, see Comparison of Pine and Its Main Competitors on the All About Pine page.

news client  or  NNTP client

A program — such as pine, alpine, nn, slrn, Opera Mail (M2), SeaMonkey Suite, or Microsoft Outlook Express — that a user uses to process Usenet and other NNTP messages. "Processing" includes reading, extracting or launching attachments, moving, printing, canceling, and composing & initiating the sending of messages. Note that many NNTP clients can save to, read from, and process the mailboxes that Procmail delivers messages to. For more information, see the definition of news reader at FOLDOC.

server

A computer or a computer program that responds to requests from — or serves — other computers or computer programs. Whether the word “server” means “server software” or “server computer” is usually clear from the context and sometimes people explicitly say which is meant. For example, the following questions are equivalent and both are in standard parlance.

• “What IMAP server software are you running?”
• “What IMAP server are you running?”
  
  

For more information, see the definition of server and client-server at FOLDOC.

MTA  or  message transfer agent  or  mail transport agent  or  mailer

The underlying program that a mail server uses to send and receive mail messages. There are many MTAs in use today, for example CommuniGate Pro, Courier, Exchange, exim, MMDF, postfix, Post.office, qmail, sendmail, smail, and Zmailer. Note that on some systems the sending MTA is different from the receiving MTA. For more information, see the definition of MTA at FOLDOC, Cameron Laird's personal notes on message transfer agents, Unix Mail Transport Systems reviewed by JdeBP, and the definition of Message Transfer Agent at .

MSA  or  mail submission agent

To “send” a message, a mail user agent traditionally submitted the message to a message transfer agent. Nowadays most systems have a layer that sits between the MUA and MTA called the mail submission agent or MSA, and the MUA submits the message to the MSA rather than to the MTA. For details, see RFC2476, Message Submission and Gregory Neil Shapiro's Message Submission Agent (MSA). An MSA is used to authenticate the sender and, optionally, to rewrite headers, catch outgoing spam, viruses, & other vermin, and to provide an alternate outgoing port number (because, e.g., port 25 is often blocked or hijacked by Internet Access Providers).

LDA  or  local delivery agent  or  MDA  or  mail delivery agent  or  DA  or  delivery agent  or  local mailer

The program your server uses to deliver mail messages to the mailboxes of users on the system. Sometimes delivery is built into the message transfer agent (for example, sendmail's mail.local and Postfix's local) and sometimes another program, such as tmail, dmail, Dovecot deliver, maildrop, or procmail, is used as the delivery agent.

mailbox  or  mail folder  or  folder

A collection of messages stored on either a local or a remote computer. A mailbox might be a single flat file (this is the structure used by spool (mbox) and c-client MBX mailboxes); it might be a directory with each message a file in the directory (this is the structure of MH, maildir, and Netnews mailboxes); or it might use some other data structure, for example the structure used by c-client mix mailboxes. For more information, see "Appendix A: Folder Formats" in the Procmail FAQ and Before You Begin: Note About MH, maildir, and Traditional Unix Spool (mbox) Mailbox Formats below.

default inbox  or  system mailbox

Whatever you or your system administrator has specified as your primary incoming mailbox. Messages that fall through all your procmail recipes are delivered to this mailbox. To find out your system mailbox, type  procmail -v  at your Unix prompt. To tell procmail to override your default inbox, set the DEFAULT environment variable in your procmailrc. Warning: Overriding $DEFAULT may have undesired consequences with some MTAs.

MAILDIR variable  and  maildir format  (which are two completely different beasts!)

Unfortunately, the word “maildir” is used in (at least) two different ways in the Procmail world. MAILDIR (in all caps) is a Procmail variable that changes Procmail's current working directory and is used to point to the directory where your personal (non-system-spool) mailboxes are stored. For more information about Procmail's MAILDIR variable, see the blinking IMPORTANT note in Step 4 below. Maildir (not in all caps) is the name of a mailbox format. For more information about the maildir mailbox format, see Using maildir format, the maildir man page, and the section below about delivering to a mailbox that is in maildir format.

 

list  or  subscription  or  solicited bulk email  or  SBE or  solicited bulk mail  or  SBM

A mailing list that is used either for group discussion or for newsletters & announcements. For more information, see the definition of mailing list at FOLDOC and Webopedia; also see the definition of UBE (Unsolicited Bulk Email), which is an antonym of these terms.

 

recipe

A set of conditions and an action to be performed if the conditions are satisfied. Note that some programs that filter messages use a different term for this concept. For example, Pine uses the term filtering rule, Visual Office use the term rule, SeaMonkey Suite and Thunderbird use the term message filter, SmartSieve uses the term Mail Filter Rule, and Mulberry uses the terms script, trigger, & rule to encompass this concept.

 

string

A sequence of characters. Letters, digits, punctuation, symbols, tabs, and spaces are all considered characters. A substring is a string that is contained in another string. For more information, see the definition of string at FOLDOC.

 

regular expression  or  RE  or  RegEx  or  RegExp

A string in which some characters are interpreted as “wild cards” or metacharacters. Here are some metacharacters and metaphrases that can be used in a Procmail regular expression (PMRE).

Metacharacter or Metaphrase	Meaning in a Procmail Regular Expression
.	any character other than newline (i.e., other than Line Feed or LF or ASCII character 10)
(string)	treat string as a single item
*	zero or more of preceding item
?	zero or one of preceding item
+	one or more of preceding item
^	newline; usually used to match beginning of line; Note: As discussed in Re: procmail help! $MATCH grabs newline too, the meaning of the ^ metacharacter is different in Procmail than in most other RE tools.
$	newline; usually used to match end of line; Note: As discussed in Re: procmail help! $MATCH grabs newline too, the meaning of the $ metacharacter is different in Procmail than in most other RE tools.
[characterList]	any single character in characterList
[^characterList]	any single character NOT in characterList
\< \>	each of \< and \> is a metaphrase for a "non-word character"; details are in the definition of word & Matching a Word below, and in Jari Aalto's Procmail and egrep differences
|	OR; ORing in Procmail is discussed below
()	() is the null item and is always a match; it is used to make a recipe condition more readable, to escape a leading backslash (\), or to escape the special meaning of other metacharacters or metaphrases. For examples, see this discussion of ()< and this discussion of ()\/. Note that it is more common to escape a metacharacter with a backslash, which is discussed in the next item.
\metacharacter	Escape the special meaning of metacharacter and treat it as the literal character that it is;  for example  \.  means the dot character rather than “backslash followed by any character” and  \*  means the asterisk character rather than “zero or more backslashes.”  Note: The less-than character (<) needs to be escaped differently. Also see David W. Tamkin's 2004-April-24 message about leading backslash problem (was persistent lock files).  Tip: Another way to escape a metacharacter is to put it in a [characterList], e.g., you can use either \. or [.] to mean a literal dot (.) in a PMRE.  Example: In Setting Keywords or Labels below, there is a recipe condition that contains a literal $ character.
\/	extraction operator (discussed below)
\newline	If the last two characters on a line are a backslash (\) followed by newline, this is a “continuation backslash” and it tells Procmail to continue the current line with the text on the next line and ignore these characters: backslash (\), newline, white space (spaces, tabs, and/or newlines).

In Procmail, regular expressions are used in the conditions of a recipe to, e.g., look for a pattern in the Subject header of a message. For more information, see the definition of regular expression at FOLDOC, regular-expressions.info, Regular Expressions HowTo at OmniWeb, Columbia University's page on regular expressions for procmail, the Regular Expression section of era eriksson's Procmail Quick Reference, Scott Wiersdorf's Regular Expressions - Procmail Mail Filtering Syntax, manual page for regexp, the Extended regular expressions section of the procmailrc manual page, and .

	Procmail “normal egrep” regular expressions do not support all the metacharacters that are used in GNU egrep. For example, PMREs are not PCREs (perl compatible regular expressions) and PMREs do not support the limiting repetition operator {min,max}. Make sure you read the regular expression documentation that is specific to Procmail. The regular expression .* means any sequence of any characters, including the empty sequence. It is not needed at the beginning or end of a Procmail regular expression (except when you are using the extraction operator and you want to extract all characters either from the beginning of the line or through to the end of the line). A Procmail regular expression sometimes matches greedily, which means that it will match the longest string possible, and sometimes matches stingily, which means that it will match the shortest string possible. For details, see Understanding procmail's minimal matching (stingy vs. greedy) in Jari Aalto's PM Tips.

 

 

word

A substring that contains only word characters - letters, digits, and underscores - and is bounded on the left by the beginning of a line or non-word character and on the right by the end of a line or non-word character. For example, in the line

           Subject: testing procmail

the substrings Subject, testing, and procmail are each words and the substrings Subject:, mail, and test are not words. The diagram below shows the relationship between words and substrings, namely: Every word is a substring but not every substring is a word.

           +----------------+
           |  +---------+   |
           |  |  words  |   |
           |  +---------+   |
           |     substrings |
           +----------------+

Since you need to look at the boundaries of a string to determine whether it is a word, the term word only makes sense when you are talking about a substring in the context of a larger string. For more information, see the discussion about matching the word “test” in Understanding and Refining Your Procmail Recipes below.

 

pico

A friendly text editor that is the default pine composer. Throughout this article you can replace pico with a different text editor if you like. To learn more about Pico, see the pico man page and What is Pico at the University of Washington Pine Information Center.

 

~  or  $HOME  or  ${HOME} 

Your home directory. You can get home by typing any of the following:

cd
cd ~
cd $HOME
cd ${HOME}

Note: Do not use ~ in a procmailrc file; use $HOME or ${HOME} instead.
Remember:  There's no place like ~ 

 

See Also

• IMAP Service Providers: Terminology, which discusses terminology that is especially relevant when comparing mail service providers including account, user, personal mailstore, alias, subaddress, catchall address, and domain hosting.
• Power Pine: Terminology, which discusses the IMAP terms RECENT, SEEN (aka READ), and UNSEEN (aka UNREAD).
• Stopping Spam Creating a Stronger, Safer Internet: Glossary (according to this, John R. Levine wrote this Glossary).

Strategies

With procmail you can organize your environment variables, recipes, mailboxes, and directories however you like. Only the .procmailrc file, which I discuss below in Step by Step Through Setting Up and Testing Procmail, is required to have this name and to reside in your home directory. This freedom is part of what gives procmail its power but it can lead to chaos if you do not have a strategy for organizing and naming your procmail-related environment variables, files, and directories.

Simplicity

The first strategy that I recommend and try to follow in this article is simplicity. For example, I recommend that you use the Procmail defaults whenever possible and do not set unnecessary environment variables in your procmailrc file(s). For more about this, including problems caused by not using the defaults, see the Warnings () notes in Step 4 below.

Modularize and Plug & Play

A key to good programming is modularization. This, along with simplicity, will help to make your Procmail configuration portable and will make it easy to plug in, unplug, or reorder recipes. Here are examples of recipe modules you might want to use:

• rc.testing
• rc.subscriptions (aka rc.sbe or rc.bulk or rc.blue)
• rc.log
• rc.forward
• rc.rmdupes
• rc.munge-windows-1252
• rc.munge.newsgroups, which I discuss on my main Pine page in this section
• rc.autoreply
• rc.vacation
• rc.green — used, along with a greenlist (AKA “whitelist” or “accept list” or “allow list” or “goodguy list” or “trusted senders”), to do Reverse Spam Filtering
• rc.greenlister — uses an incoming message to trigger an update to the greenlist; for details see Reverse Spam Filtering: Winning Without Fighting
• rc.quarantine (aka rc.virus or rc.violet) — see Snagging Viruses below
• rc.spamassassin — see Using SpamAssassin below
• Sean B. Straw's listname_id.rc, which I discuss in Generic SBE Sorting below
• Ruud H.G. van Tol's procmail .inc (include) files, which are here & here, and the .rc and other files in his procmail directory
• Dallman Ross's vsnag.genvars.rc, which is part of Dallman's Vsnag package and about which Dallman said “the genvars contents are meant for, and I'm fine with their use for, satisfying your general private procmail needs. Ditto public, if that means a server running procmail for the good of its users, including commercially, so long as I don't lose credit for what I've offered.”
• Bart Schaefer's rfc822rc.txt, which can be used to parse RFC822 addresses; an example of using this is in this 2004-Nov-26 message by Ruud H.G. van Tol

In the Step-By-Step section below, I walk you through creating the first two recipe modules and then, after testing, I show you how to unplug rc.testing. I discuss rc.quarantine and rc.spamassassin in the Advanced Recipes section.

Use Consistent Naming and Formatting Styles

Using consistent naming and formatting styles will help you keep your directories, files, and mailboxes organized. Here are the styles that I use:

*	Notes About Mailbox Names
	Here are more details about my strategy for choosing mailbox names. These notes are in addition to the comments in the starred item (*) above. I use dash (-) rather than underscore (_) or a space ( ) as a visual separator in my mailbox names. I don't like underscores because they are a pain to type (you need to hold down the SHIFT key) and they are sometimes hard to see (they can fade into the baseline of the text and looks like a space). I don't like spaces in mailbox names because in many tools, especially command-line tools, spaces need to be escaped. Another reason to avoid the underscore (_) character in a mailbox name is discussed in the 2007-August thread with subject "-" and "_" in folder names in the dbmail mailing list. The string “INBOX” has a special meaning to IMAP servers and if you give a mailbox this name, your IMAP client might not be able to see this mailbox on the IMAP server. This is because when it sends the string “INBOX” to the server, the server interprets it to mean a special mailbox, which is not necessarily the mailbox that has the name “INBOX”. Each of the strings mbox, mail.txt, INBOX, and INBOX.MTX have a special meaning to clients and servers that are based on the University of Washington's c-client tools. It is best to not use any of these for a mailbox name, unless, of course, you want a mailbox to be interpreted in the special c-client way. The string “default” has a special meaning to the qmail MTA and it is safest not to use this as a mailbox name. The strings new, cur, and tmp are used in the internal structure of maildir-formatted mailboxes and, to avoid confusion, I recommend that you do not use any of these as a mailbox name. The strings Trash, Discard, Junk, Junkmail, Spam, and Ham have a special meaning on some systems. For example, in the default Tuffmail.com configuration, the Trash, Discard, and Junkmail folders are periodically auto-deleted; at Panix.com, “mail in the Trash folder will get thrown out automatically when it's a month old.” To ensure that you have complete control over your mailboxes, do not use any of the special mailbox names on your system. For example, I use a mailbox named MaybeSpam for my possibly-spam messages and I do not need to worry about my system auto-deleting these messages (because MaybeSpam is not a special mailbox on my system). Do not use a forward slash (/), dot (.), or backslash (\) in a mailbox name because these are used as a namespace hierarchy separator on some IMAP servers. For example, Binc IMAP and UW IMAP use forward slash for the hierarchy separator; Lotus & Exchange use backslash; and Dovecot (when using Maildir-format), Mirapoint, Courier, & Cyrus all use dot. (Note that most IMAP servers can be configured to use a non-default hierarchy separator. For example, Tuffmail.com has configured their Cyrus server to use forward slash as the hierarchy separator.) Avoiding these characters in mailbox names makes it possible for you to have a consistent naming style across different IMAP systems, and should make your life easier if your system's IMAP server changes or if you move your mailboxes to a system using a different flavor of IMAP. You might consider not using a dollar sign ($) or exclamation mark (!) in a mailbox name because some IMAP servers, for example Cyrus, do not allow these characters. However, Courier and UW IMAP do seem to allow these characters in a mailbox name. You might consider not using a caret (^) in a mailbox name because Cyrus IMAP will convert any dot (.) in a mailbox name to a caret if the hierarchy separator is changed from slash to dot. If you use ^ in your mailbox names, you will not be able to distinguish between carets that have been converted from dots and carets that have always been carets. I have been wondering if there is any problem with using tilde (~) as the 1st character in a mailbox name? If you have any thoughts about this, please post in this comp.mail.imap thread that I started 2005 Nov 18. If your mail system allows dual-use names (which are aka hybrid folders), i.e., allows a mailbox and a directory (hierarchy) to have the same name, I recommend that you not use the same name for both a mailbox and a directory. This will make your life easier if you move your mailbox hierarchy to a system that does not support dual-use names. For example, if you use UW IMAP with mix-format mailboxes, Dovecot IMAP, Cyrus IMAP, our Courier IMAP, it is possible to create a container named Archive that contains both mailboxes and messages. My advice is to use this Archive container as either a container of only mailboxes (a directory) or as a container of only messages (a mailbox). Some of the other characters and strings to avoid in mailbox names that I list above were suggested in the 2001-July-09 comp.mail.imap message Re: Mailbox Names (Was: Which IMAP Server is better?) and in the 2002-Sep-23 comp.mail.imap thread characters allowed in mailbox names. Do not start a mailbox name with the hash character (#) because some IMAP servers interpret a string that begins with # as an IMAP namespace token. This is discussed in RFC 3501, section 5.1 (Mailbox Naming), which says “Two characters, "#" and "&", have meanings by convention, and should be avoided except when used in that convention.” The next item discusses the problem with ampersand (&). You might also consider not using an ampersand (&), a space, or an upper-case character in a mailbox name for the reasons discussed in the gmane.mail.imap.dovecot thread Ampersand in Folder names causing folders to disappear?, in the gmane.mail.imap.courier.general thread IMAP Folders and the ampersand, in the EMD thread Ampersand in Folder Name, and in the FastMail.FM FAQ about How do I use FastMail.FM's subdomain addresses and "plus- addresses"?  Tip: If you use a Procmail recipe to deliver to a mailbox that contains a space in its name, you need to escape the space by preceding it with a backslash (\). Another reason not to use an ampersand (&) in a mailbox name is that it might be interpreted as a MUTF7 string, rather than an ASCII string, by an IMAP server or client. Some systems have problems with a special mailbox name — such as SENT, DRAFTS, SPAM, and TRASH — if it is created in a non-standard way, e.g. via procmail. This is discussed in this and this thread at EmailDiscussions.com.

My Latest Mailbox-Naming Experiment

Request:  I'm revising my mailbox naming style and I'd like to learn about names that other people use, especially any mailbox-naming scheme that is in common use. I'm thinking about changing the prefix of my incoming mailbox names from IN- to something that will sort above all alphabetic characters in an ASCII sort. I'm leaning towards prefixing these box names with a dash (-), so, for example, incoming procmail mailing-list messages would be put in --procmail. And then I'd put the archive of that mailbox in zz-procmail-YYYY-MM-DD or ==procmail-YYYY-MM-DD because the leading zz- or == will make it sort below my incoming mailboxes. I realize that these are kind of odd-looking names so I'm interested to hear what ideas other people have — please let me know what you think and if you have suggestions.

Note 1:  Someone emailed me and mentioned that if a mailbox name starts with a dash, it is a pain to manipulate from a Unix shell. That is true, but it provides a bit of security if a lamer breaks into your account and tries to read your mailboxes using shell commands such as more or less or pine -f mailboxName (and it's not that much of a pain if you know the trick)!

Note 2:  I'm now using this naming scheme and so far I haven't had problems with Procmail or with any mail client or IMAP server that I've used. For example, here are mailbox names that could be used with my reverse-spam-filtering system:

  --magenta		
  --violet		
  --blue
  --green
  --limegreen		
  --yellow
  --red
  ==archive1		
  ==archive2		
		

To force my most important incoming mailboxes to the top of an ASCII sort, I prefix each mailbox name with an appropriate number of dashes. For example:

  -------green
  ------limegreen		
  -----yellow
  ----blue
  ---red
  --violet		
  -magenta
  ==archive1		
  ==archive2
			

I actually use a modified version of this and don't really have a mailbox name that begins with seven dashes! If you are interested in why I name my incoming mailboxes after the colors of the rainbow, see Reverse Spam Filtering: Winning Without Fighting.

Ordering Procmail Flags

I recommend that you order Procmail flags using the style that Jari Aalto suggests on his Procmail Tips page in the section called The order of the flags. I use this style throughout this page, for example in the SpamAssassin recipes below.

Managing Spam

A lot of people are learning Procmail because they want to automatically separate non-spam messages from spam messages. My strategy is to do the following, in this order:

1. Have my mail hosting provider do SMTP-level spam rejection using, for example, Greylisting, DNSBLs, or SPF (Sender Policy Framework).
2. Quarantine or delete viruses.
3. Deliver or forward known non-spam messages, possibly also setting appropriate flags/labels/keywords.
4. Pipe the possibly-spam (not known non-spam) messages through a spam-detection tool, such as SpamAssassin, to assign a spam score or spam probability.
5. Set appropriate flags/labels/keywords on possibly-spam messages.

To implement step number . . .

1. see IMAP Service Providers page, which describes over 400 providers, many of whom offer SMTP-level spam rejection.
2. see Snagging Viruses below.
3. see some or all of the following:
   1. Step by Step Through Setting Up and Testing Procmail below, which includes recipes that catch and deliver specific “Solicited Bulk Email” (mailing-list and newsletter) messages.
   2. Alternatives to the ^TO_ Macro below, which includes more recipes that catch and deliver specific Solicited Bulk Email messages.
   3. Generic SBE Sorting below.
   4. Setting Keywords or Labels below.
   5. Minimizing False Positives With Greenlists and Bluelists below.
4. see Using SpamAssassin below.
5. see Setting Keywords or Labels below.

For more about my spam-deflexion strategy and about Unsolicited Bulk Email (UBE or spam), see my Reverse Spam Filtering: Winning Without Fighting page.

For other people's spam-deflexion strategies, see the December-2005 gmane.mail.procmail discussion about Spam and Procmail, which takes place in these two threads. Especially useful is this message from G.W. Haywood, in which he describes the system that he uses on his mail servers and says:

I am collecting Spam-related links at del.icio.us/Deflexion.com/Messaging/Spam.

Before You Begin: Note About MH, maildir, and Traditional Unix Spool (mbox) Mailbox Formats

Overview

The step-by-step instructions below guide you through setting up Procmail to deliver to mailboxes that are in traditional Unix mail spool format (mbox). To deliver to a mailbox that is in maildir format, you need to append a forward slash (/) to the mailbox name. The table below gives examples of the various types of mailbox specifications that can be used in the action line of a Procmail recipe.

	If Procmail tries to deliver a message to a maildir, MH, or mbox mailbox that does not exist, Procmail will create the mailbox in the appropriate format and then deliver the message to it. If a maildir or mbox mailbox does not exist, it is safest to let Procmail create it (see previous item) rather than trying to create it yourself using Unix commands. In order for Procmail to successfully deliver to a pre-existing mailbox, the mailbox must be in the appropriate format. For example, a maildir or MH mailbox must be a directory and a Unix mbox mailbox must be a file. If you deliver to maildir mailboxes, use procmail 3.15.2 or 3.22+.  3.14 has serious problems delivering to maildir mailboxes and 3.15, 3.15.1, 3.20, and 3.21 have minor problems. (*) If you use MH mailboxes, you will probably want to use the MH/nmh rcvstore program to deliver messages. Details are in Beedub's Wiki in the section Using the procmail program. If you want to deliver to another type of mailbox, consult the relevant documentation. For example, to deliver to . . . Dovecot IMAP mailboxes, see Gerhard Wiesinger's 2007-March-17 message titled Re: deliver to subfolder with dovecot's LDA - procmail filtering at dovecot.org or at gmane.org. (Note: Gerhard's top page is a cool example of DHTML and, if you have JavaScript enabled in your browser, I recommend you check it out!) Cyrus IMAP mailboxes, see Nicholas Petreley's 2002-March-18 article titled How to set up IMAP on the cheap: How to set up Cyrus, Postfix & Procmail to block spam at LinuxWorld.com or at Archive.org. C-client mbx or mix mailboxes, see Jauder Ho's Implementing IMAP with UW imapd. Jauder's report discusses dmail, which is part of the UW IMAP Toolkit. Also see Jauder's procmailrc and Faisal N. Jawdat's Using mbx Format Mail Files, each of which includes examples of using dmail in Procmail recipes. Note the following, which is on Faisal's page: “The dmail command understands normal unix [mbox] mailboxes as well as mbx [and mix] ... so it's probably wise to migrate all your procmail recipes to using dmail before converting to mbx [or mix] format”

Details

An explanation of how Procmail delivers to various folder formats is in the latest (3.22) procmailrc man page. Here is the relevant section.

  [ . . .]
  Anything else will be taken as a mailbox  name  (either  a
  filename  or a directory, absolute or relative to the cur-
  rent directory (see MAILDIR)).  If it is a  (possibly  yet
  nonexistent) filename, the mail will be appended to it.
  
  If  it  is  a  directory,  the mail will be delivered to a
  newly created, guaranteed to be unique file named $MSGPRE-
  FIX* in the specified directory.  If the mailbox name ends
  in "/.", then this directory  is  presumed  to  be  an  MH
  folder;  i.e.,  procmail will use the next number it finds
  available.  If the mailbox name ends  in  "/",  then  this
  directory  is presumed to be a maildir folder; i.e., proc-
  mail will deliver the message to a file in a  subdirectory
  named  "tmp"  and  rename  it  to be inside a subdirectory
  named "new".  If the mailbox is  specified  to  be  an  MH
  folder  or maildir folder, procmail will create the neces-
  sary directories if they don't exist,  rather  than  treat
  the  mailbox as a non-existent filename.  When procmail is
  delivering to directories, you can specify multiple direc-
  tories  to  deliver  to  (procmail  will  do  so utilising
  hardlinks).

When delivering to a maildir or MH format mailbox, you do not need to use a lock file so the first line of a recipe does not need the second colon. For example, instead of using this as the first line:

   
:0:   

you can use this:

   
:0   

There are examples of recipes that deliver to maildir-formatted mailboxes in Step 7D and in Using SpamAssassin below.

Details Specific to maildir Format

	If you want your system mailboxes $ORGMAIL and $DEFAULT to be in maildir format, it is best to specify this in the procmail source code and recompile procmail. This is discussed in the following messages, which were posted to the Procmail mailing list. Philip Guenther's message Re: Maildirs & "Couldn't rename bogus..." David W. Tamkin's message Re: Maildir format

If you are delivering messages to maildir-formatted mailboxes that will be served to mail clients by Courier IMAP (which supports only maildir format mailboxes), put a dot (.) in front of the mailbox name in your Procmail recipe. This leading dot is a hierarchy separator and will ensure that IMAP clients can see these mailboxes. IMAP clients will display the mailbox name without the leading dot (or the trailing slash, which is just used to tell procmail the format of the mailbox and is not part of the name that's used on the Unix system).

Technical Note:  On Courier IMAP, private mailboxes reside under the “INBOX.” hierarchy. On the server, this hierarchy is usually in the directory $HOME/Maildir but users do not need to tell their IMAP clients the actual name of this directory because the IMAP protocol will pass that information to the IMAP client.

|preline /usr/local/bin/procmail ~/.procmailrc || exit 111
|preline procmail -m -p myprocmail.rc
|preline procmail

See Also

• Sam Varshavchik's message Re: Adding keywords to incoming mail in gmane.mail.imap.courier.general, in which he says “With procmail [delivering to a maildir-formatted mailbox], ordering of new messages is likely to be completely random.”
• wiki.dovecot.org: Questions And Answers -- the first few questions include useful information about Procmail and maildir-formatted folders
• era eriksson's Procmail FAQ: Appendix A -- Folder Formats
• Using maildir format by D. J. Bernstein
• maildir man page
• Courier IMAP Server
• David W. Tamkin's message Re: /var/spool/mail, MailDir and quota in the Procmail list
• Sam Stern's message Re: Maildir... in the Procmail list
• Philip Guenther's message Re: Rerun procmail filters on maildir folder
• Bart Schaefer's 2004-Dec-11 message Re: cannot create /IMAPdir//INBOX/: is a directory, which begins with this:

  “Firstly, the word MAILDIR as a variable name in procmail predates, and has absolutely nothing to do with, the mail storage format called "Maildir" which was invented quite some years later.
  
  Secondly, . . .”

• Step 7D and Using SpamAssassin below, each of which includes examples of Procmail recipes that deliver to maildir-formatted mailboxes

Step by Step Through Setting Up and Testing Procmail

These 18 steps walk you through ensuring that procmail is installed on your system, setting it up to sort your incoming mailing-list messages, and testing your setup.

Preliminaries

	Important:  For the following steps to work, you must be a regular user, not root. If you are a system administrator who is learning Procmail, I highly recommend that you step through this procedure before you try to set up a global procmail rc file. You are much more likely to understand the basics of Procmail if you learn by doing (rather than by reading only). And your system & users will be much less likely to experience an email nightmare if you do your Procmail learning — and making mistakes — as a regular user!

0. To find out if procmail is on your system and, if it is, what the path to it is, type one of the following commands.

    

   Shell	Command
   csh, tcsh	which procmail
   sh, bash, ksh	type procmail
   various	whereis procmail
   various	where procmail
   various	locate procmail

   Make a note of the path to procmail because this is needed in Step 9 below.
   
   
   

1. To find out procmail's version, locking strategies, default rcfile, and your system mailbox (aka your default INBOX), type:

   procmail -v 
   

   	If you are using a version earlier than 3.11pre4, which was released on 29 October 1995, you need to use the procmail expression ^TO rather than the expression ^TO_ in Step 15. If you are using version 3.13 or earlier, procmail cannot deliver to maildir mailboxes (unless you apply a patch to procmail or pipe a message to a program that can deliver to a maildir mailbox). If you deliver to maildir-formatted mailboxes, it is best to use procmail 3.15.2 or 3.22+.  3.14 has serious problems delivering to maildir mailboxes and 3.15, 3.15.1, 3.20, and 3.21 have minor problems.

   
   
   If your system does not have procmail or does not have the latest version, 3.15.2 or 3.22+, ask your system administrator to install it. You can find out what is the latest version of procmail and where to get it at www.procmail.org.
   
   
   
   
2. Rename (move) ~/.forward, ~/.procmailrc, and the global procmailrc, if they exist. This will ensure that you do a clean Procmail setup.
   1. To find out if ~/.forward exists and what its contents are if it does, type:

      cat ~/.forward

      If ~/.forward exists, rename it using a command similar to this:

      mv -i ~/.forward ~/.forward-2007-11-27

      This will ensure that you use the optimal ~/.forward file for your system, which is discussed in Steps 8 and 9 below.
      
      
      
   2. To find out if ~/.procmailrc exists and what its contents are if it does, type:

      cat ~/.procmailrc

      If ~/.procmailrc exists, rename it using a command similar to this:

      mv -i ~/.procmailrc ~/.procmailrc-2007-11-27

      Steps 3 and 4 (in the next section) walk you through setting up a minimal ~/.procmailrc file. If you would like to use some of the recipes that are in your old procmailrc file, I recommend that you call them by using an INCLUDERC command in your new minimal ~/.procmailrc. For an example of an INCLUDERC command, see the end of Step 4 below (in the #### Processing Section ####).
      
      
      
   3. To find out if a global procmailrc exists and what its contents are if it does, type:

      cat /etc/procmailrc
      cat /usr/local/etc/procmailrc
      

      If a global procmailrc exists and if you are the administrator of your system, rename it using a command similar to this:

      sudo mv -i /etc/procmailrc /etc/procmailrc-2007-11-27
                 ^^^^^^^^^^^^^^^
                change to the path
                  on your system

      After you have experience setting up and using a personal procmailrc, you can, if you want, re-install this renamed global procmailrc. Details about using a global procmailrc are in See if There is a Global procmailrc below.
      
      Note: As discussed here, viaVerio VPS accounts use /usr/local/etc/procmailrc for the global procmailrc file. Also note that you can view the VPS default files and directory structure in the /skel directory. (Verio is where I host the Infinite Ink web pages, including this page.)
      
      Warning: As discussed in this December-2005 gmane.mail.procmail thread, it seems that there is a Linux distribution that “out of the box” includes a broken /etc/procmailrc.
      
      
      

Set Up Your Procmail Files for Testing

3. Create ~/.procmailrc by typing:

   cd
   pico -w .procmailrc
   

   	Throughout this article I use pico for editing files. Pico is the Pine composer and is a simple easy-to-learn text editor that is available on most Unix-type systems. Pico's -w command-line argument tells pico to not wrap lines, that is, do not insert extra line breaks. To learn more about Pico, see the pico man page and What is Pico at the University of Washington's Pine Information Center. You can, of course, replace pico with your favorite text editor. In any case: Make sure your editor creates Unix ASCII text files with Line Feed (LF) line terminators. For details about this, see Step 10 in this Step-by-Step section and see Make Sure Your Configuration Files Use LF (Line Feed) Line Terminators in the Troubleshooting section below. On some systems the procmailrc file has a non-standard name or non-standard location. For example: Sonic.net uses both $HOME/.procmailrc-first and $HOME/.procmailrc. Verio Signature accounts use $HOME/etc/users/UserID/.procmailrc for sub-users (i.e., users other than the primary user). This is discussed in the message Procmail on Signature in the viaVerio Reseller Forums (registration required). Make sure you read the mail-filtering documentation for your system before you create your procmail files. Some people will do anything to avoid the “satanic syntax” of Procmail, but others embrace—and even promote—it !  :-)

   
   

 

4. Enter the following in your ~/.procmailrc either by typing it or by using copy & paste*. But first note these warnings.
   

   	Procmail does not support the expansion of tilde (~). You must use $HOME rather than the ~ character within your .procmailrc. This is documented in the procmailrc man page. Make sure you replace  MAILDIR=$HOME/Msgs  with the directory where your personal (non-system-spool) mailboxes are stored and that this directory exists and is writable by your LOGNAME. The MAILDIR variable specifies Procmail's working directory and all relative paths are relative to this directory. When you first test Procmail, do not set the PATH variable. As Philip Guenther said in this message, “you shouldn't alter PATH unless there's something actually missing from it or the order is wrong.” As I discuss in Simplicity above, do not explicitly set any variable unnecessarily. In addition to not explicitly setting the PATH variable, which is discussed in the previous item, it is almost always best to not explicitly set the DEFAULT variable. For examples of problems caused by explicitly setting variables, see: 2006-Jan thread about procmail with mutt 2005-July thread about procmail is creating Maildir files instead of mbox 2005-July thread about Can't change the Subject: The first two of these threads discuss problems caused by explicitly setting DEFAULT and the third thread discusses a problem caused by setting a user-invented variable named FORMAIL. To learn about the Procmail default environment variables on your system, see man procmailrc on your system.

   
   
   

   # Note: Anything after a hash character (#) is a comment & is ignored by procmail.
    
   #### Begin Variables Section ####
        
   # It is essential that you set SHELL to a Bourne-type shell if
   # external commands are run from your procmailrc, for example if
   # you use rc.spamassassin, rc.quarantine, or other advanced recipes.
   # Setting SHELL should not be needed for the simple sorting recipes in 
   # this step-by-step section, but to be safe and to future proof your
   # procmailrc, set it anyway! Details are in Check Your $SHELL and $PATH.