Complex criteria with Smart Mailboxes in Mail.app

Apple's Mail application permits you to create smart mailboxes, however it only allows you to do either `AND` or `OR` operations between all condition. It does, however, support complex conditions. This article describes how the `SmartMailboxes.plist` can be modified to do such complex conditions and will guide you through creating a smart mailbox called "Today Inbox + Flagged" that lists all today's email in in the inbox mailbox as well as all flagged messages regardless of their age or mailbox.

This article uses the ASCII Property List format to list the properties. If you find reading the code hard or confusing, you should be able to convert it to XML using the Property List Editor.

The SmartMailboxes.plist file is located in ~/Library/Mail/SmartMailboxes.plist. Before you start doing anything, back it up. Don't say I didn't warn you... If you are testing things out, make sure you quit mail before modifying the SmartMailboxes.plist file, otherwise you mail will never know about the changes and you will loose all your edits as soon as you quit Mail.

Wherever you see a <uuid> in the examples, you should substitute it for a unique value generated by running uuidgen in Terminal. You should never re-use UUIDs.

OK, now that we have the introduction stuff out of the way, let's dive in. Start by creating a regular smart mailbox that lists all email that has been received today. Call the mailbox "Today", and create a new "Date Received" "is today" condition.

Today Smart Mailbox

Sticking to the defaults, this mailbox will contain all mail messages that have been received today excluding those in Junk, Trash, and Sent mailboxes. Looking at the SmartMailboxes.plist, it should look somewhat similar to the following:

{
    mailboxes = (
        {
            IMAPMailboxAttributes = 72209; 
            MailboxAllCriteriaMustBeSatisfied = YES; 
            MailboxCriteria = (
                {
                    CriterionUniqueId = "<uuid>"; 
                    Header = NotInJunkMailbox; 
                    Name = "omit junk"; 
                }, 
                {
                    CriterionUniqueId = "<uuid>"; 
                    Header = NotInTrashMailbox; 
                    Name = "omit trash"; 
                }, 
                {
                    CriterionUniqueId = "<uuid>"; 
                    Header = NotInASpecialMailbox; 
                    Name = "omit sent"; 
                    SpecialMailboxType = 3; 
                }, 
                {
                    AllCriteriaMustBeSatisfied = YES; 
                    Criteria = (
                        {
                            AllCriteriaMustBeSatisfied = YES; 
                            Criteria = (
                                {
                                    CriterionUniqueId = "<uuid>"; 
                                    DateIsRelative = YES; 
                                    DateUnitType = 1; 
                                    Expression = "-1"; 
                                    Header = DateReceived; 
                                    Qualifier = IsGreaterThan; 
                                }, 
                                {
                                    CriterionUniqueId = "<uuid>"; 
                                    DateIsRelative = YES; 
                                    DateUnitType = 1; 
                                    Expression = 1; 
                                    Header = DateReceived; 
                                    Qualifier = IsLessThan; 
                                }
                            ); 
                            CriterionUniqueId = "<uuid>"; 
                            Header = Compound; 
                            Name = today; 
                        }
                    ); 
                    CriterionUniqueId = "<uuid>"; 
                    Header = Compound; 
                    Name = "user criteria"; 
                }
            ); 
            MailboxID = "<uuid>"; 
            MailboxName = "Today"; 
            MailboxType = 7;
        }
    ); 
    version = 214; 
}

The IMAPMailboxAttributes, CriterionUniqueId, Expression, Name as well as the version values may be different. Keep the values that are in your file, don't modify them.

You may notice that the "Date received is today" condition is created from two other sub conditions. This basically shows how conditions (called criteria in the property file) can be composed from other conditions.

If you stuck to the defaults, there are also three additional conditions that exclude emails from the Junk, Trash, and Sent mailboxes. These are generated from the two check boxes on the bottom of the sheet when creating a smart mailbox.

Inbox Smart Mailbox

Now, Let's create another smart mailbox. Name this one "Inbox" and the condition should be "Message is in Mailbox" "Inbox". Leaving rest to defaults, you should end up with something similar to:

{
    mailboxes = (
        {
            //Here would be all the content of the Today mailbox
        }, 
        {
            IMAPMailboxAttributes = 17; 
            MailboxAllCriteriaMustBeSatisfied = YES; 
            MailboxCriteria = (
                {
                    CriterionUniqueId = "<uuid>"; 
                    Header = NotInJunkMailbox; 
                    Name = "omit junk"; 
                }, 
                {
                    CriterionUniqueId = "<uuid>"; 
                    Header = NotInTrashMailbox; 
                    Name = "omit trash"; 
                }, 
                {
                    CriterionUniqueId = "<uuid>"; 
                    Header = NotInASpecialMailbox; 
                    Name = "omit sent"; 
                    SpecialMailboxType = 3; 
                }, 
                {
                    AllCriteriaMustBeSatisfied = YES; 
                    Criteria = (
                        {
                            CriterionUniqueId = "<uuid>"; 
                            Header = InSpecialMailbox; 
                            SpecialMailboxType = 1; 
                        }
                    ); 
                    CriterionUniqueId = "<uuid>"; 
                    Header = Compound; 
                    Name = "user criteria"; 
                }
            ); 
            MailboxID = "<uuid>"; 
            MailboxName = "Inbox"; 
            MailboxType = 7; 
        }
    ); 
    version = 219; 
}

Now, let's combine the two mailboxes into one, creating a single smart mailbox that will list all emails that are in your inbox and were received today. The really important bits are the criteria.

"Today" criteria:

{
    AllCriteriaMustBeSatisfied = YES; 
    Criteria = (
        {
            AllCriteriaMustBeSatisfied = YES; 
            Criteria = (
                {
                    CriterionUniqueId = "<uuid>"; 
                    DateIsRelative = YES; 
                    DateUnitType = 1; 
                    Expression = "-1"; 
                    Header = DateReceived; 
                    Qualifier = IsGreaterThan; 
                }, 
                {
                    CriterionUniqueId = "<uuid>"; 
                    DateIsRelative = YES; 
                    DateUnitType = 1; 
                    Expression = 1; 
                    Header = DateReceived; 
                    Qualifier = IsLessThan; 
                }
            ); 
            CriterionUniqueId = "<uuid>"; 
            Header = Compound; 
            Name = today; 
        }
    ); 
    CriterionUniqueId = "<uuid>"; 
    Header = Compound; 
    Name = "user criteria"; 
}

"Inbox" criteria:

{
    AllCriteriaMustBeSatisfied = YES; 
    Criteria = (
        {
            CriterionUniqueId = "<uuid>"; 
            Header = InSpecialMailbox; 
            SpecialMailboxType = 1; 
        }
    ); 
    CriterionUniqueId = "<uuid>"; 
    Header = Compound; 
    Name = "user criteria"; 
}

And combining the two criteria into one mailbox called "Today Inbox" we get:

{
    mailboxes = (
        {
            IMAPMailboxAttributes = 17; 
            MailboxAllCriteriaMustBeSatisfied = YES; 
            MailboxCriteria = (
                {
                    CriterionUniqueId = "<uuid>"; 
                    Header = NotInJunkMailbox; 
                    Name = "omit junk"; 
                }, 
                {
                    CriterionUniqueId = "<uuid>"; 
                    Header = NotInTrashMailbox; 
                    Name = "omit trash"; 
                }, 
                {
                    CriterionUniqueId = "<uuid>"; 
                    Header = NotInASpecialMailbox; 
                    Name = "omit sent"; 
                    SpecialMailboxType = 3; 
                }, 
                {

                    AllCriteriaMustBeSatisfied = YES; 
                    Criteria = (
                        {
                            AllCriteriaMustBeSatisfied = YES; 
                            Criteria = (
                                {
                                    CriterionUniqueId = "<uuid>"; 
                                    DateIsRelative = YES; 
                                    DateUnitType = 1; 
                                    Expression = "-1"; 
                                    Header = DateReceived; 
                                    Qualifier = IsGreaterThan; 
                                }, 
                                {
                                    CriterionUniqueId = "<uuid>"; 
                                    DateIsRelative = YES; 
                                    DateUnitType = 1; 
                                    Expression = 1; 
                                    Header = DateReceived; 
                                    Qualifier = IsLessThan; 
                                }
                            ); 
                            CriterionUniqueId = "<uuid>"; 
                            Header = Compound; 
                            Name = today; 
                        }, 
                        {
                            CriterionUniqueId = "<uuid>"; 
                            Header = InSpecialMailbox; 
                            SpecialMailboxType = 1; 
                        }
                    ); 
                    CriterionUniqueId = "<uuid>"; 
                    Header = Compound; 
                    Name = "user criteria"; 
                }
            ); 
            MailboxID = "<uuid>"; 
            MailboxName = "Today Inbox"; 
            MailboxType = 7; 
        }
    ); 
    version = 214; 
}

The combined condition now says "give me all mail that (is not in Junk) AND (is not in Trash) AND (is not in Sent mailboxes) AND (was received today) AND (is in inbox)".

OK, so far nothing really out of the ordinary. What's that? Couldn't we have done this directly in the interface and saved a lot of time? Why, yes! So far we've done what the interface can do: we have bunch of conditions that are just multiplied together and in fact you can just use the standard interface to get to this point. So far this was just an exercise to get familiar with the criteria, how they work and how they are layered. It is quite important you understand that, because in the next steps the layering will get more complex.

The next step is to create another smart mailbox with "Flagged" criteria. This will gives us all flagged messages.

Flagged Smart Mailbox

Moving closer to our target, this will have to be connected with an OR operation to the "Today Inbox" smart mailbox, as we want to show all email in our inbox from today (regardless of flags) or any other message that's been flagged (regardless of mailbox or how old it is).

The criteria to list flagged messages looks like:

{
    AllCriteriaMustBeSatisfied = YES; 
    Criteria = (
        {
            CriterionUniqueId = "<uuid>"; 
            Expression = MessageIsFlagged; 
            Header = Flag; 
            Qualifier = IsEqualTo; 
        }
    ); 
    CriterionUniqueId = "<uuid>"; 
    Header = Compound; 
    Name = "user criteria"; 
}

To chain this to the "Today Inbox" smart mailbox, we need to create another layer in the criteria that will represent the OR operator. In the criteria, this is represented with AllCriteriaMustBeSatisfied = NO block.

Here is an outline of what we want:

{
    mailboxes = (
        {
            IMAPMailboxAttributes = 17; 
            MailboxAllCriteriaMustBeSatisfied = YES; 
            MailboxCriteria = (
                {
                    CriterionUniqueId = "<uuid>"; 
                    Header = NotInJunkMailbox; 
                    Name = "omit junk"; 
                }, 
                {
                    CriterionUniqueId = "<uuid>"; 
                    Header = NotInTrashMailbox; 
                    Name = "omit trash"; 
                }, 
                {
                    CriterionUniqueId = "<uuid>"; 
                    Header = NotInASpecialMailbox; 
                    Name = "omit sent"; 
                    SpecialMailboxType = 3; 
                },
                {
                    AllCriteriaMustBeSatisfied = NO; 
                    Criteria = (
                        {
                            AllCriteriaMustBeSatisfied = YES; 
                            Criteria = (
                                {
                                    //Today criteria 
                                }, 
                                {
                                    //Inbox criteria
                                }
                            ); 
                            CriterionUniqueId = "<uuid>"; 
                            Header = Compound; 
                            Name = "user criteria";
                        }, 
                        {
                            //Flagged criteria 
                        }
                    ); 
                    CriterionUniqueId = "<uuid>"; 
                    Header = Compound; 
                    Name = "compound criteria"; 
                }
            ); 
            MailboxID = "<uuid>"; 
            MailboxName = "Today Inbox + Flagged"; 
            MailboxType = 7; 
        }
    ); 
    version = 214; 
}

The outline now says "give me all mail that: is not in Junk AND is not in Trash AND is not in Sent AND ( ( was received today AND is in inbox ) OR message is flagged )"

And filling in the outline gives us the full "Today Inbox + Flagged" smart mailbox:

{
    mailboxes = (
        {
            IMAPMailboxAttributes = 17; 
            MailboxAllCriteriaMustBeSatisfied = YES; 
            MailboxCriteria = (
                {
                    CriterionUniqueId = "<uuid>"; 
                    Header = NotInJunkMailbox; 
                    Name = "omit junk"; 
                }, 
                {
                    CriterionUniqueId = "<uuid>"; 
                    Header = NotInTrashMailbox; 
                    Name = "omit trash"; 
                }, 
                {
                    CriterionUniqueId = "<uuid>"; 
                    Header = NotInASpecialMailbox; 
                    Name = "omit sent"; 
                    SpecialMailboxType = 3; 
                },
                {
                    AllCriteriaMustBeSatisfied = NO; 
                    Criteria = (
                        {

                            AllCriteriaMustBeSatisfied = YES; 
                            Criteria = (
                                {
                                    AllCriteriaMustBeSatisfied = YES; 
                                    Criteria = (
                                        {
                                            CriterionUniqueId = "<uuid>"; 
                                            DateIsRelative = YES; 
                                            DateUnitType = 1; 
                                            Expression = "-1"; 
                                            Header = DateReceived; 
                                            Qualifier = IsGreaterThan; 
                                        }, 
                                        {
                                            CriterionUniqueId = "<uuid>"; 
                                            DateIsRelative = YES; 
                                            DateUnitType = 1; 
                                            Expression = 1; 
                                            Header = DateReceived; 
                                            Qualifier = IsLessThan; 
                                        }
                                    ); 
                                    CriterionUniqueId = "<uuid>"; 
                                    Header = Compound; 
                                    Name = today; 
                                }, 
                                {
                                    CriterionUniqueId = "<uuid>"; 
                                    Header = InSpecialMailbox; 
                                    SpecialMailboxType = 1; 
                                }
                            ); 
                            CriterionUniqueId = "<uuid>"; 
                            Header = Compound; 
                            Name = "user criteria"; 
                        }, 
                        {
                            AllCriteriaMustBeSatisfied = YES; 
                            Criteria = (
                                {
                                    CriterionUniqueId = "<uuid>"; 
                                    Expression = MessageIsFlagged; 
                                    Header = Flag; 
                                    Qualifier = IsEqualTo; 
                                }
                            ); 
                            CriterionUniqueId = "<uuid>"; 
                            Header = Compound; 
                            Name = "user criteria"; 
                        }
                    ); 
                    CriterionUniqueId = "<uuid>"; 
                    Header = Compound; 
                    Name = "compound criteria"; 
                }
            ); 
            MailboxID = "<uuid>"; 
            MailboxName = "Today Inbox + Flagged"; 
            MailboxType = 7;  
        }
    ); 
    version = 214; 
}

And that's pretty much it! A while ago I was going to make an application to with some visual interface to make this easier to do. Well, it turns out that representing this visually is not as easy as I thought. If there is anyone willing to work out the GUI for this, I'm willing to spend some time writing the parsing/generating bits. Later, rplotkin wrote full hint on editing the SmartMailboxes.plist, but I still get people asking me about it, so this is my answer.

Keep in mind that Mail interface was never intended to deal with complex queries like this, so you won't be able to edit the "Today Inbox + Flagged" mailbox in Mail. Mail will happily open the edit sheet on it, but doing anything other than hitting the cancel button will overwrite your smart mailbox with the top level condition. You won't have problems with the regular smart mailboxes though, and editing those will leave your custom one alone. It also works perfectly find with .Mac syncing.

You'll also notice that there are MailboxIDs, Names, and Expressions can apparently refer to smartmailbox:// URIs. I haven't had the chance to experiment with this, but it seems like you can refer to expressions in other smart mailboxes. This would be awesome if it is so and if you know anything about it, please let me know.

I hope this helps some of you to get more out of the smart mailboxes and hopefully Apple will come up with some way of doing this right inside of Mail.

Comments

I just knew this had to be possible... thanks so much for working it out. It beats me why Mail's built-in options for Rules are much more flexible than for smart mailboxes. Must be the same kind of logic, surely?

At least with OSX 10.5, something similar can be achieved from within the e-mail GUI itself:

  1. Create a smart folder named "Today" that has your "today" messages in it.
  2. Create a smart folder named "Flagged" that has your "flagged" messages in it.
  3. Create a smart folder named "Today Inbox + Flagged" and tell it to include all messages that are in folder "Today" and all messages that are in folder "Flagged".

Voila - smart folders nested within smart folders.

The possibilities are almost endless!

Thanks Apple.

Now, if only I could figure out a way to create smart folders based on the header information to help identify where all my spam is coming from....

@Noah: Nice, I didn't even notice. The only thing I don't like about that is that by doing that, you'll end up with enormous amount of smart folders for some more complex criteria. But for simple things like above, your way/the new way is much simpler and cleaner.