secluded/content/posts/a-perfect-email-setup-for-me.md at 5427a2cccfed514d8209f20c621f6c30b4e0db51

14 KiB

Raw Blame History

title

author

cover

lastmod

Why tags?

Because they're better. Hierarchies are useful for prose and code but not for files, emails, notes, or anything where an item may fit within multiple categories. Imagine you get an email from your Computer Science professor that includes test dates, homework, and information about another assignment. In that same email, he asks every student to reply with something they learned from the previous class as a form of attendance. In a hierarchy, the best place for this might just be a TODO folder even though it would also fit under School, CS, Dates, To read, and Homework. Maybe you have a few minutes and want to clear out some emails that don't require any interaction. In a tag-based workflow, this would be a good time to open To read, get that email out of the way, and remove the To read tag. It would still show up under the other tags so you can find it later and take the time to fully answer the professor's question, add those dates to your calendar, and add the homework assignments to your TODO list. Hierarchies can be quite cumbersome to work with, especially when one folder ends up getting all the data. Tags ensure that you only see what you want when you want it. Tags are more efficient and they will remain my organisation system of choice.

The tools

In short, the tools we will be using are...

mbsync to download our emails
notmuch, the primary way emails will be organised
afew to apply initial notmuch tags based on subject, sender, recipient, etc.
NeoMutt to interact with those emails, reply, compose, add/remove tags, etc.
msmtp for relaying our replies and compositions to our mail provider

Yes, it's a lot. Yes, it's time-consuming to set up. Yes, it's worth it (in my opinion).

`mbsync`

As I said above, IMAP is limiting; we need to use some other method of downloading our emails. There's an awesome piece of software called mbsync which is built for exactly this purpose. Its configuration can be rather daunting if you have as many accounts as I do (19) but it's not terrible.

The following sections are named Near, Far, and Sync. Near and Far are terms mbsync uses to profile how your emails are stored, where they're stored, and how to interact with them. In this guide, Far will our mail provider's IMAP server and Near will be our local Maildir.

Far

IMAPAccount amo_ema
Host imap.nixnet.email
CertificateFile /etc/ssl/certs/ca-certificates.crt
SSLType STARTTLS
User amolith@nixnet.email
PassCmd "secret-tool lookup Title amolith@nixnet.email"

IMAPStore amo_ema-remote
Account amo_ema

Near

MaildirStore amo_ema-local
SubFolders Verbatim
Path ~/new-mail/amo_ema/
Inbox ~/new-mail/amo_ema/INBOX/

In the first block, localrepository and remoterepository tell OfflineIMAP where to look for your emails. use_exa-local is an arbitrary naming scheme I use to differentiate between the various local and remote accounts. It can easily be swapped with something else.

Sync

Channel amo_ema
Far :amo_ema-remote:
Near :amo_ema-local:
SyncState *
Patterns *
Create Both

The repository sections describe how the emails are stored or retrieved. In the local block, you'll notice that the type is Maildir. In this format, each email is given a unique filename and stored in a hierarchy of folders within your account. This is often how your emails are stored on your provider's mail server.

pythonfile is used here to authenticate with the remote server. This can be complicated and depends entirely on how you manage your passwords. I use KeePassXC and love it. When I set OfflineIMAP up, however, it didn't have libsecret compatibility. This would have made setup significantly easier but, as it already just works™, I don't really see a reason to change it.

This new feature allows libresecret-based applications to query KeePassXC for your passwords or store them there on your behalf. CLI/TUI applications that need a secure mechanism for background authentication can use secret-tool lookup Title "TITLE_OF_PASSWORD" as the password command. See the pull request for more details. Because this wasn't a feature when I first set it up, I put my passwords in plaintext files and encrypted them with the GPG key stored on my YubiKey. As long as my key is plugged in, OfflineIMAP can authenticate and download all my emails just fine. The process for using a GPG key not stored on a hardware token is pretty much the same and I'll talk about that process instead.

These are the contents of my ~/.offlineimap.py.

  #! /usr/bin/env python2
  from subprocess import check_output
  def get_pass(account):
      return check_output(["gpg", "-dq", f" ~/.mail_pass/{account}.gpg"]).strip("\n")

This runs gpg -dq ~/.mail_pass/use_exa.gpg then strips the newline character before returning it to OfflineIMAP. -d tells GPG that you're passing it a file you want decrypted and -q tells it not to give any output other than the file's contents. For a setup that works with this Python script, put your passwords in plaintext files with the account name as the file name (e.g. use_exa). You'll then encrypt it with gpg -er <YOUR_KEY_ID> use_exa. Running gpg -dq use_exa.gpg should display your password. Repeat for every account and store the resulting files in ~/.mail_pass/.

The other option, sync_deletes, is whether or not to delete remote emails that have been deleted locally. I enabled that because I want to have easy control over how much remote storage is used.

Here's the next block again so you don't have to scroll up:

[Repository use_exa-remote]
type = IMAP
remotehost = imap.example.com
starttls = yes
ssl = no
remoteport = 143
remoteuser = user@example.com
remotepasseval = get_pass("use_exa")
auth_mechanisms = GSSAPI, XOAUTH2, CRAM-MD5, PLAIN, LOGIN
maxconnections = 1
createfolders = True
sync_deletes = yes

This one's pretty self-explanatory. type, remotehost, starttls, ssl, and remoteport should all be somewhere in your provider's documentation. remoteuser is your email address and remotepasseval is the function that will return your password and allow OfflineIMAP to authenticate. You'll want enter the name of your password file without the .gpg extension; the script takes care of adding that. Leave auth_mechanisms alone and the same for maxconnections unless you know your provider won't rate limit you or something for opening multiple connections. sync_deletes is the same as in the previous block.

Copy those three blocks for as many accounts as you want emails downloaded from. I have 510 lines just for Account and Repository blocks due to the number of address I'm keeping track of.

`notmuch`

notmuch is a fast, global-search, and tag-based email system. This what does all of our organisation as well as what provides the "virtual" mailboxes NeoMutt will display later on. Configuration is incredibly simple. This file goes in ~/.notmuch-config.

  [database]
  path=/home/user/mail/

  [user]
  name=Amolith
  primary_email=user@example.com

  [new]
  tags=unread;new;
  ignore=Trash;

  [search]
  exclude_tags=deleted;spam;

  [maildir]
  synchronize_flags=true

First section is the path to where all of your archives are, the [user] section is where you list all of your accounts, [new] adds tags to mail notmuch hasn't indexed yet and ignores indexing the Trash folder, and [search] ignores mail tagged with deleted or spam. The final section tells notmuch to add maildir flags which correspond with notmuch tags. These flags will be synced to the remote server the next time OfflineIMAP runs and things will be somewhat organised in your webmail interface.

After creating the configuration file, run notmuch new and wait for all of your mail to be indexed. This could take a short amount of time or it could take minutes up to an hour, depending on how many emails you have. After it's finished, you'll be able to run queries and see matching emails:

  $ notmuch search from:user@example.com
  thread:0000000000002e9d  December 28 [1/1] Example User; Random subject that means nothing

This is not terribly useful in and of itself because you can't read it or reply to it or anything. That's where the Mail User Agent (MUA) comes in.

`afew`

afew is an initial tagging script for notmuch. After calling notmuch new, afew will add tags based on headers such as From:, To:, Subject:, etc. as well as handle killed threads and spam. The official quickstart guide is probably the best resource on getting started but I'll include a few tips here as well.

NeoMutt

`msmtp`

msmtp is what's known as a Mail Transfer Agent (MTA). You throw it an email and it will relay that to your mail provider's SMTP server so it can have the proper headers attached for authentication, it can be sent from the proper domain, etc. All the necessary security measures can be applied that prevent your email from going directly to spam or from being rejected outright.

msmtp's configuration is also fairly simple if a bit long, just like OfflineIMAP's.

  # Set default values for all following accounts.
  defaults

  # Use the mail submission port 587 instead of the SMTP port 25.
  port 587

  # Always use TLS.
  tls on

This section just sets the defaults. It uses port 587 (STARTTLS) for all SMTP servers unless otherwise specified and enables TLS.

  account user@example.com
  host smtp.example.com
  from user@example.com
  auth on
  user user@example.com
  passwordeval secret-tool lookup Title "user@example.com"

This section is where things get tedious. When passing an email to msmtp, it looks at the From: header and searches for a block with a matching from line. If it finds one, it will use those configuration options to relay the email. host is simply the SMTP server of your mail provider, sometimes this is mail.example.com, smtp.example.com, etc. I've already explained from, auth simply says that a username and password will have to be provided, user is that username, and passwordeval is a method to obtain the password.

When I got to configuring msmtp, KeePassXC had just released their libsecret integration and I wanted to try it. secret-tool is a command line tool used to store and retrieve passwords from whatever keyring you're using. I think KDE has kwallet and GNOME has gnome-keyring if you already have those set up and want to use them; the process should be quite similar regardless.

As mentioned above secret-tool stores and retrieves passwords. For retrieval, it expects the command to look like this.

secret-tool lookup {attribute} {value} ...

I don't know what kwallet and gnome-keyring's attributes are but this can be used with KeePassXC by specifying the Title attribute. If the password to your email account is stored in KeePassXC with the address as the entry title, you can retrieve it by simply running...

secret-tool lookup Title "user@example.com"

If you have a different naming system, you'll have to experiment and try different things; I don't know what KeePassXC's other attributes are so I can't give other examples.

passwordeval gpg -dq ~/.mail_pass/use_exa.gpg

Now that the whole block is assembled, copy/paste/edit for as many accounts as you want to send email from.

14 KiB

Raw Blame History

Why tags?

The tools

`mbsync`

Far

Near

Sync

`notmuch`

`afew`

NeoMutt

`msmtp`

Summary

TODO Pong fluffy when finished

14 KiB Raw Blame History

Why tags?

The tools

mbsync

Far

Near

Sync

notmuch

afew

NeoMutt

msmtp

Summary

TODO Pong fluffy when finished

14 KiB

Raw Blame History

`mbsync`

`notmuch`

`afew`

`msmtp`