Ebooks, Hazel, and xargs

Although I have a Kindle, I prefer reading ebooks on my iPad, and the ebook reader I like the most is Marvin.1 This means I have to convert my Kindle books to ePubs (the format Marvin understands) and put them in a Dropbox folder that Marvin can access. The tools I use to do this efficiently are Calibre, Hazel, and, occasionally, a one-liner shell script.

If you have ever felt the need to convert the format of an ebook, you’ve probably used Calibre, so I won’t spend more than a couple of sentences describing how I use it.2 When I buy I new book from Amazon, it gets sent to my Kindle. I then copy the book from the Kindle into the Calibre library and convert the format to ePub. This doesn’t do the conversion in place and overwrite the Kindle file; it makes a new file in ePub format.

Calibre organizes its library in subfolders according to author and book. At the top level is the calibre folder, which I keep in Dropbox. At the next level are folders for each author. Within these are folders for each book from a particular author. Finally, within each book folder are the ebook files themselves and some metadata.

Calibre folder structure

This is great, but I find it easiest to configure Marvin to download ePubs from a single Dropbox folder, one that I’ve cleverly named epubs. So I want an automated way to copy the ePub files from the calibre folder structure into the epubs folder.

This is a job for Hazel. Following Noodlesoft’s guidance, I set up two rules for the calibre folder.

Hazel Calibre rules

The first one, “Copy to epubs,” just looks for files with the epub extension and copies them to the epubs folder.

Hazel ePub copy rule

By itself, this rule does nothing, because the ePub files aren’t in the calibre folder itself, they’re two levels down, and Hazel doesn’t look in subfolders without a bit of prodding. That prodding comes from the “Run on subfolders” rule:

Hazel subfolders rule

This action was copied directly from Noodlesoft’s documentation, which says

To solve this problem [running rules in subfolders], Hazel offers a special action: “Run rules on folder contents.” If a subfolder inside your monitored folder matches a rule containing this action, then the other rules in the list will also apply to that subfolder’s contents.

With these rules in place, I don’t have to remember to copy the newly made ePub file into the epubs folder—Hazel does it for me as soon as it recognizes that a new ePub is in the calibre folder structure.

At least it should do it for me. Sometimes—for reasons I haven’t been able to suss out—Hazel doesn’t make the copies it’s supposed to. I’ll open up Marvin expecting to see new books ready to be downloaded to my iPad, and they won’t be there. When that happens, I bring out the heavy artillery: a shell script that combines the find and xargs commands to copy any and all ePubs under the calibre directory into the epubs directory. The one-liner script, called epub-update, looks like this:

#!/bin/bash

find ~/Dropbox/calibre/ -iname "*.epub" -print0 | xargs -0 -I file cp -n file ~/Dropbox/epubs/

The find part of the pipeline collects all files with an epub extension (case-insensitive) under the calibre directory and prints them out separated by a null byte. This null separator is pretty much useless when printing out file names for people, but it’s great when those files are going to be piped to xargs, especially when the file names are likely to have spaces within them.

The xargs part of the pipeline takes the null-separated list of files from the find command and runs cp on them. In the cp command, file is used as a placeholder for the file name and the files are copied to the epubs directory. The -n option tells cp not to overwrite files that already exist.

The advantage of using epub-update when Hazel hiccups is that I don’t have go hunting through subfolders to find the file that didn’t get copied.

I suppose if I were smart, I’d set up my Amazon account to send new purchases to my Mac instead of my Kindle. Then I could automate the importing of new ebooks into the Calibre library and eliminate more of the manual work at the front end of the process. One of the advantages of doing posts like this is that the process of writing up my workflows forces me to confront my own inefficiencies.


  1. iBooks doesn’t give me the control over line spacing and margins that Marvin does. I may change my mind after seeing the all-new, all-improved Books app in iOS 12, but even if I switch, the automations described here will still be useful. 

  2. Also, because it has the worst GUI of any app on my Mac, I can’t bear to post screenshots of it. 


Hypergeometric Obama

On Friday, Barack Obama will speak at the University of Illinois as part of a ceremony at which he will receive the Paul Douglas Award for Ethics in Government. The speech will be at the Auditorium at the south end of the Quad, which doesn’t seat nearly as many people as wanted to see him.

According to this report, 22,611 students signed up for a ticket lottery for just 1,300 openings. My two sons were among the horde, and they learned yesterday, along with 21,309 of their friends, that they didn’t win.

How likely was it that neither would get a seat? The chance of any individual student being picked was

[\frac{1,300}{22,611} = 0.0575]

which means that the chance of any individual student not being picked was [1 - 0.0575 = 0.9425]. So the probability that neither would be picked was

[(0.9425)(0.9425) = 0.8883]

or just under 90%. So the fact that neither of them got in wasn’t a surprise. We can also calculate the probability that both were lucky,

[(0.0575)(0.0575) = 0.00331]

which is extremely low, and the probability that one got in and one didn’t,

[2(0.0575)(0.9425) = 0.1084]

which isn’t too bad, but unfortunately it didn’t work out.

These calculations are simple and give us answers that are accurate enough for our purposes, given the large numbers (1,300 and 22,611) involved. But we’ve made some approximations that won’t be reasonable if the numbers are small.

For example, let’s assume the same general problem, but this time we’ll assume there are 10 applicants—two of whom are my sons—for 2 tickets. What are the probabilities of zero, one, and two of my sons getting tickets?

If we follow the procedure above, we get

[(0.20)(0.20) = 0.040]

for both sons getting a ticket,

[2(0.20)(0.80) = 0.32]

for one son getting a ticket and one not, and

[(0.80)(0.80) = 0.64]

for neither getting a ticket. These calculations are internally consistent, in that they add up to unity, but they’re wrong.

The problem is that these calculations are based on an assumption that one son winning a ticket is independent of the other son winning, but they are not independent.

The equation for calculating the probability of an intersection of two events, call them [A] and [B], is

[P(A \cap B) = P(B\,|\,A)P(A) = P(A\,|\,B)P(B)]

where [P(B\,|\,A)] is the probability of event [B] given that event [A] occurs, and similarly, [P(A\,|\,B)] is the probability of event [A] given that event [B] occurs. If the events are independent, then the conditions don’t matter,

[P(B\,|\,A) = P(B)\qquad \mathrm{and} \qquad P(A\,|\,B) = P(A)]

and

[P(A \cap B) = P(A)P(B)]

But that’s not the situation here. If Son A gets a ticket, then Son B’s chance of getting a ticket isn’t [2/10 = 0.20], it’s [1/9 = 0.1111] because with Son A having a ticket, there’s only one ticket left for the nine remaining applicants. Which means the probability that both sons get a ticket is

[\frac{1}{9}\frac{2}{10} = \frac{1}{45} = 0.02222]

which just over half of our earlier, mistaken, calculation of [0.040].

Similarly, the probability that one son gets a ticket and the other doesn’t is

[\frac{8}{9}\frac{2}{10} + \frac{2}{9}\frac{8}{10} = \frac{16}{45} = 0.3556]

and the probability that neither get a ticket is

[\frac{7}{9}\frac{8}{10} = \frac{28}{45} = 0.6222]

(And now we see why it was OK to play a little loose with the numbers back in our original calculations with 22,611 applicants for 1,300 tickets. There just isn’t enough difference between

[\frac{1,300}{22,611} \frac{1,300}{22,611}]

and

[\frac{1,300}{22,611} \frac{1,299}{22,610}]

to make it worth even the minimal effort.)

All of the denomimators in the previous results were 45. It will probably not surprise you that 45 is the number of combinations of ten things taken two at a time. It’s also the binomial coefficient, and is usually written like a fraction but without the horizontal dividing line and surrounded by parentheses. It’s calculated though factorials:

[\binom{n}{k} = \frac{n!}{k! \, (n-k)!}]

In our particular case of [n=10] and [k=2],

[\binom{10}{2} = \frac{10!}{2! \; 8!} = \frac{(10)(9)}{2} = 45]

Imagine ten lottery balls with the numbers 0 through 9 printed on them. Mix them up and draw two. There are 45 different results you can get if you don’t care about the order of the two balls. Here they are:

0-1  0-2  0-3  0-4  0-5  0-6  0-7  0-8  0-9
     1-2  1-3  1-4  1-5  1-6  1-7  1-8  1-9
          2-3  2-4  2-5  2-6  2-7  2-8  2-9
               3-4  3-5  3-6  3-7  3-8  3-9
                    4-5  4-6  4-7  4-8  4-9
                         5-6  5-7  5-8  5-9
                              6-7  6-8  6-9
                                   7-8  7-9
                                        8-9

If two of the balls—say 3 and 7, just as an example—represent winning a ticket, scanning the list shows that there’s one combination with both 3 and 7; 16 combinations with a 3 or a 7 but not both; and 28 combinations with neither a 3 nor a 7. These are the numerators in the answers above.

There’s another way to visualize this that might make more sense to you. Image a line of ten people. Two of them get handed tickets (X), the other eight don’t (O). Here are the 45 ways the two tickets can be distributed:

XXOOOOOOOO  XOXOOOOOOO  XOOXOOOOOO  XOOOXOOOOO  XOOOOXOOOO  XOOOOOXOOO  XOOOOOOXOO  XOOOOOOOXO  XOOOOOOOOX
            OXXOOOOOOO  OXOXOOOOOO  OXOOXOOOOO  OXOOOXOOOO  OXOOOOXOOO  OXOOOOOXOO  OXOOOOOOXO  OXOOOOOOOX
                        OOXXOOOOOO  OOXOXOOOOO  OOXOOXOOOO  OOXOOOXOOO  OOXOOOOXOO  OOXOOOOOXO  OOXOOOOOOX
                                    OOOXXOOOOO  OOOXOXOOOO  OOOXOOXOOO  OOOXOOOXOO  OOOXOOOOXO  OOOXOOOOOX
                                                OOOOXXOOOO  OOOOXOXOOO  OOOOXOOXOO  OOOOXOOOXO  OOOOXOOOOX
                                                            OOOOOXXOOO  OOOOOXOXOO  OOOOOXOOXO  OOOOOXOOOX
                                                                        OOOOOOXXOO  OOOOOOXOXO  OOOOOOXOOX
                                                                                    OOOOOOOXXO  OOOOOOOXOX
                                                                                                OOOOOOOOXX

Imagine now that my two sons are at the left end of the line (they could be anywhere—like positions 3 and 7—the results won’t change). There’s one arrangement where they get both tickets (the upper left corner), 16 arrangements where one of them gets a ticket and the other doesn’t (the top two rows except for the upper left corner), and 28 arrangements where neither of them get a ticket (everywhere else).

There is, as you might imagine, a generalization of this problem to account for any number of applications, tickets, and sons. It’s called the hypergeometric distribution. Using the nomenclature in the linked Wikipedia article, we’ll assume a population of [N] things (applications), of which [K] are successes (tickets). If we drawn [n] samples (sons) from that population (without replacement), then the probability that [k] of the samples will be successes is

[\frac{\binom{K}{k} \binom{N-K}{n-k}}{\binom{N}{n}}]

The denominator should look familiar, it’s the number of combinations of [N] things taken [n] at a time.

The first term in the numerator is the number of ways the successes in the sample ([k]) can be taken from the successes in the population ([K]). The second term in the numerator is the number of ways the failures in the sample ([n - k]) can be taken from the failures in the population ([N - K]). This product is a formalization of the counting we did above.

Let’s use this formula to repeat the example with ten applications for tickets ([N = 10]), two tickets ([k = 2]), and two sons ([n = 2]). The probability that both sons will get a ticket is

[\frac{\binom{2}{2} \binom{8}{0}}{\binom{10}{2}} = \frac{(1)(1)}{45} = 0.0222]

The probability that exactly one son will get a ticket is

[\frac{\binom{2}{1} \binom{8}{1}}{\binom{10}{2}} \frac{(2)(8)}{45} = 0.3556]

And the probability that neither one will get a ticket is

[\frac{\binom{2}{0} \binom{8}{2}}{\binom{10}{2}} = \frac{(1)(28)}{45} = 0.6222]

Just like before, but with less counting.

The SciPy library for Python has a sublibrary called stats with a set of functions for handling the hypergeometric distribution. Here’s how to do this last calculation in Python:

python:
>>> from scipy.stats import hypergeom
>>> hypergeom.pmf(0, 10, 2, 2)
0.62222222222222201

The pmf function stands for “probability mass function,” and it represents the formula defined above. The first argument is the number of successes in the sample, the second is the size of the population, the third is the number of successes in the population, and the fourth is the sample size.

I think this ordering of the arguments was an incredibly stupid choice by the designers of the library, as it puts the population numbers in between the sample numbers and the order of the population numbers is the reverse of the order of the sample numbers. It’s hard to imagine a less intuitive way to define the argument order. And don’t get me started on the symbols they use in the documentation. But at least the function works.

We can now go back to our original problem of 1,300 tickets, 22,611 applications, and 2 sons and do it right.

python:
>>> hypergeom.pmf(2, 22611, 1300, 2)
0.0033031794730568834
>>> hypergeom.pmf(1, 22611, 1300, 2)
0.10838192109526341
>>> hypergeom.pmf(0, 22611, 1300, 2)
0.88831489943503728

As expected, no practical difference between these answers and the approximations we started out with. But it’s the journey, not the destination, right?


An image and PDF grab bag

In my job, I often refer to provisions of building codes or material and equipment standards in my reports. Usually, simply quoting the relevant provisions is sufficient, but sometimes I need to attach one or more pages from these documents as an addendum. In the old days, that meant photocopies; now it typically means pulling pages out of PDFs. Preview is pretty good tool for this, as it allows you to use the thumbnail sidebar to extract and rearrange pages. But I recently ran into a situation where Preview couldn’t do the job alone, and I had to use a series of command-line tools to get the job done.

The problem is with the American Institute of Steel Construction, which has decided to publish its essential Steel Construction Manual as a website instead of a PDF. Each “page” of the website looks like the corresponding page of the print edition of the manual.

AISC title page from website

Having this website instead of a PDF is moderately annoying when I’m trying to use the manual, it’s really annoying when I need to pull out excerpts, because I have to make screenshots of each page, edit them, convert them to PDFs, resize them to fit on letter-sized pages, and assemble them into a single coherent PDF document. Here’s how I do it.

First, I take the screenshots on my 9.7″ iPad Pro in portrait mode (see above) because that gives me good resolution of a single page. I could get slightly higher resolution by taking the screenshots on my 2017 27″ 5k iMac, but that machine isn’t available when I’m working at home, where the Mac on my desk is a non-Retina 2012 27″ iMac. After screenshotting, I have a bunch of JPEG files on my iPad, which I copy over to my Mac via the Files app and Dropbox.

Next, I move to the Mac (whichever one is handy) and crop each image down to just the page image, eliminating all the browser chrome and navigation controls. For this I use the mogrify command from the ImageMagick suite of tools to crop the images in place.1 After a bit of trial and error, I learned that

mogrify -crop 1214x1820+162+224 image.jpg

gives the crop size and offset that leaves just the page image.

Cropped page from AISC

Of course, I don’t want to enter this command for every screenshot, so I wrote a shell script, called aisc-crop, which loops through all of its arguments, running the mogrify command on each:

bash:
!/bin/bash

for f in "$@"
do
  mogrify -crop 1214x1820+162+224 "$f"
done

With this, I can crop all the images in a directory with a single command:

aisc-crop *.JPG

Now that I have the page images I want, it’s time to turn them into PDFs. For this, I use the built-in sips command, but sips wants the extension to be .pdf before it does the conversion. So I use Larry Wall’s old rename Perl script:

rename 's/JPG/pdf/' *.JPG

Now the files are ready for conversion:

sips -s format pdf *.pdf

Time to put all the pages together into a single PDF document. For this, I like using PDFtk (which can also be installed via Homebrew):

pdftk *.pdf cat output aisc-pages.pdf

At this point, I have a PDF document with all the pages I want, but the pages aren’t letter-sized. If I open the document in Preview and Get Info on it, I see this:

PDF page size after conversion

The page size is so big because sips treated every pixel in the JPEG as a point in the converted PDF. Since there are 72 points per inch, the PDF pages are [\frac{1820}{72} = 27.28\; \mathrm{in}] high. To get the PDF down to letter size, I use pdfjam, which got installed along with my TeX installation:

pdfjam --paper letterpaper --suffix letter aisc-pages.pdf

Now I have a document named aisc-pages-letter.pdf that’s the right physical size with higher dpi of the embedded images. I could have gotten the same result by “printing” aisc-pages.pdf to a new PDF with the Scale to Fit option selected in the Print sheet, but where’s the fun in that?

Now I can open the document in Preview and rearrange the pages if I didn’t take the screenshots in the right order. Otherwise, I’m done. As is often the case, it takes longer to explain than to do.


  1. ImageMagick used to be kind of hard to install on a Mac, but not anymore. As Jason Snell showed us a couple of months ago, just use Homebrew and brew install imagemagick


Adding SWA calendar events on iOS

One of my favorite pieces of home-grown automation is my swicsfix script, which edits the calendar event that Southwest Airlines provides after you make a reservation and adds alarms set to go off shortly before the 24-hour checkin window opens.1 It’s a great solution, especially when combined with Hazel, but I have to be at a Mac to use it. Since I often make reservations when I’m on the road with only my iPad and iPhone, I wanted an iOS-based solution. What I came up with is a combination of Workflow (which will soon be Shortcuts) and Pythonista.

Before getting into the details of the workflow and the script, let’s see how it works. We start on the Southwest webpage for the reservation. Tap on the “Add to calendar” link to bring up a popup with links to each flight in that reservation.

Southwest calendar links

In this case, it’s a round-trip reservation, so there are two links. Long-pressing on one of them brings up a popup menu of things that can be done with that link:

Long-press popup

Choosing “Share…” brings up the standard Sharing Sheet, from which we choose Workflow. That brings up a list of Action Extension workflows that accept URLs as their input.

Workflow popup

Choose “SWA to Calendar” from this list and wait for the workflow to run. The workflow ends up at Pythonista (something I want to change, but I haven’t figured out the best way yet) and soon an email will come with the calendar event for the flight as an attachment.

Emailed calendar entry

This is like any other event you get emailed. Tap on it to add it to one of your calendars.

Add emailed event to calendar

Because this is a round-trip reservation, go through the same steps for the other leg.

Now we’ll dig into the details. Here are the steps of the “SWA to Calendar” workflow.

SWA to Calendar workflow

It’s defined as an Action Extension, and it accepts URLs as input. The first step is to percent-encode the input URL. This is then used in the second step to create a new URL that will invoke Pythonista and run a script. The new URL is cut off in the screenshot because the variable at the end didn’t get word-wrapped properly. Here it is:

pythonista3://swicsfix-mail.py?root=icloud&action=run&argv=[URL Encoded Text]

where the [URL Encoded Text] part at the end shouldn’t be typed in literally. It’s a magic variable that Workflow should show at the bottom of the screen while you’re typing, and you enter it by tapping on it.

Workflow magic variables list

The last step is to open the URL just created, which will run the swicsfix-mail.py Pythonista script. Because I want this script synced to all my devices, I have it saved in iCloud, which is why the root=icloud part is in the URL above.2

Now we get to the meat of this workflow: the swicsfix-mail.py script.

python:
  1:  import requests
  2:  from icalendar import Calendar
  3:  import sys
  4:  import copy
  5:  from datetime import timedelta
  6:  import smtplib
  7:  from email.message import EmailMessage
  8:  from email.mime.text import MIMEText
  9:  import re
 10:  import keychain
 11:  
 12:  # Parameters
 13:  mailFrom = 'somebody@somewhere.com'
 14:  mailTo = 'somebody@somewhere.com'
 15:  fmServer = 'smtp.fastmail.com'
 16:  fmPort = 465
 17:  fmUser = keychain.get_password('fastmail', 'user')
 18:  fmPassword = keychain.get_password('fastmail', 'password')
 19:  
 20:  # iCalendar transformation function
 21:  def fixICS(s):
 22:    '''Fix Southwest Airlines iCalendar (ICS) text.
 23:    
 24:    Change the summary to the flight and confirmation numbers.
 25:    Delete the description and Microsoft-specific fields.
 26:    Set audible alarms to 24 hours + 15 minutes and
 27:    24 hours + 2 minutes before flight.
 28:    Return a tuple with the summary and the fixed iCalendar text.
 29:    '''
 30:      
 31:    cal = Calendar.from_ical(s)
 32:    event = cal.walk('vevent')[0]
 33:    alarm = event.walk('valarm')[0]
 34:    
 35:    # Make no changes if summary doesn't contain "Southwest Airlines".
 36:    if 'Southwest Airlines' not in event['summary']:
 37:      sys.exit()
 38:    
 39:    # The last word in the location is the flight number.
 40:    # The last word in the summary is the confirmation number.
 41:    flight = event['location'].split()[-1]
 42:    confirmation = event['summary'].split()[-1]
 43:    
 44:    # Erase the event's verbose description and rewrite its summary.
 45:    event['description'] = ''
 46:    event['summary'] = 'SW {} ({})'.format(flight, confirmation)
 47:    
 48:    # Get rid of the mistaken MS fields.
 49:    for e in ('x-microsoft-cdo-alldayevent', 'x-microsoft-cdo-busystatus'):
 50:      try:
 51:        del event[e]
 52:      except KeyError:
 53:        continue
 54:    
 55:    # Set an alarm to 24 hours, 15 minutes before the flight, rewrite its
 56:    # description, and make it audible.
 57:    alarm['trigger'].dt = timedelta(days=-1, minutes=-15)
 58:    alarm['description'] = 'Check in SW {} ({})'.format(flight, confirmation)
 59:    alarm['action'] = 'AUDIO'
 60:    alarm.add('ATTACH;VALUE=URI', 'Basso')
 61:    
 62:    # Delete any UIDs before copying.
 63:    for u in ('uid', 'x-wr-alarmuid'):
 64:      try:
 65:        del alarm[u]
 66:      except KeyError:
 67:        continue
 68:    
 69:    # Add a new alarm 24 hours, 2 minutes before the flight by copying
 70:    # the previous alarm and changing its trigger time.
 71:    newalarm = copy.deepcopy(alarm)
 72:    newalarm['trigger'].dt = timedelta(days=-1, minutes=-2)
 73:    event.add_component(newalarm)
 74:    
 75:    return event['summary'], cal.to_ical().decode()
 76:  
 77:  
 78:  # Get the iCal data by opening the URL passed in.
 79:  icalURL = sys.argv[1]
 80:  r = requests.get(icalURL)
 81:  summary, ics = fixICS(r.text)
 82:  
 83:  # Set the name of the attached file from the summary.
 84:  icsName = re.sub(r' \([^)]+\)', '', summary)
 85:  icsName = re.sub(r'[ /:]', '-', icsName) + '.ics'
 86:  
 87:  # Compose the message.
 88:  msg = EmailMessage()
 89:  msg['Subject'] = summary
 90:  msg['From'] = mailFrom
 91:  msg['To'] = mailTo
 92:  msg.set_content("Calendar entry\n")
 93:  
 94:  # Add the attachment.
 95:  ics = MIMEText(ics)
 96:  ics.add_header('Content-Disposition', 'attachment', filename=icsName)
 97:  msg.make_mixed()
 98:  msg.attach(ics)
 99:  
100:  # Send the message through FastMail.
101:  server = smtplib.SMTP_SSL(fmServer, port=fmPort)
102:  server.login(fmUser, fmPassword)
103:  server.send_message(msg)
104:  server.quit()

I’m not going to explain the fixICS function because it’s basically the same code as in the aforelinked post, just wrapped into a function. It takes the iCalendar text supplied by Southwest and returns a tuple with the “corrected” iCalendar text and the event summary.

Lines 12–17 establish parameters used later in the script. If you want to make your own version of this, you’ll have to change all of these to what fits your situation. The mailFrom and mailTo variables are the From and To entries in the email the script sends; they don’t have to be the same. The rest of the variables in this section are specific to FastMail, the email server I use. If you use another email server, you’ll have to change the server address, the port, and the login information. You may also have to change a function call down near the bottom of the script, which we’ll get to in a bit.

I use the Pythonista keychain library to avoid having the login information explicitly saved in the script. That, of course, means I have to put the login information into the keychain before running this script. It’s a short script:

python:
1:  import keychain
2:  
3:  keychain.set_password('fastmail', 'user', 'myusername')
4:  keychain.set_password('fastmail', 'password', 's3cr3tp455w0rd')

You could even do this interactively from the Pythonista console.

The script starts its real work on Line 78. It takes the URL passed in through the argv parameter defined in the second Workflow step and saves it in the icalURL variable. This URL is the for the Southwest link we long-pressed. Line 79 uses the Requests library to follow that link to the iCalendar text from Southwest. Line 80 calls fixICS to run convert it into the form I like. The summary (which we’ll use for the email subject line and the attachment file name) is saved in the summary variable, and the converted iCalendar text is stored in the ics variable.

The summary variable looks like this:

SW 1234 (WX84R4)

with the flight number first and then the confirmation number in parentheses. If it’s a flight with a connection, the two flights are separated by a slash:

SW 1234/5678 (WX84R4)

Lines 83–and 84 use this to create the attachment file name, icsName. Basically, they get rid of the confirmation number and change all spaces, slashes, and colons to hyphens. As far as I know, flight numbers never have colons, but thirty-plus years of dealing with Macs makes me paranoid about the possibility of a colon in a file name. Just as twenty-plus years of using Unix makes me paranoid about slashes.

Lines 87–97 create the email message. Lines 88–91 define the From and To addresses, the Subject, and the body of the message. Lines 94–97 define the iCalendar text in ics as a MIME type and add it to the message as an attachment.

Finally, Lines 100–103 send the message. Line 100, which opens a connection to the email SMTP server, assumes the server uses SSL authentication and encryption. This is true for FastMail and many other email servers but might not be true for yours, so you may need to change this line. The other ways of connecting to an SMTP server are described in the Python smtplib library documentation.

This script took a long time to write, not because it was especially complicated, but because I didn’t want to use email to add the event. I spent most of my time trying to serve the ICS data via HTTP. The attempts mostly looked like this:

python:
from http.server import BaseHTTPRequestHandler, HTTPServer
import webbrowser

class MyHandler(BaseHTTPRequestHandler):
  def do_GET(s):
    """Respond to a GET request."""
    s.send_response(200)
    s.send_header("Content-Type", "text/calendar")
    s.end_headers()
    s.wfile.write(ics.encode())
  def log_message(self, format, *args):
    return

# Start the server and open Safari.
port = 8888
httpd = HTTPServer(('127.0.0.1', port), MyHandler)
webbrowser.get('Safari').open_new_tab('http://localhost:{}'.format(port))
httpd.handle_request()

The idea was use Python to create a local HTTP server on port 8888 that would deliver the iCalendar data when called. The webbrowser.get line would then open Safari to make the call. When this worked, it was really slick, because there was no messing around with Mail. The script would put me in Safari and a popup would appear to allow me to add the event to my calendar (similar to the popup that appears when I tap the event attachment in Mail).

Unfortunately, it didn’t always work. The popup that allowed me to add the event to my calendar would often disappear as soon as it appeared. I tried different HTTP headers, different ways of calling Safari through webbrowser—nothing worked consistently. I even uploaded the iCalendar file to the leancrew server and linked Safari to that static file. That, too, would work sometimes but not always. So after much fruitless effort, I fell back to the email solution, which is clumsy but reliable.

If you have any suggestions for an HTTP-based solution, I’m all ears. In the meantime, I have something that works.


  1. Yes, Southwest now has early bird checkin that makes these alarms less useful, but I seldom use it. 

  2. It’s also the reason I couldn’t use Workflow’s “Run Script” action as the last step. It only knows how to run scripts saved in the local area, not in iCloud.