SSH keys

This is the last in my short series of screenshot posts. The first two cover the macro/scripts SnapClip and SnapSCP; this one isn’t about screenshots per se, but it describes how to set up SSH keys on your both your server and your local Mac so the SnapSCP macro can upload your screenshots securely without forcing you to type in a password or passphrase.

Let me start by saying that not a single bit of this post is original. In fact, I don’t really know much about SSH, SSH keys, or the ssh-agent program. I’m just collecting instructions from various sources and putting them together in one place so you don’t have to bounce around from page to page, trying to piece together a process that works. Some of the instructions in this post are old; I first performed some of these steps as a Linux user 15 years ago. Some of the instructions are required only because Apple tweaked the key handling system in Sierra; I had to perform them quite recently because what had been working for years suddenly stopped after an OS upgrade.

Frankly, I’m somewhat reluctant to write this post, as my lack of expertise means I can’t really troubleshoot if things don’t work for you. Still, these steps work for me and ought to work for you if you’re running a Mac with Sierra and need to connect to a Linux or other Unix-like (including macOS) server via SSH, SCP, or SFTP.

The instructions here assume you’ll be starting from scratch. If you’ve already performed some of the steps in a previous attempt at getting SSH keys to work, look through the steps to see if you did them right and pick up where you left off.

Create the keys

Start by opening a Terminal window. You’ll probably start out in your home directory, but to be sure, run this command:

cd

Now create an RSA key pair by running this command:

ssh-keygen -t rsa

The ssh-keygen command is interactive. It will tell you what it’s doing and ask you for input. The initial response will be

Generating public/private rsa key pair.
Enter file in which to save the key (/Users/drdrang/.ssh/id_rsa):

except, of course, the default place for the key will be in your home directory, not mine. Hit the Return key to accept the default. You’ll then be asked for a passphrase:

Enter passphrase (empty for no passphrase):

Do not use an empty passphrase. You may have seen people on the internet suggesting you use an empty passphrase as a way to avoid entering a password or passphrase every time you want to connect via SSH, SCP, or SFTP. Those people are misguided. When you get done with these instructions, you’ll have the best of both worlds: a passphrase to maintain secure access and a system that doesn’t require you to enter it all the time.

What should your passphrase be? Well, that’s up to you. I used this variant of the Diceware algorithm to create a six-word passphrase, but there are lots of ways to make a good passphrase. The internet is filled with advice on this topic. You could do worse than read this article and this one from the folks at AgileBits (makers of 1Password). Their advice is geared toward generating master passwords for 1Password, but it’s applicable here, too.

After you’ve entered your passphrase, you’re asked to enter it again.

Enter same passphrase again: 

Assuming you entered it correctly the second time, you’ll be told of your success and what ssh-keygen has done:

Your identification has been saved in /Users/drdrang/.ssh/id_rsa.
Your public key has been saved in /Users/drdrang/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:toaZ40HBr/WsWVgbJrIesr6FkmIVeeyuHMpw4u0rQwE drdrang@mba.local
The key's randomart image is:
+---[RSA 2048]----+
|                 |
|E   o.           |
|.  o oo          |
| .  +  o         |
|  .. .o S +      |
| ....o X O o     |
|=.+.o.@ + =      |
|=*+.o* = +       |
| ++B=.o o        |
+----[SHA256]-----+

I don’t know what to use the fingerprint or the randomart image for, but luckily I don’t have to. By the way, if you are an expert on this stuff and are horrified that I’ve posted something that can be reverse-engineered to get my passphrase or keys, don’t worry. This is the output of a throwaway ssh-keygen run.

At this point, you have a key pair in the .ssh directory of your home directory: a private key, id_rsa, and a public key, id_rsa.pub.

Copy the public key to the server

To get the server to understand your newly created RSA identity, you need to put a copy of your public key, id_rsa.pub, into the ~/.ssh directory on the server. There are lots of ways to do this, but the easiest is this:

ssh-copy-id -p 9876 username@12.34.56.78

where you replace username with your user name on the server, 12.34.56.78 with the IP address of the server, and 9876 with the SSH port number of the server. If your server uses 22 as the SSH port (which is the standard), you can omit the whole -p 9876 part.

You’ll get this kind of response:

The authenticity of host '12.34.56.78 (12.34.56.78)' can't be established.
RSA key fingerprint is b1:2d:33:67:ce:35:4d:5f:f3:a8:cd:c0:c4:48:86:12.
Are you sure you want to continue connecting (yes/no)?

Type yes in response to the question and you’ll see

Warning: Permanently added '12.34.56.78' (RSA) to the list of known hosts.
username@12.34.56.78's password:

Go ahead and type in your server password (not your SSH passphrase; the server doesn’t know it yet) and you’ll see something like this:

Now try logging into the machine, with "ssh 'username@12.34.56.78'", and check in:

  ~/.ssh/authorized_keys

to make sure we haven't added extra keys that you weren't expecting.

At this point, you should have a file named known_hosts in the .ssh folder on your Mac.

To make sure everything worked, try to log into the server:

ssh -p 9876 username@12.34.56.78

Now you should be prompted for your SSH passphrase, not your user password, because the server knows you through your public key.

Check the authorized_keys file on the server:

cat ~/.ssh/authorized_keys

You should see a stream of characters that matches the contents of ~/.ssh/id_rsa.pub on your Mac.

At this point, you may be wondering what you’ve gotten yourself into. You used to be able to log into the server with just a password. Now you need this ridiculously long passphrase. Patience, grasshopper, we’re nearly there.

Configure your keychain

So far, there’s nothing Mac-specific in these instructions. The things you’ve been doing on your Mac would work just as well on any computer. That’s about to change.

SSH allows you to manage your keys through the ssh-agent utility. Discussions of ssh-agent you may find on the web often refer to a keychain. This is not the Keychain that Mac and iOS devices use to manage passwords, but Apple has kind of tied the two ideas together. With a little work, we can get our SSH keys automatically entered when we start a SSH, SCP, or SFTP session the same way Keychain can automatically enter our username and password when we visit Amazon.

It’s this keychain stuff that changed with Sierra. Some of what follows applies to both pre-Sierra and Sierra, some applies to Sierra only. I’ll point out which is which.

Sierra and pre-Sierra: First, add your private key to the keychain using the ssh-add command:

ssh-add -K ~/.ssh/id_rsa
ssh-add -A

Now your keys are being managed by the keychain, and you should be able to log into the server without being asked for your passphrase.

Sierra only: Unfortunately, Sierra doesn’t maintain your keychain across restarts. By default, this forces you to re-enter the two ssh-add commands every time you restart your Mac. But there’s a way around it.

Look in your ~/.ssh folder for a file named config. If you have one, open it up in a text editor. If not, create one and open it in a text editor.1 Add the following lines to it:

Host *
  UseKeychain yes
  AddKeysToAgent yes
  IdentityFile ~/.ssh/id_rsa

With this addition, Sierra should maintain your keychain across restarts, and your SSH keys will be managed automatically. And for the purposes of taking and uploading screenshots, the SCP command that’s buried in the SnapSCP macro will run without needing you to enter a passphrase.

Credits

As I said, I’m no expert on SSH or keys. Over the years, though, I’ve read several articles on these topics, and a couple have really stood out. First, this one from Digital Ocean gives the the most succinct description of creating keys and getting them installed on your server. Digital Ocean actually has a lot of good, practical articles on getting things working on a Linux server. I’d be reading them even if ANIAT wasn’t hosted there.

As for key management on the Mac, this Super User Q&A is the best, especially Jeff McCarrell’s discussion of ssh-agent and ChrisJF’s config file solution for Sierra. What would we do without the StackExchange family of sites?

Update 03/1/2017 7:03 AM
Two updates from readers:

  • Chris Finazzo directed me to this Apple Technical Note, which confirms the keychain changes in Sierra and the way to work around them. If you feel uncomfortable running code from a comments-driven website, this should settle your fears.

  • Michael Bishop says ssh-copy-id wasn’t included with OS X until Sierra. I don’t have a pre-Sierra machine available to confirm, but that seems right. While writing the post, I tried to link to an Apple-hosted man page for ssh-copy-id and found that there wasn’t one, even though running man ssh-copy-id brings one up. I suspect this is related to Apple relegation of man pages to the “legacy” section of its developer website. They’re probably not putting much effort into adding new man pages to the site.


  1. If you do a lot of SSH work, the config file is also a great way to set up shortcuts to servers you commonly access. For example, my config file allows me to SSH into my iMac at work with a simple ssh work command. ↩︎


Screenshots with SnapSCP

This post covers my second screenshot utility, SnapSCP. It differs from the SnapClip in that it doesn’t put the image on the clipboard, it uploads it to the server that runs this blog and puts a link to the image on the clipboard so I can embed it in a post.

Because I want this post to be self-contained, a lot of the description from the SnapClip post will be repeated. Sorry about that, but I think it’s easier to work through code when all the explanation is in one place.

SnapSCP is modeled on the builtin macOS ⇧⌘4 keyboard shortcut for taking a screenshot of a portion of the screen. (It uses macOS’s screencapture command.) Like ⇧⌘4, it starts by changing your pointer to a crosshair, which you can drag across the screen to select an arbitrary rectangle for capturing.

Screenshot with cursor

Alternately, if you want to select an entire window, tap the spacebar. The pointer will change to a camera, and whatever window the camera is over will turn blue, indicating that it will be what’s captured when you click.

Screenshot with window selection

It’s at this point that SnapSCP diverges from ⇧⌘4. A window pops up on your screen with a editable text field and a checkbox.

SnapSCP

The text field is a description of the image and will form part of the file name. The complete file name will be what you enter in the field prefixed by today’s date in yyyymmdd format and a hyphen and suffixed by .png. For example, the image above is saved on the server as

20170227-SnapSCP.png

and the HTML link to it, which SnapSCP puts on the clipboard, is

<img class="ss"
src="http://leancrew.com/all-this/images2017/20170227-SnapSCP.png"
alt="SnapSCP" title="SnapSCP" />

Note that the “base” name of the file gets used for both the alt and title attributes. The alt attribute is for accessibility, and the title attribute is what pops up in your browser if you hover your mouse pointer over the image.

The class attribute, ss, is defined in ANIAT’s CSS file to center the image and limit its maximum width so it doesn’t spill out into the sidebar over to the right. If you want to use SnapSCP for yourself, you’ll need to adjust some of this to fit your situation.

You can use spaces in the name field. The link for the top image is

<img class="ss"
src="http://leancrew.com/all-this/images2017/20170225-Screenshot%20with%20window%20selection.png"
alt="Screenshot with window selection"
title="Screenshot with window selection" />

The alt and title attributes have the spaces, and the src attribute replaces them with %20 in accordance with the standard rules for URL encoding.

You might be wondering why, since I write my posts in Markdown, I use full-blown HTML for embedded images. Mainly, it’s because I’ve never liked Markdown’s “bang syntax” for images. I don’t find

![SnapSCP](http://leancrew.com/all-this/images2017/20170227-SnapSCP.png "SnapSCP") {.ss}

easier to read or to type than the HTML equivalent.1

If you select “Background border”, SnapSCP will put a blue border around the screenshot, just like you see in the screenshot above. Although this option appears for both arbitrary rectangle and window screenshots, it’s intended to be used only for the latter.

The blue background border is my attempt to strike a happy medium between the two types of window screenshots ⇧⌘4 and screencapture can produce: a window with a big dropshadow border,

Window with border

or a bare window with no edges,

Window with no border

The dropshadow border is way too big and lots of people in the Mac world hate it, but the edgeless window gives me vertigo; I feel as if I’ll walk off the edge and fall to my death. It doesn’t even look like a window. SnapSCP’s blue border (which is the same as SnapClip’s) is relatively narrow but gives the sense of a window with the Desktop behind it, which is what I’m looking for in a screenshot.

Window with SnapClip border

The color of the border is Solid Aqua Dark Blue, which is, by an amazing coincidence, the color I use for my Desktop.

Desktop color chooser

Now that what SnapSCP does and how it looks, let’s see how it’s built. It’s a Keyboard Maestro macro with a hot key of ⌃⇧4:

SnapSCP macro

The first step of the macro is the execution of this Python script:

python:
 1:  #!/usr/bin/python
 2:  
 3:  import Pashua
 4:  import tempfile
 5:  import Image
 6:  import sys, os, os.path
 7:  import subprocess
 8:  import urllib
 9:  from datetime import date
10:  
11:  # Parameters
12:  dstring = date.today().strftime('%Y%m%d')
13:  type = "png"
14:  localdir = os.environ['HOME'] + "/Pictures/Screenshots"
15:  tf, tfname = tempfile.mkstemp(suffix='.'+type, dir=localdir)
16:  bgcolor = (61, 101, 156)
17:  border = 16
18:  optipng = '/usr/local/bin/optipng'
19:  server = 'username@leancrew.com:path/to/images2017/'
20:  port = '9876'
21:  
22:  # Dialog box configuration
23:  conf = '''
24:  # Window properties
25:  *.title = Snapshot
26:  
27:  # File name text field properties
28:  fn.type = textfield
29:  fn.default = Snapshot
30:  fn.width = 264
31:  fn.x = 54
32:  fn.y = 40
33:  fnl.type = text
34:  fnl.default = Name:
35:  fnl.x = 0
36:  fnl.y = 42
37:  
38:  # Border checkbox properties
39:  bd.type = checkbox
40:  bd.label = Background border
41:  bd.x = 10
42:  bd.y = 5
43:  
44:  # Default button
45:  db.type = defaultbutton
46:  db.label = Save
47:  
48:  # Cancel button
49:  cb.type = cancelbutton
50:  '''
51:  
52:  # Capture a portion of the screen and save it to a temporary file.
53:  status = subprocess.call(["screencapture", "-io", "-t", type, tfname])
54:  
55:  # Exit if the user canceled the screencapture.
56:  if not status == 0:
57:    os.remove(tfname)
58:    print "Canceled"
59:    sys.exit()
60:  
61:  # Open the dialog box and get the input.
62:  dialog = Pashua.run(conf)
63:  if dialog['cb'] == '1':
64:    os.remove(tfname)
65:    print "Canceled"
66:    sys.exit()
67:  
68:  # Add a desktop background border if asked for.
69:  snap = Image.open(tfname)
70:  if dialog['bd'] == '1':
71:    # Make a solid-colored background bigger than the screenshot.
72:    snapsize = tuple([ x + 2*border for x in snap.size ])
73:    bg = Image.new('RGB', snapsize, bgcolor)
74:    bg.paste(snap, (border, border))
75:    bg.save(tfname)
76:  
77:  # Rename the temporary file using today's date (yyyymmdd) and the 
78:  # name provided by the user.
79:  name = dialog['fn'].strip()
80:  fname =  '{localdir}/{dstring}-{name}.{type}'.format(**locals())
81:  os.rename(tfname, fname)
82:  bname = os.path.basename(fname)
83:  
84:  # Optimize the PNG.
85:  subprocess.call([optipng, '-quiet', fname])
86:  
87:  # Upload the file via scp.
88:  subprocess.call(['scp', '-P', port, fname, server])
89:  
90:  # Generate a link to the uploaded image.
91:  bname = urllib.quote(bname)
92:  print '<img class="ss" src="http://leancrew.com/all-this/images2017/{bname}" alt="{name}" title="{name}" />'.format(**locals())

The script relies on two nonstandard libraries, i.e., two libraries that don’t come with macOS. The first is Pashua, which handles the SnapClip window and its controls. Pashua is an application written by Carsten Blüm and has libraries for several scripting languages, including Python.

The second nonstandard library is Pillow, which handles the addition of the border to the screenshot. Pillow is a fork of and drop-in replacement for PIL, the Python Imaging Library, a venerable piece of software that’s been around for about two decades.

In addition to Pashua and Pillow, SnapClip also uses OptiPNG, which recompresses PNG files in place losslessly. I typically get a 30–40% reduction in file size on screenshots, definitely enough to make it worthwhile. I installed OptiPNG through Homebrew, but there are other ways to get it.

Let’s go through the script.

Lines 11–20 set up a bunch of parameters that will be used later in the program. If you need to customize the script, this is probably where you’ll do it. Line 12 sets the date string for the file name prefix. Line 13 sets the image type. The screenshots are temporarily saved in a Screenshots folder in my Pictures folder, so Line 14 sets the variable localdir to point there. Line 15 then uses the tempfile library to create a secure temporary file for the screenshot. Line 16 sets the color of the border to match the RGB parameters of Solid Aqua Dark Blue, and Line 17 sets the width of the border to 16 pixels. Lines 18 gives the full path to the impbcopy commands. Finally, Line 19 sets the login name and path to the images folder on the server and Line 20 sets the server’s SSH port number[^port] as a string.

[port]: The standard port number for SSH is 22, but many web hosts—mine included—use a different port number. I don’t know if this is done more for obfuscation or to prevent collisions, but I know it’s fairly common.

Lines 23–50 set up the geometry of the Pashua window. I won’t go through every line here. It should be fairly obvious what each section does, and you can get the details from the Pashua documentation.

Line 53 runs screencapture via the subprocess module. It’s launched in interactive mode (-i), does not capture the dropshadow if capturing a window (-o), and saves the image in PNG format (-t type) to the temporary file (tfname).

Lines 56–59 stop the program and delete the temporary file if the user aborts the screenshot by hitting the Escape key or ⌘-Period. Because the output of this script is saved to a Keyboard Maestro variable, the print command in Line 57 signals a later step in the macro that the user canceled.

Line 62 runs Pashua to put up the SnapClip window and collect its input.

Lines 63–66 stop the program and delete the temporary file if the user clicks the Cancel button in the SnapClip window. Again, the print command in Line 64 is used to signal a later step in the macro.

Lines 69–75 check to see if the user asked for a Desktop background border and add it if necessary. Line 65–66 creates a solid-colored image that’s two border widths larger in each dimension than the raw screenshot. Lines 66–67 then paste the raw screenshot on top of the solid-colored image and save the result back to the temporary image file.

Lines 79–80 take the string typed into the text field (fn) and turns it into a full path to the permanent file name. Line 81 then renames the image file. Line 82 pulls out just the name of the file (no path components) and saves it to the variable bname for later use.

Line 84 uses optipng to reduce the size of the image file.

Line 87 then uploads the file to the server via the SCP protocol. For this command to work smoothly, the user must have set up SSH login credentials on both the local computer and the server. I’ll describe how that’s done in the next post in this series.

Finally, Line 91 URL-encodes the file name and Line 92 constructs and prints the <img> tag for the uploaded file.

Keyboard Maestro takes the output of the script and saves it to the ssurl variable. Under normal circumstances, ssurl will contain the <img> tag, but if the user canceled the process at either of the two steps described above, ssurl will contain the string “Canceled.”

The next step in the macro is a conditional. Keyboard Maestro tests the value of ssurl, and if it’s not “Canceled,” the value is put on the clipboard and the “Glass” sound is played to tell the user that the macro is complete. If the value of ssurl is “Canceled,” then the macro just stops.

Note that this macro leaves a copy of the screenshot in the “Screenshots” folder because I like having a local backup.

I find this macro very efficient at getting screenshots up onto the server and into my posts. What makes it work without a hassle, though, is the way I have my Macs set up to automatically handle the passing of SSH credentials to the server. That’s the topic of the final post in this series.


  1. Attributes aren’t part of John Gruber’s Markdown, but they are part of PHP Markdown Extra, which is—with a little tweaking—what I use on ANIAT. ↩︎


Screenshots with SnapClip

I’ve been writing and rewriting this screenshot script/workflow/thing since 2006. Every now and then I have an idea of how to improve it, and I inflict the update on you. Usually, instead of describing the whole thing, I refer you back to one or more of the earlier posts and only explain what’s new. That makes for convoluted reading (assuming you’re reading at all), so I decided to write three posts on the topic that cover everything:

I’ll put links to the latter two post in the list above once they’re written.

SnapClip is modeled on the builtin macOS ⇧⌘4 keyboard shortcut for taking a screenshot of a portion of the screen. (As we’ll see in a bit, it uses macOS’s screencapture command.) Like ⇧⌘4, it starts by changing your pointer to a crosshair, which you can drag across the screen to select an arbitrary rectangle for capturing.

Screenshot with cursor

Alternately, if you want to select an entire window, tap the spacebar. The pointer will change to a camera, and whatever window the camera is over will turn blue, indicating that it will be what’s captured when you click.

Screenshot with window selection

It’s at this point that SnapClip diverges from ⇧⌘4. A window pops up on your screen with a couple of choices.

SnapClip

If you select “Background border”, SnapClip will put a blue border around the screenshot, just like you see in the screenshot above. Although this option appears for both arbitrary rectangle and window screenshots, it’s intended to be used only for the latter.

The second option, “Save file to Desktop,” does exactly what you think. In some cases, you want to save the screenshot in addition to having it on your clipboard. Like ⇧⌘4, the filename is based on the date and time at which the screenshot was taken, but it isn’t as verbose. The format is simply

yyyymmdd-HHMMSS.png

The default action button is labeled “Clipboard” as a reminder that the purpose here is to get the screenshot onto the clipboard, even if a copy is also saved to a file.

The blue background border is my attempt to strike a happy medium between the two types of window screenshots ⇧⌘4 and screencapture can produce: a window with a big dropshadow border,

Window with border

or a bare window with no edges,

Window with no border

The dropshadow border is way too big and lots of people in the Mac world hate it, but the edgeless window gives me vertigo; I feel as if I’ll walk off the edge and fall to my death. It doesn’t even look like a window. SnapClip’s blue border is relatively narrow but gives the sense of a window with the Desktop behind it, which is what I’m looking for in a screenshot.

Window with SnapClip border

The color of the border is Solid Aqua Dark Blue, which is, by an amazing coincidence, the color I use for my Desktop.

Desktop color chooser

Now that we know how SnapClip works and what it looks like in action, let’s see how it’s built. It’s a Keyboard Maestro macro with a hot key of ⌃⌥⌘4:

SnapClip macro

The macro has only one step, but that step is a doozy. It’s this Python script:

python:
 1:  #!/usr/bin/env python
 2:  
 3:  import Pashua
 4:  import tempfile
 5:  from PIL import Image
 6:  import sys, os
 7:  import subprocess
 8:  import shutil
 9:  from datetime import datetime
10:  
11:  # Local parameters
12:  type = "png"
13:  localdir = os.environ['HOME'] + "/Pictures/Screenshots"
14:  tf, tfname = tempfile.mkstemp(suffix='.'+type, dir=localdir)
15:  bgcolor = (61, 101, 156)
16:  border = 16
17:  desktop = os.environ['HOME'] + "/Desktop/"
18:  fname = desktop + datetime.now().strftime("%Y%m%d-%H%M%S." + type)
19:  impbcopy = os.environ['HOME'] + '/Dropbox/bin/impbcopy'
20:  optipng = '/usr/local/bin/optipng'
21:  
22:  # Dialog box configuration
23:  conf = '''
24:  # Window properties
25:  *.title = Snapshot
26:  
27:  # Border checkbox properties
28:  bd.type = checkbox
29:  bd.label = Background border
30:  bd.x = 10
31:  bd.y = 60
32:  
33:  # Save file checkbox properties
34:  sf.type = checkbox
35:  sf.label = Save file to Desktop
36:  sf.x = 10
37:  sf.y = 35
38:  
39:  # Default button
40:  db.type = defaultbutton
41:  db.label = Clipboard
42:  
43:  # Cancel button
44:  cb.type = cancelbutton
45:  '''
46:  
47:  # Capture a portion of the screen and save it to a temporary file.
48:  status = subprocess.call(["screencapture", "-io", "-t", type, tfname])
49:  
50:  # Exit if the user canceled the screencapture.
51:  if not status == 0:
52:    os.remove(tfname)
53:    sys.exit()
54:  
55:  # Open the dialog box and get the input.
56:  dialog = Pashua.run(conf)
57:  if dialog['cb'] == '1':
58:    os.remove(tfname)
59:    sys.exit()
60:  
61:  # Add a desktop background border if asked for.
62:  snap = Image.open(tfname)
63:  if dialog['bd'] == '1':
64:    # Make a solid-colored background bigger than the screenshot.
65:    snapsize = tuple([ x + 2*border for x in snap.size ])
66:    bg = Image.new('RGB', snapsize, bgcolor)
67:    bg.paste(snap, (border, border))
68:    bg.save(tfname)
69:  
70:  # Optimize the file.
71:  subprocess.call([optipng, '-quiet', tfname])
72:  
73:  # Put the image on the clipboard.
74:  subprocess.call([impbcopy, tfname])
75:  
76:  # Save to Desktop if asked for.
77:  if dialog['sf'] == '1':
78:    shutil.copyfile(tfname, fname)
79:  
80:  # Delete the temporary file
81:  os.remove(tfname)

The first thing of note is that the script relies on two nonstandard libraries, i.e., two libraries that don’t come with macOS. The first is Pashua, which handles the SnapClip window and its controls. Pashua is an application written by Carsten Blüm and has libraries for several scripting languages, including Python.

The second nonstandard library is Pillow, which handles the addition of the border to the screenshot. Pillow is a fork of and drop-in replacement for PIL, the Python Imaging Library, a venerable piece of software that’s been around for about two decades.

In addition to Pashua and Pillow, SnapClip also makes use of two command line utilities that don’t come standard with macOS. Alec Jacobson’s impbcopy is a short Objective-C program that works sort of like pbcopy but for images. It reads the image data from a file and puts it on the clipboard. Alec gives the source code and instructions for compiling impbcopy. If you’d rather not get into compiling software, you can download a copy I compiled and try that. I make no guarantees, but it works for me.

The other command line utility, OptiPNG, recompresses PNG files in place losslessly. I typically get a 30–40% reduction in file size on screenshots, definitely enough to make it worthwhile. I installed OptiPNG through Homebrew, but there are other ways to get it.

With those preliminaries out of the way, let’s go through the script.

Lines 11–20 set up a bunch of parameters that will be used later in the program. If you need to customize the script, this is probably where you’ll do it. Line 12 set the image type. The screenshots are temporarily saved in a Screenshots folder in my Pictures folder, so Line 13 sets the variable localdir to point there. Line 14 then uses the tempfile library to create a secure temporary file for the screenshot. Line 15 sets the color of the border to match the RGB parameters of Solid Aqua Dark Blue, and Line 16 sets the width of the border to 16 pixels. Lines 17–18 then set the full path to the Desktop file where the image will be saved if the user so chooses. Lines 19–20 give the full paths to the impbcopy and optipng commands.

Lines 22–45 set up the geometry of the Pashua window. I won’t go through every line here. It should be fairly obvious what each section does, and you can get the details from the Pashua documentation.

Line 48 runs screencapture via the subprocess module. It’s launched in interactive mode (-i), does not capture the dropshadow if capturing a window (-o), and saves the image in PNG format (-t type) to the temporary file (tfname).

Lines 51–53 stop the program and delete the temporary file if the user abort the screenshot by hitting the Escape key or ⌘-Period.

Line 56 runs Pashua to put up the SnapClip window and collect its input.

Lines 57–59 stop the program and delete the temporary file if the user clicks the Cancel button in the SnapClip window.

Lines 61–68 check to see if the user asked for a Desktop background border and add it if necessary. Line 65–66 creates a solid-colored image that’s two border widths larger in each dimension than the raw screenshot. Lines 66–67 then paste the raw screenshot on top of the solid-colored image and save the result back to the temporary image file.

Line 71 uses optipng to reduce the size of the temporary image file, and Line 74 copies it to the clipboard with impbcopy.

Line 77 checks to see if the user asked to save the image. If so, Line 78 copies the temporary file to the Desktop with the yyyymmdd-HHMMSS.png name.

Finally, Line 81 deletes the temporary file.

I use SnapClip practically every day. On the fun side, it’s how I get screenshots into Twitter and Messages. On the business side, it’s how I copy Amazon receipts into my expense reports. Although I usually use it “bare,” without adding a background border and without saving a copy to the Desktop, having those additional features gives me a tool that handles almost all my screenshot needs. The only thing I commonly do that SnapClip doesn’t is upload images for showing here in the blog. For that, I use the very similar SnapSCP utility that I’ll describe in the next post.


Updating my invoicing email automation

Last month, I described a script I used to create

The email part of the script assumed I was using MailMate, which was true at the time, but isn’t anymore. I needed to rewrite the script for Apple Mail.

Fortunately, that script was originally written for Apple Mail; I’d converted it to MailMate three years ago. Even more fortunately, I’d kept the original script around—all it needed was a little touching up and the addition of the Reminders section (Reminders didn’t exist when I wrote the first version of this script).

I’m not going to explain the inner workings of the script. The various posts linked to in the paragraph above do that. But here’s what it does in a nutshell:

  1. It extracts the text from the invoice (a PDF file) using Derek Noonburg’s Xpdf.
  2. From the text, it gets the project name, project number, due date, and amount of the invoice.
  3. It looks up, in a text file “database” of all my projects, the name of the client.
  4. It looks up the email address of the client in the Contacts app.
  5. It generates an email to the client with all the pertinent information and the invoice PDF attached.
  6. It creates a new reminder for me to follow up on the invoice in seven weeks.1

Here’s what the generated email looks like:

Invoice email

The script doesn’t send the email because sometimes I like to add a sentence or two before sending.

OK, here’s the invoice script:

python:
  1:  #!/usr/bin/python
  2:  
  3:  import os
  4:  import os.path
  5:  import sys
  6:  from applescript import asrun
  7:  from subprocess import check_output
  8:  
  9:  # Templates for the subject and body of the message.
 10:  sTemplate = '{0}: Drang invoice {1}'
 11:  bTemplate = '''Attached is Drang Engineering invoice {0} for {1} covering\
 12:   recent services on the above-referenced project. Payment is due {2}.
 13:  
 14:  Thank you for using Drang. Please call if you have any questions or need\
 15:   further information.
 16:  
 17:  --
 18:  Dr. Drang
 19:  leancrew.com
 20:  
 21:  '''
 22:  
 23:  # AppleScript template for getting project contact info.
 24:  cScript = '''
 25:  tell application "Contacts"
 26:    set contact to item 1 of (every person whose name contains "{}")
 27:    return value of item 1 of (emails of contact)
 28:  end tell
 29:  '''
 30:  
 31:  # AppleScript template for composing email.
 32:  mScript = '''
 33:    tell application "Mail"
 34:      activate
 35:      set newMsg to make new outgoing message with properties {{subject:"{0}", content:"{1}", visible:true}}
 36:      tell the content of newMsg
 37:        make new attachment with properties {{file name:"{4}"}} at after last paragraph
 38:      end tell
 39:      tell newMsg
 40:        make new to recipient at end of to recipients with properties {{name:"{2}", address:"{3}"}}
 41:      end tell
 42:    end tell
 43:  '''
 44:  
 45:  # AppleScript template for setting a reminder.
 46:  rScript = '''
 47:  set rem to "Invoice {} on {}"
 48:  set rmdate to (current date) + 49*days
 49:  tell application "Reminders"
 50:    tell list "Invoices"
 51:      set minder to (make new reminder with properties {{name:rem, remind me date:rmdate}})
 52:    end tell
 53:    activate
 54:    show minder
 55:  end tell
 56:  '''
 57:  
 58:  # Establish the home directory for later paths.
 59:  home = os.environ['HOME']
 60:  
 61:  # Open the project list file and read it into a string.
 62:  pl = open("{}/Dropbox/pl".format(home)).readlines()
 63:  
 64:  # Get the selected invoice PDF names from the command line.
 65:  pdfs = sys.argv[1:]
 66:  
 67:  # Make a new mail message for each invoice.
 68:  for f in pdfs:
 69:    f = os.path.abspath(f)
 70:  
 71:    # Use pdftotext from the xpdf project (http://foolabs.com/xpdf) to extract
 72:    # the text from the PDF as a list of lines.
 73:    invText = check_output(['/usr/local/bin/pdftotext', '-layout', f, '-'])
 74:  
 75:    # Pluck out the project name, project number, invoice number, invoice amount,
 76:    # and due date.
 77:    for line in invText.split('\n'):
 78:      if 'Project:' in line:
 79:        parts = line.split(':')
 80:        name = parts[1].split('  ')[0].strip()
 81:        invoice = parts[2].lstrip()
 82:      if 'project number:' in line:
 83:        number = line.split(':')[1].split()[0].lstrip()
 84:      if 'Invoice Total:' in line:
 85:        parts = line.split(':')
 86:        amount = parts[1].split()[0].strip()
 87:        due = parts[2].lstrip()
 88:  
 89:    # Get the email address of the client.
 90:    try:
 91:      client = [x.split('|')[2] for x in pl if number in x.split('|')[1]][0]
 92:      email = asrun(cScript.format(client))
 93:    except:
 94:      client = ''
 95:      email = ''
 96:  
 97:    # Construct the subject and body.
 98:    subject = sTemplate.format(name, invoice)
 99:    body = bTemplate.format(invoice, amount, due)
100:  
101:    # Create a mail message with the subject, body, and attachment.
102:    asrun(mScript.format(subject, body, client, email, f))
103:    
104:    # Add a reminder to the Invoices list.
105:    asrun(rScript.format(invoice, name))

The only thing you might find weird is how I execute AppleScript through Python. That’s done through a very simple library I wrote when Hamish Sanderson’s appscript library bit the dust. My applescript library is described in this post.

In addition to issuing a command like

invoice inv12345.pdf

in the Terminal, I can also run the script by right-clicking on the invoice PDF and choosing this Service, created in Automator:

Automator workflow

If a client doesn’t pay within seven weeks, I get a reminder and need to send a followup email. That gets generated with this very similar script, called dun:

python:
 1:  #!/usr/bin/python
 2:  
 3:  import os
 4:  import os.path
 5:  import sys
 6:  from applescript import asrun
 7:  from subprocess import check_output
 8:  from datetime import datetime, timedelta, date
 9:  
10:  # Templates for the subject and body of the message.
11:  sTemplate = '''{0}: Drang invoice {1}'''
12:  bTemplate = '''The attached invoice, Drang {0} for {1},\
13:   is still outstanding and is now {2} days old. Whatever you can\
14:   do to get it paid would be appreciated.
15:  
16:  Thank you for your attention. Please call if you have any questions or need\
17:   further information.
18:  
19:  --
20:  Dr. Drang
21:  leancrew.com
22:  
23:  '''
24:  
25:  # AppleScript template for getting project contact info.
26:  cScript = '''
27:  tell application "Contacts"
28:    set contact to item 1 of (every person whose name contains "{}")
29:    return value of item 1 of (emails of contact)
30:  end tell
31:  '''
32:  
33:  # AppleScript template for composing email.
34:  mScript = '''
35:    tell application "Mail"
36:      activate
37:      set newMsg to make new outgoing message with properties {{subject:"{0}", content:"{1}", visible:true}}
38:      tell the content of newMsg
39:        make new attachment with properties {{file name:"{4}"}} at after last paragraph
40:      end tell
41:      tell newMsg
42:        make new to recipient at end of to recipients with properties {{name:"{2}", address:"{3}"}}
43:      end tell
44:    end tell
45:  '''
46:  
47:  # Establish the home directory for later paths.
48:  home = os.environ['HOME']
49:  
50:  # Open the project list file and read it into a string.
51:  pl = open("{}/Dropbox/pl".format(home)).readlines()
52:  
53:  # Get the selected invoice PDF names from the command line.
54:  pdfs = sys.argv[1:]
55:  
56:  # Make a new mail message for each invoice.
57:  for f in pdfs:
58:    f = os.path.abspath(f)
59:  
60:    # Use pdftotext from the xpdf project (http://foolabs.com/xpdf) to extract
61:    # the text from the PDF as a list of lines.
62:    invText = check_output(['/usr/local/bin/pdftotext', '-layout', f, '-'])
63:  
64:    # Pluck out the project name, project number, invoice number, invoice amount,
65:    # and due date from the upper portion of the first page.
66:    for line in invText.split('\n')[:20]:
67:      if 'Project:' in line:
68:        parts = line.split(':')
69:        name = parts[1].split('  ')[0].strip()
70:        invoice = parts[2].lstrip()
71:      if 'project number:' in line:
72:        number = line.split(':')[1].split()[0].lstrip()
73:      if 'Invoice Total:' in line:
74:        parts = line.split(':')
75:        amount = parts[1].split()[0].strip()
76:        due = parts[2].lstrip()
77:  
78:    # Get the email address of the client.
79:    try:
80:      client = [x.split('|')[2] for x in pl if number in x.split('|')[1]][0]
81:      email = asrun(cScript.format(client)).strip()
82:    except:
83:      client = ''
84:      email = ''
85:    addr = "{0} <{1}>".format(client, email)
86:  
87:    # Determine the age of the invoice.
88:    dueDate = datetime.strptime(due, '%B %d, %Y')
89:    dunDate = datetime.today()
90:    age = (dunDate - dueDate).days + 30     # invoices are net 30
91:  
92:    # Construct the subject and body.
93:    subject = sTemplate.format(name, invoice)
94:    body = bTemplate.format(invoice, amount, age)
95:    
96:    # Create a mail message with the subject, body, and attachment.
97:    asrun(mScript.format(subject, body, client, email, f))

There’s a bit of date calculation in the script in Lines 87–90, which allows me to point out (gently) the degree of delinquency.

Dunning email

There’s a Service that runs this script, too. It looks just like the one above but for the name of the script.

In last night’s post, I said Apple deserved credit for maintaining Mail’s plugin system, even though it seems antithetical to Apple’s current app philosophy. The same goes for Mail’s AppleScript support.


  1. When invoices get paid, I delete the associated reminder. This process hasn’t been automated… yet. ↩︎