My trouble with Hazel

I resisted getting Hazel for several years. Despite the praise it got from people I respect, it just didn’t seem like something I needed. My thinking, I guess, was that when I create or download files that need some action taken on them, I’m right there at the computer and can take that action myself. And maybe there will be times I don’t want that action taken. A system like Hazel wouldn’t be able to make decisions like that. Eventually, though, the siren call of deeper automation became irresistible. I bought Hazel 3–4 years ago, and over time I’ve set up a few rules that I think were truly useful. But Hazel’s unreliability at executing those rules has me ready to uninstall it.

Apart from some temporary rules I’ve used from time to time, I’ve had only three longterm uses for Hazel:

  1. To run my Southwest Airlines calendar event fixer, a script that edits the .ics files I get from SWA to include alarms that remind me to check in 24 hours before flights. This rule runs on certain .ics files saved to my Downloads folder and has never failed me.
  2. To copy ePub files that I make in Calibre to another folder so I can sync them with Marvin, the ereader I use.
  3. To add color tags to files with certain extensions so I can distinguish them in Files, which refuses to show file extensions except in the Show Info popup.

It’s the latter two rules that are unreliable, and I’m pretty sure I understand why. They don’t run on files in a particular folder, they run on files that are in some subfolder of a particular folder. Each tagging rule, for example, is really two rules, one that does the tagging,

Hazel rule for TeX files

and another that tells Hazel to run the tagging rule on all subfolders of my ~/projects folder,

Hazel rule for project subfolders

The reliability problem, I think, lies in the latter rule. While this is Noodlesoft’s standard guidance for processing subfolders, it seems more than coincidental that the only rules that fail are those that rely on this subfolder rule.

Unfortunately, I can’t get around this problem by setting up the tagging rules to apply to specific subfolders. At any given time, there are dozens of these subfolders, and new subfolders are made regularly as I work on new projects. Setting up new rules for these subfolders as I create them would be more work than just tagging the files by hand.

To get around Hazel’s unreliability for ePub files, I wrote a one-line shell script, epubs-update, that uses find and xargs to do the copying. I wrote about it before, but here it is again:


find ~/Dropbox/calibre/ -iname "*.epub" -print0 | xargs -0 -I file cp -n file ~/Dropbox/epubs/

When I started having reliability problems with the tagging rules, I wrote a similar script, color-tags:


find ~/projects/ -mtime -7 -iname "*.py" -print0 | xargs -0 tag --set Green
find ~/projects/ -mtime -7 -iname "*.tex" -print0 | xargs -0 tag --set Blue
find ~/projects/ -mtime -7 -iname "*.log" -print0 | xargs -0 tag --set Gray

This latter script uses JD Berry’s tag command, installed via Homebrew. It isn’t the most efficient script in the world—potentially setting tags on files that already have their tags set—but it does the job. I cut down on the inefficiency somewhat by including the -mtime -7 option. That limits the files found to just those modified in the past seven days.1

Since I’m always working at the command line when processing Python scripts and LaTeX files, running color-tags isn’t a burden—a Terminal window will always be open when I need to run it. And I suppose I could set up a launchd agent for running it periodically. We’ll see how I feel about it as I get more experience with it.

As for epubs-update, that’s clearly a command that needs to be run only after I add ePubs to Calibre. Probably the best way to launch it is through a Keyboard Maestro macro that’s active only when Calibre is.

The SWA calendar entry fixer doesn’t really need Hazel, either. Before I ever set up a Hazel rule for those .ics files, I was using a Service built with Automator to run it. I never deleted that Service, so it’s no big deal to return to it.

You could argue that since I own Hazel, it wouldn’t hurt to keep it active. But I see it as a waste of resources, even if those resources are minimal. And this is not a suggestion that you should stop using Hazel. If it works for you—and I’m sure it works for most of you—keep at it. But my three-year experiment with it is over.

  1. Why seven days? It was a guess at a decent compromise between completeness (where I wouldn’t include the -mtime option at all) and speed (the script takes over 8 seconds to run without the -mtime option). The purpose of tagging is to be able to distinguish between file types while working from my iPad, and this script is meant to tag files that I expect to see on my iPad. I seldom spend more than a week working on these types of files before moving at least once between Mac and iPad, so I suspect I’ll never need to tag older files. Again, this is a guess; I’ll need some time with it before I know if seven was the best choice. 

Replacing a bit of Transmit

Has it really been only two years since Panic announced the end of development for Transmit iOS? I guess it seems longer because the eventual death of Transmit has been weighing on my mind ever since.

Transmit still works, but it fits into the system less comfortably with each iOS update. And no true replacements for it have sprung up, which I guess confirms the logic of Panic’s decision to stop development.

It’s not as though there are no other SFTP apps for iOS.1 But they aren’t as easy to configure and they don’t provide the niceties that Transmit does. I suppose if they did, their developers would have run into the cost/benefit wall that Transmit did.

And when I talk about Transmit’s niceties, I’m not talking about its visual design. Transmit is a good-looking app, to be sure, but what makes it stand out—still—is how well it understands what the users of an SFTP app need to do. For example, one of my favorite features is that it knows how to handle different ways of addressing a file on a server. The Copy Path command gives you the standard Unix-style path to the file, and Copy URL translates that to your particular server configuration and also URL-encodes it.

Transmit Copy Path or URL

You want the former if you’re doing some internal manipulation but the latter if you’re going to embed a link to that file in a web page. Having both at your fingertips is the kind of thoughtful detail that makes an app not merely useable but useful.

Secure Shellfish, the latest app to try to replace Transmit on my devices, provides no such affordances. You can long-press on a file to get at its name (clumsily, and without the file extension) via the Rename feature, but there’s nothing in the popup menu that lets you do what’s so easy to do in Transmit.2

So if I want to use Shellfish—and I may be forced to use when iOS 14 rolls around3—I have to fill in the gap with Shortcuts. Luckily, it’s pretty simple, at least for my most common use of Transmit’s Copy URL command, which is getting the URL of an image file on the web server so I can include it in a blog post.

Here’s a shortcut meant to be called from the Share Sheet within Files while Shellfish is the active file provider:

0 Shellfish Image URL Step 00 This is only for images.
1 Shellfish Image URL Step 01 Assemble the URL. The Shortcut Input appears twice in this text but with different properties selected each time. First we extract the file name, then the file extension.
2 Shellfish Image URL Step 02 Encode spaces and special characters that might be in the image name.
3 Shellfish Image URL Step 03 Put it on the clipboard for later pasting.

It was easy to write because I save images to a particular folder (a new folder each year) on the server. That means most of the URL is given, and only the filename (with extension) has to be extracted. Shortcuts has a means of doing that and it knows how to encode the constructed URL before putting it on the clipboard.4 With this, I have a limited version of Transmit’s Copy URL.

Secure Shellfish is free to download, but the full app requires a $10 in-app purchase, which is what I paid for Transmit. This also confirms Panic’s decision, because it now requires both $10 and the unpaid services of an amateur programmer to get a limited version of the Transmit’s functionality.

Update Feb 3, 2020 7:18 AM
Reader Simon Gredal sent me an important correction last night.

When you configure a server in Secure Shellfish, you’re given an opportunity to set an initial directory (which I’ve obscured in the screenshot below) and to associate a website URL fragment with that directory.

Shellfish configuration

This is very much like the system Transmit uses. I just missed it entirely when configuring Shellfish to connect to my web host.

With this configuration set, if you long-press on an item accessed via Shellfish in Files, a menu pops up.

Shellfish long-press menu

Selecting Share from this menu gives me access to my Shellfish Image URL shortcut. But selecting Manage brings up this window:

Shellfish Manage window

Here we see the encoded URL for the file. Tapping the Share icon next to it brings up the Share Sheet, from which I can choose Copy to put the URL on the clipboard.

This is one step more than using my shortcut, but it is possible, and it’s more flexible than my shortcut because it works for any type of file. So although it takes more tapping to get at, Shellfish does provide the affordance I wanted.

My apologies to the Shellfish folks for missing this feature in the original post, and my thanks to Simon for setting me straight.

  1. I know these are commonly called “FTP apps,” but does anyone really use FTP anymore? 

  2. I’m perfectly willing to believe this is more the fault of the Files app, which Secure Shellfish has to fit inside, than of Shellfish itself, but assigning blame properly doesn’t solve the problem. 

  3. A warning: when an iOS update finally does kill Transmit, I intend write a post entitled Sic Transmit gloria mundi, and I don’t want anyone stealing that title. 

  4. I should mention that Shellfish provides Shortcuts with an Upload File action that can be configured to return the URL of the uploaded file. I’m not using it because here I need to get at the URL of a file that’s already on the server. 

2019 Apple scorecard

Jason Snell’s annual Apple report card came out yesterday, and it was, as usual, interesting to hear both the harmony and discord among the choir of Apple watchers.

The questionnaires were sent out late last year and were compiled earlier in January. If you’ve been following John Gruber’s recent posts on the deficiencies of the iPad UI, you’ll find what may have been the seeds of those posts in his comments to Jason. I’ve been trying to sum up my own complicated thoughts on the iPad for some time. Gruber’s posts jibe with a lot of my thinking, but my feelings are more than just “me, too.” I hope to have something coherent written soon.

Back to the report card: There is a small but admirable thing Jason does with his graph of overall scores. I didn’t notice it until this year, but it’s been the case for as long as he’s been doing the report card.

Snell scorecard

Image from Six Colors.

Compare the length of the bar for HomeKit, which has a score of 2.8, to that of Wearables, which has a score of 4.6. You’ll see that the latter is twice the length of the former, even though 4.6 isn’t twice 2.8.

This is correct because Jason doesn’t ask for rankings between zero and five, he asks for rankings between one and five. Therefore, the baseline at the left edge of the scorecard is one, not zero, and the Wearables bar should be twice the length of the HomeKit line because [4.6 - 1 = 2(2.8 - 1)]. It’s attention to the little things that make for good graphs.

One more thing: For consistency, Jason has maintained the same ordering for the categories since the 2016 report card, but there are lots of ways to look at the results. Here, I’ve clipped out and rearranged just the “product” categories:

Snell product scores rearranged

Services, of course, isn’t a product, but Apple treats it as such in its quarterly reports, so I’ve followed its lead.

This seems right, doesn’t it? For “Wearables,” you can substitute “AirPods,” and it’s clearly the outstanding current product. Then there’s kind of a stew of second-tier quality, where we all might have our own slight reordering, but there’s certainly consensus that the iPhone should be at or near the top of this group and the Mac should be at or near the bottom. And then there’s the AppleTV.

Lines and Drafts

If you’re a Drafts user, you’ve probably noticed this oddity in how it displays—or, more accurately, doesn’t display—the trailing final line number for drafts with a certain type of content. Here, for example, are two ever-so-slightly different versions of a draft:

Drafts lines

Where is the extra character in the version on the right? It’s a trailing linefeed character after the fifth line. If I had timed the screenshots to show the blinking cursor at the end of the file, you’d see that the end of the file on the left is just after the final e, while the end of the file on the right is at the start of the sixth line. But Drafts doesn’t consider that sixth proto-line worthy of numbering.1

Generally speaking, I agree with Greg Pierce’s decision on this. A linefeed character at the end of a file doesn’t really mean there’s another line there. But there are times when I wish the line number was there. For example, when I’m writing a blog post and there are Markdown reference links at the bottom of the draft, I sometimes look at the last number and wonder if there’s a linefeed at the end of it. If there isn’t, that means I’ve screwed something up and the next reference link I add (which is done through this Drafts action) will be tacked onto the end of the last line instead of being on a line of its own.

Draft blog post

Another aspect of how Drafts handles lines came up last night and is the real reason for this post. I was trying to work out an answer to John Catalano’s question about incrementing the number of leading hashes (to denote a Markdown header of a particular level) to the current line. I misread the question and wrote a JavaScript action that would increment the number of hashes in all the lines of the current selection (which would reduce to the cursor’s current line if there’s no selection). When I looked back at the question this morning, I saw my mistake—and that Greg had given John a better answer—but I kept thinking about my solution and how part of it could be reused to handle many situations in which a series of lines need to be processed.

The script I ended up with was an improvement on the one you’ll see in my forum answer, but it does basically the same thing: increment the number of hashes in the selected lines until it gets to three hashes, at which point it resets to zero. Here’s the script:

 1:  // Extend selection to full lines without final trailing newline
 2:  var [start, len] = editor.getSelectedLineRange();
 3:  var full = editor.getTextInRange(start, len);
 4:  full = full.replace(/\n$/, "");
 5:  len = full.length;
 6:  editor.setSelectedRange(start, len);
 7:  var lines = full.split("\n");
 9:  // Adjust the leading hashes for each line in turn
10:  // Skip empty lines
11:  for (var i=0; i<lines.length; i++) {
12:    if (lines[i].length == 0) {
13:      continue;
14:    } else if (lines[i].substring(0, 4) == "### ") {
15:      lines[i] = lines[i].substring(4);
16:    } else if (lines[i].substring(0, 1) == "#") {
17:      lines[i] = "#" + lines[i];
18:    } else {
19:      lines[i] = "# " + lines[i];
20:    }
21:  }
23:  // Rejoin the lines and replace the selected text in the editor
24:  var s = lines.join("\n");
25:  editor.setSelectedText(s);

In a nutshell, the first stanza extends the selection to encompass all the lines that are at least partially selected and then splits that into an array, with each element consisting of one line of text (with no trailing linefeed). The second stanza operates on each of those lines in turn. The third stanza then glues the lines back together with linefeed characters and replaces the selected lines in the editor with the glued-together string.

The part I really care about—the part I think is reusable—is the first stanza, Lines 1–7. There are many situations in which appending, prepending, or otherwise transforming a series of lines is useful, and this section of code is a nice setup for that kind of operation.

It starts by calling the getSelectedLineRange function of the global editor object on Line 2. According to the documentation, this

Get[s] the current selected text range extended to the beginning and end of the lines it encompasses.

The value returned is a two-element array consisting of the starting character position and the length of the text range. The key to getSelectedLineRange, and what makes it different from getSelectedRange, is that the starting character position is the beginning of the line containing the start of the selection and the range extends to the end of the line containing the end of the selection.

If the action is called with this selection

Drafts selection

start will be 11 (there are 11 characters [remember the linefeed] before the S at the start of the second line) and len will be 35 (12 from the second line, 11 from the third, and 12 from the fourth). Note that the end of the fourth line is just after the linefeed that separates it from the fifth.

Given that all the other linefeeds in this range are included, it’s consistent to include the one at the end of the last line in the range. As we’ll see, though, there’s a situation for which the last character in the range won’t be a linefeed, and we’ll have to code defensively to cover that situation.

Line 3 uses getTextInRange to grab the range we just described and put it in the variable full.

Looking ahead to Line 7, we see that we’re going to use split("\n") to convert full into the array lines. Because we don’t want lines to include an empty element at the end, we should strip off any final linefeed from full before running split. That’s what the replace in Line 4 deals with. It insures that the last character of full is removed if it’s a linefeed.

Why can’t we just remove the last character of full without checking for what it is? When won’t full have a linefeed at the end? That’s where the case we looked at back at the beginning of this post comes in. If the last line of the draft doesn’t end with a linefeed, and that line is part of the selection, then full won’t end with a linefeed.

Line 5 adjusts the value of len to account for the (usually) new length of full, and Line 6 selects all the text we’re going to be transforming.2 Finally, Line 7 splits full into lines.

After the operations on lines is done, we use join("\n") to put lines back together into a single string (Line 24) and replace the selection in the editor with that string (Line 25).

If you’re thinking this is one of those posts I write mostly for myself, you’re right. With luck, writing this will help me remember to use this split-operate-rejoin process then next time I need to process a series of lines in Drafts. If it helps you too, that’s a bonus.

  1. In the Editor settings, Drafts calls the little numbers in the left margin paragraph numbers instead of line numbers, which makes sense because of the way word wrapping works. But if you’re writing code or other structured text, it’s better to think of them as line numbers. 

  2. This selection isn’t strictly necessary, but it makes the later substitution of the transformed text into the editor easy, and I like to highlight the text being worked on.