From TextExpander to Keyboard Maestro… again

July 10, 2021 at 10:46 PM by Dr. Drang

After a good bit of thinking, I canceled my TextExpander subscription today. This is not the first time I’ve left TextExpander—I dropped it when Smile first adopted a subscription payment model about five years ago and stayed away even when Smile listened to the complaints and lowered the subscription price.

Eventually, though, I returned. TextExpander was the only realistic snippet solution for iOS and iPadOS, and as I found myself writing more and more on my iPad, I couldn’t live without it. Also, I like making temporary snippets to handle common phrases—like the name of a product or a company—that appear often in my writing as I work on a particular project but will never be used after the project is finished. TextExpander has a very efficient way of adding new snippets.

Things have changed over the past few months. My M1 MacBook Air has brought me back to the Mac in a big way. I no longer write anything longer than a text or an email on my iPad, and I don’t expect that to change. So cross-platform expansion isn’t as important as it once was.1 And although Smile seems to have fixed the crashing problem I was having a month or two ago, I’m still leery of TextExpander’s reliance on a bespoke syncing service.

So I’m back to using Keyboard Maestro as my snippet expansion tool. It works well, and I didn’t have to do too much work to switch over. In a rare display of forethought, I didn’t delete my snippet macros. I had merely disabled them when I started using TextExpander again—now I just had to re-enable them.

Keyboard Maestro snippet groups

Yes, there were some snippets from TextExpander that I’d made in the past few years that needed to be moved over to Keyboard Maestro, but that didn’t take much time. Some were even improved in the translation.

And I decided to tackle the one big advantage TextExpander had over Keyboard Maestro: the ability to make a new snippet quickly. By combining AppleScript with Keyboard Maestro itself, I now have a way to make a KM snippet out of what’s on the clipboard.

For example, let’s say I’m writing a report about products made by Mxyzptlk Industries. To make a snippet for that name, I copy it to the clipboard and invoke my new Make Temporary Snippet from Clipboard macro. That brings up this window,

Make Snippet input window

where I can define the trigger (I chose “;mi”) and adjust the expansion if necessary. After clicking OK, I have a new snippet in my Snippet - Temporary group.

Mxyzptlk snippet

Here’s the macro that does it:

Make Snippet macro

The first step asks the user for the snippet information, prepopulating the expansion field with the contents of the clipboard. The second step does all the real work, running this AppleScript:

applescript:
 1:  tell application "Keyboard Maestro Engine"
 2:    set trigger to getvariable "snippetTrigger"
 3:    set expansion to getvariable "snippetExpansion"
 4:  end tell
 5:  
 6:  set triggerXML to "<dict>
 7:  <key>MacroTriggerType</key>
 8:  <string>TypedString</string>
 9:  <key>SimulateDeletes</key>
10:  <true/>
11:  <key>TypedString</key>
12:  <string>" & trigger & "</string>
13:  </dict>"
14:  
15:  set expansionXML to "<dict>
16:  <key>Action</key>
17:  <string>ByTyping</string>
18:  <key>MacroActionType</key>
19:  <string>InsertText</string>
20:  <key>TargetApplication</key>
21:  <dict/>
22:  <key>TargetingType</key>
23:  <string>Front</string>
24:  <key>Text</key>
25:  <string>" & expansion & "</string>
26:  </dict>"
27:  
28:  tell application "Keyboard Maestro"
29:    tell macro group "Snippet - Temporary"
30:      set m to make new macro with properties {name:expansion}
31:      tell m
32:        make new trigger with properties {xml:triggerXML}
33:        make new action with properties {xml:expansionXML}
34:      end tell
35:    end tell
36:  end tell

Lines 1–4 pull in the values of the variables set during the previous macro step. Lines 6–26 define the XML text that defines what will become the new macro’s trigger and action. Finally, Lines 28–36 create a new macro in the Snippet - Temporary group and define it according to the XML. I took the overall structure of this section of the script from the Keyboard Maestro Wiki.

How did I know the XML format? I created a test macro by hand, exported it, and opened the Test.kmmacros file in BBEdit. From there, it was easy to see the parts that defined the trigger and the action. I did a little editing to accommodate the inclusion of the trigger and expansion variables and pasted the result into the script.

Making a Keyboard Maestro macro that runs an AppleScript that creates a new Keyboard Maestro macro is a lot of fun. More important, though, is that it brings KM up to TE’s level when it comes to making new snippets. Now I’m not making any compromises in using Keyboard Maestro for text expansion.

Update 07/11/2021 8:47 AM
Unsurprisingly, there is prior art in the “quick creation of a Keyboard Maestro snippet” field. This tweet from @DasPretzels led me to two macros that do basically what mine does: this one from Tom and this one from Peter Lewis himself. They both have a significant improvement on mine, in that they start with a Copy step, which eliminates one keystroke. So I added that to the top of my macro and also added a Delete Current System Keyboard step to the end. This leaves the clipboard in the same state it was before the macro was invoked.

  1. Even when I was still writing on my iPad, TextExpander had become less useful. At some point, fill-ins just stopped working, and I had to weaken several snippets to accommodate that loss of functionality. ↩

A stolen word count Quick Action

July 4, 2021 at 9:50 AM by Dr. Drang

I mentioned yesterday that installing NetNewsWire brought back some old RSS feeds that I had mistakenly dropped somewhere along the road from the old NetNewsWire to Google Reader to my homemade RSS reader. One of them is Erica Sadun’s blog. A lot of what she writes is too deep into Xcode and real programming for me to understand, but her more elementary stuff is at my level.1

A few months ago, she wrote about a simple, two-step word-counting Quick Action she built in Automator. It takes whatever text you have selected—in any app—and pops up a window with the word and character counts.2

Sadun Word count

The first step is this shell script:

echo `echo $1 | wc -w` words. `echo $1 | wc -c` characters.

And the second step is this AppleScript:

applescript:
on run {input, parameters}
  display dialog input as string buttons {"OK"}
end run

After copying these steps outright, I decided to make a few changes. First, I noticed that the results window didn’t have a title, and the OK button wasn’t set to be the default—tapping the Return key wouldn’t dismiss it. So I made a couple of additions to the AppleScript:

applescript:
on run {input, parameters}
  display dialog input as text buttons {"OK"} default button 1 with title "Word Count"
end run

As for the shell script, I felt a little nervous about passing a long stretch of text in as an argument, so I decided to change the script to this:

wc -wc | awk '{printf "%d words and %d characters", $1, $2}'

and have the selected text come in as standard input instead of as an argument. The output of wc -wc is a string with a pair of numbers separated by whitespace. Awk is perfect for handling text like this because it reads stdin automatically, splits it on whitespace, and assigns the resulting substrings to the variables $1, $2, $3, etc.

This worked well, and if I were smart I would’ve stopped there. But I thought about using this Quick Action on longer stretches of text and how it wouldn’t format the numbers with commas at the hundreds/thousands boundary. Large counts would be easier to read with commas.

As it happens, awk’s printf command inherits a formatting code from the system printf that puts commas at the appropriate places. The code, unfortunately, is %'d, and the single straight quotation mark is a pain in the ass when constructing a shell command. I’d like to be able to use this in the pipeline:

awk '{printf "%'d words and %'d characters", $1, $2}'

but that won’t work because the shell interprets all the single quotation marks as string delimiters—the ones I want to use as formatting codes never get to awk.

As is often the case in shell scripting, there’s a way around this, but it’s incredibly confusing and ugly:

awk '{printf "%'"'"'d words and %'"'"'d characters", $1, $2}'

I found this solution here, and it took me a while to figure out how it works. Basically, it’s concatenating five separate strings, which you can probably see better if I color-code them:

'{printf "%'"'"'d words and %'"'"'d characters", $1, $2}'

Two of the strings (the blue ones) are delimited by single quotes and contain a double quote. Two others (the yellow ones) are delimited by double quotes and consist entirely of a single quote. Also, it’s important that the variables $1 and $2 are in single quotes to keep the shell from interpreting them before awk gets a chance to. When all of these are put together, this is the command string that awk sees:

{printf "%'d words and %'d characters", $1, $2}

Whew!

And that wasn’t the end of it. Even though this awk command worked at the command line, it didn’t work in Automator because Quick Actions don’t run in my normal command line environment.

The problem was with the locale, which isn’t set in the environment under which Quick Actions run.3 Luckily, the same web page that showed me the multiple quoting trick also showed me how to set the environment variable. Ultimately, the shell script step in my Quick Action was4

wc -wc | LC_ALL="en_US" awk '{printf "%'"'"'d words and %'"'"'d characters", $1, $2}'

Did I say “ultimately”? That was premature. The Quick Action worked fine with this shell script, but those five consecutive quotation marks bothered me. I knew I’d have trouble understanding them later (even if I had this blog post to explain them). I also knew that Python has a straightforward way to format numbers with commas. So I threw away the awk command and substituted in a longer, but easier to read, chunk of Python:

wc -wc | python3 -c 'import sys
w, c = map(int, sys.stdin.read().split())
print(f"{w:,d} words and {c:,d} characters")'

Python is a modular language and doesn’t automatically parse its input, so I needed to do some extra work on the front end. The second line reads in the standard input, splits it on whitespace, and converts the resulting strings into integers. That set up the variables w and c to be interpreted by the f-string in the print command. This is distinctly longer than the awk solution, but it’s also distinctly clearer.5

Here’s a screenshot of the Quick Action in Automator:

Word Count Quick Action

And here’s an example of its output:

Word count window

I hope Ms. Sadun forgives me for what I did to her simple automation.

  1. In this way, she’s a lot like Michael Tsai. I have to skip past many of his posts because they’re way over my head, but I stay subscribed for the ones I can follow. ↩

  2. For me, the value of this Quick Action isn’t for counting words I’m writing; BBEdit will tell me that in the status bar at the bottom of the window. This is for counting words in other people’s writing on (mainly) web pages. ↩

  3. You may have run into similar problems in which the Quick Action environment doesn’t set the PATH to what you expect. ↩

  4. Strictly speaking, setting LC_ALL is overkill. Just setting LC_NUMERIC to “en_US” would be sufficient to get the comma separators working. ↩

  5. Note that this script works only in Python 3. Apple has recently (starting with Catalina?) supplied Python 3 in addition to Python 2, but you have to install the Command Line Developer Tools to get it. ↩

Back to NetNewsWire

July 3, 2021 at 10:56 PM by Dr. Drang

Ever since NetNewsWire came back from the dead, I’ve been thinking I’d eventually switch to it. I’ve been happy with my homemade RSS reading system, but it’s generally better to be in the hands of an expert, and you’d be hard-pressed to find someone more expert at RSS parsing than Brent Simmons. This morning, I switched over. It’s possible I’ll find something in NNW that’ll make me switch back, but it’s looking good so far.

I have a few unbreakable rules for an RSS reader:

  1. It must work on all my devices.
  2. It must sync between all my devices.
  3. It cannot force me to use a third-party RSS service to do the syncing.

Initially, the rejuvenated NNW broke all my rules; until recently, it broke Rule 3. But with the addition of iCloud syncing, it ticks all the boxes.

By the way, Rule 3 is not entirely due to my being a cheap bastard. I just don’t trust RSS services to stay in business. I keep thinking of Gabe Weatherhead’s wise assessment of the Google Reader apocalypse:

Bigger revelation: Google built a service that you configure with all your interests and biases. They couldn’t make it profitable.
— macdrifter (@macdrifter) Mar 13 2013 7:45 PM

This is not a review of NetNewsWire. If you’re a Mac user interested in RSS readers, you’ve read the reviews and know how fast and “Mac-assed” it is. You’re probably already using it.

But I do want to tell you how I got the list of feeds from my homemade system into NNW. First, the list is built directly into the Python code:

python:
jsonsubscriptions = [
    'http://leancrew.com/all-this/feed.json',
    'https://daringfireball.net/feeds/json',
    'https://sixcolors.com/feed.json',
    'https://www.robjwells.com/feed.json',
    'http://inessential.com/feed.json',
    'https://macstories.net/feed/json',
    'http://automationorchard.com/resources.json',
    'https://furbo.org/feed/json']

xmlsubscriptions = [
    'http://feedpress.me/512pixels',
    'http://alicublog.blogspot.com/feeds/posts/default',
    'http://bitsplitting.org/feed/',
    'https://kieranhealy.org/blog/index.xml',
    'http://brett.trpstra.net/brettterpstra',
    'http://www.libertypages.com/clarktech/?feed=rss2',
    'https://david-smith.org/atom.xml',
    'http://stratechery.com/feed/',
    'http://feeds.feedburner.com/IgnoreTheCode',
    'http://indiestack.com/feed/',
    'http://feeds.feedburner.com/theendeavour',
    'http://www.kungfugrippe.com/rss',
    'http://www.caseyliss.com/rss',
    'http://www.macdrifter.com/feeds/all.atom.xml',
    'http://macsparky.com/blog?format=rss',
    'http://www.marco.org/rss',
    'http://merrillmarkoe.com/feed',
    'http://mjtsai.com/blog/feed/',
    [and so on…]
    ]

As you can see, it’s really two lists, one for JSON feeds and one for regular RSS/Atom/XML feeds. I copied this code into a new BBEdit document, and used its Text▸Prefix/Suffix Lines… command to strip off the quotation marks and commas. Then a couple more deletions got me down to a simple list of feed URLs:

http://leancrew.com/all-this/feed.json
https://daringfireball.net/feeds/json
https://sixcolors.com/feed.json
https://www.robjwells.com/feed.json
http://inessential.com/feed.json
https://macstories.net/feed/json
http://automationorchard.com/resources.json
https://furbo.org/feed/json
http://feedpress.me/512pixels
http://alicublog.blogspot.com/feeds/posts/default
http://bitsplitting.org/feed/
https://kieranhealy.org/blog/index.xml
http://brett.trpstra.net/brettterpstra
http://www.libertypages.com/clarktech/?feed=rss2
https://david-smith.org/atom.xml
http://stratechery.com/feed/
http://feeds.feedburner.com/IgnoreTheCode
http://indiestack.com/feed/
http://feeds.feedburner.com/theendeavour
http://www.kungfugrippe.com/rss
http://www.caseyliss.com/rss
http://www.macdrifter.com/feeds/all.atom.xml
http://macsparky.com/blog?format=rss
http://www.marco.org/rss
http://merrillmarkoe.com/feed
http://mjtsai.com/blog/feed/
[and so on…]

Now it was time to convert this list to OPML format so NNW could import it. To see what kind of OPML NetNewsWire was expecting, I added two feeds “by hand” and exported them. The result was

xml:
<?xml version="1.0" encoding="UTF-8"?>
<!-- OPML generated by NetNewsWire -->
<opml version="1.1">
    <head>
        <title>Subscriptions-iCloud.opml</title>
    </head>
<body>
    <outline text="And Now It's All This" title="And Now It's All This" description="" type="rss" version="RSS" htmlUrl="https://leancrew.com/all-this/" xmlUrl="http://leancrew.com/all-this/feed.json"/>
    <outline text="Daring Fireball" title="Daring Fireball" description="" type="rss" version="RSS" htmlUrl="https://daringfireball.net/" xmlUrl="https://daringfireball.net/feeds/json"/>
    </body>
</opml>

I copied the top several lines and bottom two lines and pasted them directly into my new file (I did change the <title>). I guessed that I could get away with leaving out all the <outline> attributes except xmlUrl, so another go-round with Text▸Prefix/Suffix Lines…, this time Inserting rather than Removing, left me with

xml:
<?xml version="1.0" encoding="UTF-8"?>
<opml version="1.1">
<head>
    <title>old-subscriptions.opml<title>/>
</head>>
<body>
    <outline xmlUrl="http://leancrew.com/all-this/feed.json"/>
    <outline xmlUrl="https://daringfireball.net/feeds/json"/>
    <outline xmlUrl="https://sixcolors.com/feed.json"/>
    <outline xmlUrl="https://www.robjwells.com/feed.json"/>
    <outline xmlUrl="http://inessential.com/feed.json"/>
    <outline xmlUrl="https://macstories.net/feed/json"/>
    <outline xmlUrl="http://automationorchard.com/resources.json"/>
    <outline xmlUrl="https://furbo.org/feed/json"/>
    <outline xmlUrl="http://feedpress.me/512pixels"/>
    <outline xmlUrl="http://alicublog.blogspot.com/feeds/posts/default"/>
    <outline xmlUrl="http://bitsplitting.org/feed/"/>
    <outline xmlUrl="https://kieranhealy.org/blog/index.xml"/>
    <outline xmlUrl="http://brett.trpstra.net/brettterpstra"/>
    <outline xmlUrl="http://www.libertypages.com/clarktech/?feed=rss2"/>
    <outline xmlUrl="https://david-smith.org/atom.xml"/>
    <outline xmlUrl="http://stratechery.com/feed/"/>
    <outline xmlUrl="http://feeds.feedburner.com/IgnoreTheCode"/>
    <outline xmlUrl="http://indiestack.com/feed/"/>
    <outline xmlUrl="http://feeds.feedburner.com/theendeavour"/>
    <outline xmlUrl="http://www.kungfugrippe.com/rss"/>
    <outline xmlUrl="http://www.caseyliss.com/rss"/>
    <outline xmlUrl="http://www.macdrifter.com/feeds/all.atom.xml"/>
    <outline xmlUrl="http://macsparky.com/blog?format=rss"/>
    <outline xmlUrl="http://www.marco.org/rss"/>
    <outline xmlUrl="http://merrillmarkoe.com/feed"/>
    <outline xmlUrl="http://mjtsai.com/blog/feed/"/>
    [and so on…]
</body>
</opml>

My guess was right. NetNewsWire accepted my hastily made OPML file without complaint and figured out all the titles and other details on its own. I put in ten minutes of work at the most and had a fully functioning blog reader with over 40 subscriptions. A testament to BBEdit’s tooling and NetNewsWire’s adherence to Postel’s Law.

One last pleasant surprise: For reasons I don’t fully understand, when I first opened NetNewsWire, it populated the “On My Mac” account with several blogs I used to read. Presumably these subscriptions were saved in some preferences file in my Library folder, but I was doing this on my new MacBook Air, and I didn’t use Migration Assistant when setting it up. A similar thing happened in the “On My iPhone” and “On My iPad” accounts when I installed NNW on those devices.

Anyway, before deleting those subscriptions (remember, I want all of them synced through my iCloud account), I noticed a few blogs that I’d somehow lost track of over the years and which hadn’t made it into my homemade system. It was nice to see that they’re active, and I added them to my list of iCloud subscriptions.

Update Jul 4, 2021 11:16 PM
In a tweet, Brent Simmons told me that NetNewsWire populates the “On My Device” account with 16 subscriptions upon installation—and it’s the same 16 subscriptions for everyone. It just so happens that I have at some point subscribed to all 16 of them.

Hey, I sped up Apple Mail Rules

June 27, 2021 at 3:17 PM by Dr. Drang

When the Hey email software/service came out last summer, I had two instant negative reactions:

  1. A proprietary layer on top of an open system? No, thank you.
  2. The headline feature is screening email? Like I did with procmail back in the 90s? For $99? No, thank you.

Since then, a lot has been written about Hey.1 It’s certainly more than just message screening, but very little about it is compelling to me. I’m lucky in that email has never been a big burden to me.

But it did get me thinking about procmail and Rules, which is the Apple Mail equivalent. Making new rules and extending old ones is easy, but it isn’t fast. Too much clicking and selecting and typing.

A particular type of spam inspired me to figure out an easier way to use Rules. It’s mail that starts like this:

Hi, Dr. Drang!

I just read your great article on . We here at [whatever.com] are experts on [topic I don’t care about] and would love to write new articles for your site. Please get in touch…

What bothers me about these is not just the obvious insincerity, it’s the persistence of these bastards. Because if I don’t answer (which, of course, I never do), they’re back a week or two later, “checking in” to see what I think. And a week or two after that, they’re back again because maybe I’m not the right person for them to have contacted, and maybe I could pass their email on to someone else in the massive Leancrew empire.

For some reason, marking the first message as spam doesn’t prevent the later ones from appearing in my inbox. The only guaranteed way to block the followup emails from these pests is to make a Mail Rule for them. Like this:

Apple Mail Rules tab

Persistent Spam rule

But this is where my impatience and the fiddliness of Rules got in the way. I wanted to rid myself of this kind of email, but the click-click-click of adding a new sender to a rule like this took too long. The one thing that was definitely appealing about Hey was the speed with which you could block senders.

So I started thinking about how to automate Mail to simulate the ease of blocking senders with Hey. Because the various steps of adding a rule condition were in my head, I built a Keyboard Maestro macro that simulated all those steps:

Persistent Spam Keyboard Maestro macro

I’ve been using this macro for several months now, and it’s been quite reliable, but as I was writing this post, I began to have doubts. Could this be done better in AppleScript? Yes, and it was really easy:

applescript:
 1:  tell application "Mail"
 2:    -- Start by getting the sender's address and the message's account
 3:    set selectedMsgs to selected messages of first message viewer
 4:    set thisMsg to first item of selectedMsgs
 5:    set acct to account of mailbox of thisMsg
 6:    set spamAddr to extract address from sender of thisMsg
 7:    get acct
 8:    
 9:    -- Add that address to a new condition of the rule
10:    set psRule to rule "Persistent Spam"
11:    tell psRule
12:      set newCondition to make new rule condition at beginning of rule conditions
13:      tell newCondition
14:        set rule type to from header
15:        set qualifier to equal to value
16:        set expression to spamAddr
17:      end tell
18:    end tell
19:    
20:    -- Delete the message
21:    set mailbox of thisMsg to mailbox "Trash" of acct
22:    
23:  end tell

The most complicated part of this script is the center section in Lines 9–18. But even that’s pretty simple. It adds a new condition to the Persistent Spam rule and then sets the parameters for that rule. Boom.

I disabled the old Keyboard Maestro macro and created a new one: a single step that runs this AppleScript. Unlike the old macro, it runs instantaneously—no windows appearing and disappearing, no pauses.

I’m not sorry about the time I spent making the complicated Keyboard Maestro macro. It got me thinking about the logic of building and modifying a Mail Rule, and I probably wouldn’t have found the AppleScript commands for rules as easy to understand as I did if I hadn’t had that experience. I am embarrassed for falling into the most common automation trap: having the computer mimic what a person would do instead of working directly on the problem. But it’s not the first time I’ve made that mistake, and I’m sure it won’t be the last.

Regardless of how I got here, I now have a fast and simple way to do the one aspect of Hey I envied its users for.

  1. Some of it was even about Hey itself, instead of its developers. ↩

Older posts