JXA’s parenthesis paradox

I don’t like programming in AppleScript in general, but there is one construct that I’ve always liked. Because I’m not a computer scientist, I don’t know what to call it,1 but I can give an example that came up recently:

tell application "System Events" to get the name of every process whose visible is true

The System Events preamble doesn’t matter. It’s the

    _____ of _____ whose _____ is _____

that I find interesting. It’s compact, easy to understand, and, I would say, the most useful example of AppleScript’s “English-like” syntax.

In trying to write the JavaScript for Automation (JXA) equivalent of this, I learned (sort of) how parenthesis work in JXA when dealing with properties and filters by reading this excellent article by Christian Kirsch. I’m writing about it here before I forget.

Let’s start with something simpler:

tell application "System Events" to get every process

This returns a long list—over 100 items—which I’m going to condense and format as one item per line:

{application process "loginwindow" of application "System Events"
application process "ViewBridgeAuxiliary" of application "System Events"
application process "NotificationCenter" of application "System Events"
application process "Dictionary" of application "System Events"
application process "com.apple.WebKit.WebContent" of application "System Events"
application process "System Events" of application "System Events"}

The JXA equivalent is also pretty simple:

Application('System Events').processes();

It’s result is recognizable as the JavaScript equivalent to the AppleScript result:

[Application("System Events").applicationProcesses.byName("loginwindow")
Application("System Events").applicationProcesses.byName("ViewBridgeAuxiliary")
Application("System Events").applicationProcesses.byName("NotificationCenter")
Application("System Events").applicationProcesses.byName("Dictionary")
Application("System Events").applicationProcesses.byName("com.apple.WebKit.WebContent")
Application("System Events").applicationProcesses.byName("System Events")]

So far, so good. Now let’s add one piece of complexity. Instead of returning a complete description of each process, let’s get just their names. In AppleScript, that’s easy:

tell application "System Events" to get the name of every process

The the isn’t necessary—get name of every process would work just as well—but it adds to the readability. This returns a list of the same length but with simpler items.

"System Events"}

In JXA, we do this

Application('System Events').processes.name();

to get a JavaScript list of the same items:

"System Events"]

To me, the tricky part of this is how we removed the parentheses from processes before invoking names. Trying

Application('System Events').processes().name();

results in an error:

TypeError: Application('System Events').processes().name is not a function.

This is kind of misleading, as it suggests the problem is with name when the problem is really with processes(). We know from our previous example that

Application('System Events').processes();

returns a JavaScript array. It’s an array of processes, yes, but it’s still an array. And name() can only be applied to processes, not to an array of processes. So to get around this while still providing reasonable compact syntax, JXA provides processes without the parentheses as an object specifier to which name() can be applied on an item-by-item basis. Again, see Christian Kirsch’s article for a better explanation.

Now we go for the last step. In AppleScript, it’s

tell application "System Events" to get the name of every process ¬
    whose visible is true

which returns (not condensed)

"Script Editor"

(While it’s not the point of the post, this is the list of apps I happen to be running right now—the ones in the Dock. Invoking this line of AppleScript at some other time would return a different list.)

In JXA, the command is

Application('System Events').processes.whose(
    {visible: true}).name();

and the result is

"Script Editor"

Because the name of the process is what we want to return, name has to go last in the chain of calls, and it’s the one that gets the parentheses. We’re applying the whose filter to the processes, so it has to go after processes. The properties we’re filtering on (just visible in this case) are set up as a JavaScript object that’s passed as an argument to whose.

Suppose we had two criteria for our filter? Let’s say we want all the visible processes that start with the letter M (which will return just Mail and Messages). In AppleScript, it’s easy to add an extra clause:

tell application "System Events" to get the name of every process ¬
    whose visible is true and name begins with "M"

In JXA, the syntax gets nasty:

Application('System Events').processes.whose(
    {visible: true, name: {_beginsWith: "M"}}).name();

I think it’s obvious that we need to add a second piece to the object passed to whose, but that it should be

name: {_beginsWith: "M"}

is anything but obvious to me. The rule for constructing this sort of filter—where a function is applied to the property—is that its key is the property we’re going to filter on, and the value is yet another object whose key is the function to be applied to the property and whose value is the argument to that function.

But wait, there’s more! The name of the function comes from the AppleScript terminology, where multi-word functions are smashed together using camelCase, and the whole thing is preceded by an underscore. Thus,

name begins with "M"


name: {_beginsWith: "M"}

Simple, right?

The more I delve into JXA, the more I think I should stick with AppleScript and get more clever about using do shell script with Perl or sed for string processing.2

  1. Something to do with properties and filters, I imagine. 

  2. By the way, I know there are AppleScript libraries for string processing. Ages ago, I used Satimage’s Smile libraries, but as Apple began breaking longstanding pieces of automation, I got leery of using third-party add-ons. As far as I know, though, Smile’s text and regex functions are still working just fine. 

JavaScript vs. AppleScript

I’m years into my attempt to use JavaScript for Automation (JXA) more than AppleScript, but Apple and its crummy JXA documentation keeps getting in my way.

As an example of a case where I did use JXA, let’s look at a Keyboard Maestro macro I use quite often at work. I get lots of data from clients in Excel files that are formatted more for presentation than for analysis. I move the data over to Numbers (I hate working in Excel), clean it up into a simple tabular form, and export it as CSV for analysis with Pandas and other bits of Python.

In this process, it’s important that I keep the Numbers version of the table and the CSV version in sync. So I made a macro that saves the Numbers file and exports a CSV file at the same time. It’s just a single step with a short bit of JavaScript code.

Export as CSV

And here’s the code itself:

 1:  numbersApp = Application("Numbers");
 3:  // Get the top Numbers document and the path to its file
 4:  let topDoc = numbersApp.documents[0];
 5:  let topPathString = topDoc.file().toString()
 7:  // Make the path to the CSV file we're going to export to
 8:  let csvPath = Path(topPathString.replace(/\.numbers$/, ".csv"));
10:  // Save the Numbers file and export it as CSV
11:  numbersApp.save(topDoc);
12:  numbersApp.export(topDoc, {to: csvPath, as: "CSV"});

I wrote this in JXA instead of AppleScript because of Line 8. Although some text processing is easy in AppleScript, most of it is a pain in the ass. And simple regex substitutions fall into the “pain in the ass” category. If it’s just one text substitution, you can cheat with a little do shell script, like this:

 1:  tell application "Numbers"
 2:    -- Get the frontmost Numbers document and a path to its file
 3:    set topDoc to front document
 4:    set topPathString to file of topDoc as text
 6:    -- Make a path to the CSV file we're going to export to.
 7:    set csvPathString to do shell script "sed 's/\\.numbers$/.csv/' <<< " & quoted form of topPathString
 9:    -- Save the Numbers file and export it as a CSV
10:    save topDoc
11:    export topDoc to file csvPathString as CSV
12:  end tell

The combination of sed and a here string in Line 7 is basically the same as Line 8 of the JavaScript, but you have to be very careful with the quoting anytime you delve into do shell script. The JavaScript is cleaner code, so that’s what I went with.1

Still, there is one part of the JXA code I don’t like: Line 12:

numbersApp.export(topDoc, {to: csvPath, as: "CSV"});

I just find it hard to get the hang of this syntax. There are three arguments to this function. One of them is passed in as a regular variable, but the other two have to be rolled into an object. That’s just weird.

I might find it easier to get over my argument hangup if Apple put more effort into its JXA documentation. And by “more” effort I mean “any.” Here’s the AppleScript dictionary entry for the Numbers export command. It’s written more or less as you would write it in AppleScript.

Numbers export in AppleScript

Now here’s the same documentation for the JavaScript version:

Numbers export in JavaScript

Does this look like JavaScript to you? It doesn’t to me. You have to have seen example code using similar methods to know that the first argument is the document and the second is an object with key/value pairs for all the indented parameters. And while we’re told that export is a method, we’re not told what it’s a method of. My first instinct is that it should be a method of the document, not the app.

Basically, the JavaScript documentation is the AppleScript documentation with a few regex substitutions applied. Change nouns to objects, verbs to methods, close up certain spaces, apply a few capitalization rules, and sprinkle in some colons. Boom! An afternoon project for an intern, not something that should come out of the most valuable company in the world.

This, I think, is one reason why AppleScript hangs on. There are tons of people who know JavaScript from web development, but Apple’s half-assed JXA documentation keeps them from transferring that knowledge easily to Mac automation.

  1. I didn’t write the AppleScript code until I started putting together this post and felt the need for a counterexample. 

Repeated failure

A couple of days ago, Federico Viticci tweeted his wish list for a new or updated Reminders API and asked for other people for theirs. Stephen Hackett answered with this:

parameters for repeating tasks

Stephen’s simple request is exactly what I want, too. In fact, a way to script repeating tasks has been the top item on my wish list since well before Reminders existed.

How? Because Calendar—and iCal before it—has always had the same deficiency: nothing in its AppleScript dictionary (or now its Shortcuts action list) that addresses repeating items. Both Reminders and Calendar tease us by having repeating items available in the user interface

Reminders repeats

Calendar repeats

but no way to automate these capabilities directly.

If you’re working on a Mac, you can use Keyboard Maestro to simulate the various button clicks and menu selections needed to define a repeating item. I’ve done that for one particular type of repeating reminder that I create fairly often. But Apple shouldn’t make you rely on a third-party utility, especially when all the other attributes of calendar events and reminders are accessible through AppleScript and Shortcuts.1

So yes, Apple should add support for automating repeating items in Reminders and Calendar. But since Apple hasn’t bothered to do so in the 20 years that Calendar has been around, I’m not optimistic.

Update 4/24/2022 2:39 PM
Tobias (via Twitter) and George Green (via email) suggested creating an iCalendar file to automate recurring calendar events. Given that I used to use .ics files to get around the clumsiness of iCal’s date entry system, I should’ve thought to include a note about it in the post. Unfortunately, although Reminders tasks and Calendar events are related, I believe this trick is limited to Calendar events only.

John Flavin suggested the CalendarLib EC AppleScript library. It has a modify recurrence command that should do the trick.

CalendarLib EC modify recurrence command

But as with the iCalendar technique, it’s for Calendar events only, not Reminders.

Finally, Tobias (in another tweet) suggested using EventKit in a Swift script. Like CalendarLib EC, this would access the event database directly; unlike CalendarLib EC, it would work with Reminders as well as Calendar. This, along with Shortcuts’ support for Swift scripts, is making me think I should finally get around to learning Swift.

  1. Yes, you might be able to use UI Scripting, but UI Scripting is a tremendous pain in the ass—I have never found it to be as simple as Apple’s examples make it seem. 

Last man standing

Back in 2009, I wrote a post about various (mostly unsatisfying) ways of viewing manual (man) pages on the Mac. Many of the commands and apps in that post are either dead or abandoned, and some years ago I returned to a system that’s not much different from what I used when I was running Linux back in the early 2000s. I recently expanded my script to handle sections the same way the man command does.

At its most basic, the man command is invoked like this:

man [section] command

where command is the name of the command you want to learn about. Man pages are categorized in sections, and you can add the optional [section] if necessary to eliminate ambiguity.

The main problem with the man command is that it takes over your terminal window—you can’t refer to it while you’re constructing a command unless you open a second terminal window first or use the control-click trick. To get around this problem, I wrote a short shell script, called bman, that opens the man page in a new BBEdit window. Since I always have BBEdit running, this works just about as fast as the regular old man command.

I invoke it this way from the command line,

bman [section] command

using the same basic structure as man itself. For example, bman col will pop up this window:

Man page for col in BBEdit

If the man page is long, I can use all the familiar BBEdit search commands I use every day to find what I want.

Here’s the code for bman:

 1:  #!/bin/bash
 3:  # Interpret the arguments as command name and section. As with `man`,
 4:  # the section is optional and comes first if present.
 5:  if [[ $# -lt 2 ]]; then
 6:    cmd=${1}
 7:    sec=''
 8:  else
 9:    cmd=${2}
10:    sec=${1}
11:  fi
13:  # Get the formatted man page, filter out backspaces and convert tabs
14:  # to spaces, and open the text in a new BBEdit document.
15:  man $sec $cmd | col -bx | bbedit
17:  # Set the document name and cursor position within BBEdit
18:  osascript <<OSAEND
19:  tell application "BBEdit"
20:    set name of front document to "$cmd"
21:    select insertion point before character 1 of front document
22:  end tell

I think I’ve put enough comments in to explain what’s going on. What may be unfamiliar to you is the col command in the pipeline on Line 15. The man command outputs a lot of weird control characters to create bold and underlined words, and the purpose of col -b is to filter those out. Also, the -x option turns tabs into spaces, which I’ve found helpful in keeping the text aligned the way it should be.

Apple used to have all of the OS X man pages online.1 Accessing them from a web browser was certainly slower than getting them locally, but the online versions were nicely formatted with proportional fonts. Unfortunately, during one of Apple’s many fits of automation hatred, the online man pages were either deleted or moved to such a secure and undisclosed location that even Google can’t find them anymore.

If you use a text editor other than BBEdit, you can probably adjust the bman script to fit your needs. The lines that handle the arguments should stay the same, as should the Line 15 up to the last term in the pipeline. You’ll have to change the bbedit to whatever command invokes your editor. As for the AppleScript in Lines 18–23, you’re on your own.2

Update 3/27/2022 8:28 AM
Via Twitter, mikeymikey told me about the x-man-page:// URL scheme, which I had somehow never heard of before. But like the control-click trick, it brings up a new Terminal window with the man page, I came to the conclusion some time ago that I preferred to have my man pages displayed in BBEdit.

This morning’s email brought me messages from Christopher Waterman and Olav Brinkmann, both of whom told me that I could not only set the BBEdit window title directly from the bbedit command (which I had mentioned in the final footnote), but that there was also a way to scroll to the top. With Line 15 changed to

man $sec $cmd | col -bx | bbedit --view-top --clean -t $cmd

I can eliminate all the AppleScript that comes after.

Olav suggested the --clean option. It sets the state of the new document to “unmodified,” which means you can close the document without needing to confirm you don’t want to save it. Christopher suggested adding -m "unix-man-page", which sets the language for syntax coloring, but I haven’t found that it changes the way the page looks. I’ll look into it.

(Side note within a side note: Shouldn’t I be putting the variables in quotes? Yeah, that’s generally a good habit to get into, but sections and command names never have spaces in them, so I think I’m safe here. If you want to adopt this script and feel nervous about the lack of quotes, by all means add them.)

Christopher and Olav are so nice that neither of them pointed out the irony in my writing a post about viewing man pages while missing options that are clearly in the bbedit man page.

  1. That I’m calling them “OS X man pages” should tell you how long ago this was. 

  2. Many of you will know that I could have set the new document’s title by adding a -t option to the bbedit command on Line 15. I didn’t know that when I first wrote bman, and since I need AppleScript to put the cursor at the top of the document anyway, I’ve kept the title change in the AppleScript section.