Converting lists
July 1, 2023 at 12:07 PM by Dr. Drang
Sometimes I realize I’ve been doing something the slow way for a really long time, and it’s kind of embarrassing. A few days ago, I needed to take a column of numbers from a spreadsheet and convert it into a comma-separated list for pasting into Mathematica. I pasted the numbers into BBEdit and did a find/replace to turn
0
12
12
16
12
12
16
12
12
0
0
0
0
0
16
12
12
20
12
12
into
0, 12, 12, 16, 12, 12, 16, 12, 12, 0, 0, 0, 0, 0, 16, 12, 12, 20, 12, 12,
where I’d searched on linefeeds and replaced them with comma-space pairs. As I was deleting the trailing comma and space so I could ⌘A and ⌘C, it struck me how often I’ve done this. I haven’t converted lists like this for pasting into Mathematica that often, but I sure have done it a lot for making lists in Python, Perl, and even AppleScript. Why haven’t I automated this into a one-step process?
So I did, using Keyboard Maestro. The macro is to be run with a list of newline-separated numbers on the clipboard (we’ll deal with a list of strings later), and when the macro is invoked, the output will be a list of comma-separated numbers. There will be no trailing newline because I’ll be pasting the output between brackets (for Python or Perl) or braces (for AppleScript or Mathematica), and a trailing newline would have to be deleted.
You might be wondering whether a column of numbers copied from a spreadsheet fits the criterion of “newline-separated.” One of the great things about the Mac clipboard is that it can store its contents in multiple formats and paste the format that’s most appropriate. If you’re pasting into a spreadsheet, it will keep the cell structure, including any formulas; if you’re pasting into a plain text document, it will paste the visible text as a series of lines. As you’ll see in a minute, we’re going to use the pbpaste command, which
removes the data from the pasteboard and writes it to the standard output. It normally looks first for plain text data in the pasteboard and writes that to the standard output; if no plain text data is in the pasteboard it looks for Encapsulated PostScript; if no EPS is present it looks for Rich Text. If none of those types is present in the pasteboard, pbpaste produces no output.
Here’s what the ridiculously short macro looks like,
and you can download it if you can’t be bothered to type out that little shell script.1 I have it stored in my Global Macro Group, so the pasting works in any app.
The shell script is
pbpaste | perl -nle 's/,//g; push @a, $_; END{print join(", ", @a)}'
As we saw above, pbpaste outputs the list as several lines of text, which gets piped to perl. The Perl one-liner reads in each line of input (that’s the -n option), deletes its trailing newline (-l), deletes any commas that might be in it (s/,//g),2 and puts the resulting item at the end of a list (push). After all the lines have been read (END), the list is printed out with comma-space pairs between the items (join). So if I copy this column,
1,174
918
782
844
1,213
and invoke the macro with ⌃⌥⌘V, this will be pasted into the active app:
1174, 918, 782, 844, 1213
If you’re familiar with Perl’s -l option, you might be wondering why there’s no newline at the end of this output. Well, Perl does add a trailing newline, but Keyboard Maestro has an option in the gear menu that trims it before pasting.
To paste the output of the shell script, Keyboard Maestro first puts it on the clipboard. But I don’t want that; I want the clipboard the way it was before I ran the macro. Running this macro is like pasting, and pasting shouldn’t change the clipboard. This is where the second action comes in.
Keyboard Maestro keeps a clipboard history and has commands for manipulating that history. By deleting past clipboard item 0, I’ve deleted the item that Keyboard Maestro itself just put onto the clipboard, bringing it back to the state it was before I invoked the macro.
Sometimes the list I want to convert is a list of strings instead of just numbers. To tell the programming language that the items are strings, I have to enclose them in quotation marks. I’ll use double quotation marks, because all of the languages I use understand them, and I’ll escape any double quotation marks that happen to appear within the strings. Here’s the resulting macro,
which you can download here.
The shell script is only slightly more complicated:
pbpaste | perl -nle 's/"/\\"/g; push @a, $_; END {print q("), join(q(", "), @a), q(")}'
Here, we don’t delete commas, but we do escape double quotation marks. And the END section is more complicated because we need to enclose every item in quotation marks. I’m using Perl’s q operator to make the quoting of quotation marks easier to read.
If I put
Here are
some lines
with "really good"
text
on the clipboard and run this macro with ⌃⌥⇧⌘V, the output will be
"Here are", "some lines", "with \"really good\"", "text"
A few last notes:
- Both of these macros could be done using only “native” Keyboard Maestro actions. I didn’t do it that way because I’m much more familiar with Perl than I am with KM’s string and list manipulation actions. Similarly, I could have used
awkorsedin the shell script, but I’m more comfortable with Perl. - I could turn these two into a single macro by either asking the user which type of conversion they want or looking at the clipboard and seeing if it’s all numbers. But for now I’ll leave it as two macros and see how I like using them this way.
- As for why I didn’t think to write these macros—especially the first one—years ago, I suppose it was because I thought I was already saving time by doing the search-and-replace in BBEdit. It didn’t occur to me that I was doing the same search-and-replace again and again.
