And now it’s all this

I just said what I said and it was wrong
Or was taken wrong

Embedding JavaScript in Shortcuts with Scriptable

January 11, 2020 at 11:08 PM by Dr. Drang

Earlier this week, Scriptable added a feature that makes executing simple JavaScript functions within Shortcuts distinctly easier. I decided to show how it works by writing yet another version of my ROT13 shortcut.

The new action is “Run Inline Script,” and it does pretty much what you’d expect: it allows you to write JavaScript within a Shortcuts action that gets executed by Scriptable.

The documentation for the new action (which you can see within Shortcuts by tapping the ⓘ, is this:

Runs a script in Scriptable. The parameter can be accessed using args.shortcutParameter. This parameter can be a text, list, dictionary or a file. When passing a file as input the action will attempt to read the file as JSON or a plain text. If the file cannot be read as JSON or plain text, a path to the file will be passed instead. Use args.plainTexts, args.images, args.urls and args.fileURLs to read the other input parameters. When passing large images and files as input, the script may fail due to memory constraints. In that case you should enable “Run In App”. Use JavaScript’s “return” keyword or Script.setShortcutOutput() to output a value. In case you don’t output a value, add Script.complete() to indicate that your script have finished running.

This is correct, but as we’ll see in a minute, there’s an even easier way to pass information into an inline script.

Here’s my new JavaScript-based ROT13 shortcut, called “ROT13 Scriptable Inline.” If you compare it to what I wrote before, you’ll see that the only change is in Step 10, which is now “Run Inline Script” instead of “Run Script.”

Step	Action	Comment
0		This shortcut takes the URL of the tweet.
1		Do an HTTP GET on the tweet URL.
2		Treat the data from the GET as HTML.
3		Find the meta tag that has the text of the tweet.
4		Extract the text of the tweet from the match; this will include HTML entities.
5		Because the next step turns newline characters into spaces, protect them by turning them into runs of 10 equals signs (we’ll turn them back later).
6		Convert all the remaining HTML entities into characters by converting the HTML to rich text.
7		Convert the rich text into plain text so Scriptable can handle it.
8		Do the ROT13 conversion via Scriptable; the JavaScript itself is shown below. By turning off the “Show When Run” switch (it’s on by default), the script runs immediately; otherwise, there’s a pause for the user to tap a button to allow the script to run.
9		It’s impossible to see, but there’s a newline in this text box.
10		Convert the runs of equals signs we created in Step 5 back into newlines.
11		Display the converted text in an alert.

(I’m trying out a new way to describe Shortcuts—a table of the actions with a column of step numbers to the left and a column of commentary to the right. It’s a pain in the ass to make, but I think it gives the reader a better way to understand what’s going on. If people find it useful, I’ll try to figure out a way to automate the creation of the screenshots of the individual actions and their assembly into a table.)

The JavaScript in Step 10, which you can’t see all of in the screenshot, is this:

javascript:
 1:  // Create a dictionary that maps characters to their ROT13 transform
 2:  var pchars = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ";
 3:  var tchars = "nopqrstuvwxyzabcdefghijklmNOPQRSTUVWXYZABCDEFGHIJKLM";
 4:  var trans = {};
 5:  for (var i=0; i<pchars.length; i++) {
 6:    trans[pchars[i]] = tchars[i];
 7:  }
 8:  
 9:  // Get the text from a Shortcuts variable, convert it, and return it
10:  let s = '❑Text';
11:  s = s.split('').map(x =>  trans[x] ? trans[x] : x).join('');
12:  return s;

The ROT13 conversion is done using the same code we used earlier. The difference between this and the standalone script is in the simpler input on Line 10 and output on Line 12.

For input, we don’t have to pass a parameter and use the args.shortcutParameter construct. Instead, we can assign a Shortcuts magic variable directly to a JavaScript variable. This is not mentioned in the documentation we looked at above, but I think it’s easier to understand, and it makes the JavaScript more integrated with Shortcuts. The ❑Text on the right-hand side of Line 10 is my plain text attempt to show the magic variable (which you can see in the screenshot); it’s the one bound to the output text from Step 8.

Similarly, we don’t have to use Script.setShortcutOutput() for output—we can just return the output value and later steps in the shortcut can use it (this is described in the documentation). If you need to return something more complex than a string of text, you can generate JSON by doing something like

javascript:
return {item1:"Some text", item2:42, item3:[1,2,3,4]};

and then using the “Get Dictionary from Input” action.

Fundamentally, what Scriptable’s new inline action does is address the problem John Gruber complained about in his tweet:

If you have Scriptable, you can now do, you know, actual scripting in Shortcuts.

---

ROT13: Odyssey Two

December 29, 2019 at 2:50 PM by Dr. Drang

If you had told me ten years ago that I would spend the waning days of the decade discussing, programming, and writing about ROT13, I would’ve thought you’d gone soft in the head. But here I am, with a handful of ROT13 shortcuts on my iPhone and iPads, preparing to write about the ins and outs of doing a ridiculously simple task on ridiculously powerful hardware.

It started as described in my last post: a few tweets encoded in ROT13, a couple of decoding shortcuts written by Matt Cassinelli and Jason Snell, and my attempt to combine and tweak those shortcuts to handle a few edge cases. After the post went up (and Jason linked to it with his own post), there was a small Twitter discussion about other ways the shortcut could’ve been written. That discussion exploded when John Gruber entered the room and dropped this:

In other words, “Dear boy, why don’t you just try acting?”

Apart from the replies arguing about what is and isn’t scripting, there was some good input on how to embed calls to JavaScript and Python in shortcuts and how to make Shortcuts itself do things that “real” scripting languages can do. I fiddled around with some of these ideas and ended up with a small number of shortcuts that all do basically the same thing as the one I describe in the last post but with different ways of handling the ROT13 transformation.

The point of this exercise was not ROT13, it was expanding my toolbox of techniques so I’d know about more options to solve real problems in Shortcuts. I (and, I suspect, you) have done the same thing in other contexts when learning new techniques in Perl, Python, Scheme, JavaScript etc. It’s how we grow as programmers, even amateur ones.

The shortcuts I built fall into three basic categories:

1. Shortcuts that call a prebuilt ROT13 action provided by a third-party app.
2. Shortcuts that call a ROT13 script written by me in a third-party scripting app.
3. Shortcuts that do the ROT13 transformation entirely within Shortcuts itself.

The shortcut I built in the last post is an example of the first category. You can also use a free app called ESSE, which also supplies an action to Shortcuts from which you can choose from a variety of text transformations.¹ Comparing the two shortcuts, you can see that they are identical except for the one step that does the ROT13 transformation.

There are other text transform apps in the App Store. Any of them that include ROT13 and provide a transformation action to Shortcuts could be used exactly the same way.

In this category, Shortcuts is acting as a glue language: preparing data and calling on another app to process it. This sort of behavior is a common feature of scripting languages like AppleScript, Perl, Python, Ruby, and the shell.

In the second category are these two shortcuts: the one on the left calls out to Scriptable to do the ROT13 transformation, and the one on the right calls out to Pythonista.

Scriptable is a good modern iOS citizen in that it includes commands that allow it to accept data from and return data to Shortcuts. The ROT13 script that’s called by the shortcut is this:

javascript:
 1:  // Create a dictionary that maps characters to their ROT13 transform
 2:  var pchars = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ";
 3:  var tchars = "nopqrstuvwxyzabcdefghijklmNOPQRSTUVWXYZABCDEFGHIJKLM";
 4:  var trans = {};
 5:  for (var i=0; i<pchars.length; i++) {
 6:    trans[pchars[i]] = tchars[i];
 7:  }
 8:  
 9:  // Get the input from Shortcuts, convert it, send it back
10:  var s = args.shortcutParameter;
11:  s = s.split('').map(x =>  trans[x] ? trans[x] : x).join('');
12:  Script.setShortcutOutput(s);
13:  Script.complete()

I don’t want to discuss the pure ROT13 portion of this code². The lines that are important for communicating with Shortcuts are 10, 12, and 13.

Lines 10 and 12 do pretty much what you expect. Line 10 gets whatever is passed into Scriptable from Shortcuts and saves it to a variable. Line 12 then sends the result back out to Shortcuts. Line 13 is a weird kludge. For reasons I don’t fully understand, telling Scriptable that it’s finished is good practice because it can speed up a script.

The speed difference can be dramatic. Here’s how the shortcut runs when Scriptable includes that line:

And here’s how it runs with that line commented out:

No one would want to use the shortcut if it sat around like that.

One last thing about the Scriptable-based shortcut. You’ll note that after the conversion from HTML to rich text, the rich text is then put into a text block before sending it to Scriptable. Without that step, Scriptable returned lots of junk, probably because it understands rich text and doesn’t automatically treat it as plain text. None of the other shortcuts needed that step.

Thanks to Rosemary Orchard, Federico Viticci, and Majd Koshakji for their tweets regarding Scriptable and Shortcuts.

I said above that Scriptable is a good modern iOS citizen. Pythonista is… not. While the Run Script action for Pythonista includes a parameter for passing data into a Python script, I’ve found that feature unreliable in iOS/iPad 13. The only safe way to get data into and out of Pythonista is through the clipboard. This means either destroying what was on the clipboard originally or adding Shortcuts code to save the clipboard before calling Pythonista and restore it after returning, which is what I did in the shortcut above.

And speaking of returning, Pythonista won’t return automatically. You have to use URL schemes to get back to whatever app you came from. This is not unreasonable if you want to return the Shortcuts. Finishing the script with

webbrowser.open('shortcuts://')

will take you back to the currently running shortcut and it will pick up from where it left off. But if your shortcut is to be used from the Share Sheet³, the webbrowser.open function has to be fed the URL of the app you were in when you brought up the Share Sheet. This makes the Pythonista script less flexible, as we’ll see in a minute.

The rot13.py script called by the Pythonista-based shortcut is this:

python:
 1:  from string import ascii_letters
 2:  import clipboard
 3:  import webbrowser
 4:  
 5:  # Make a ROT13 translation table
 6:  r = 13
 7:  a = ascii_letters
 8:  b = a[r:2*r]+a[:r]+a[3*r:]+a[2*r:3*r]
 9:  t = str.maketrans(a, b)
10:  
11:  # Get the input text, translate it, and put it on the clipboard
12:  s = clipboard.get()
13:  clipboard.set(s.translate(t))
14:  webbrowser.open('tweetbot://')

Again, let’s focus on the code that communicates with Shortcuts. In this case, we’re really communicating with the clipboard (Lines 12-13) and then heading back to the app from which we came (Line 13).

And here you see the lack of flexibility inherent in the webbrowser.open technique. This script works only when reading Twitter in Tweetbot; it won’t work in the official Twitter app, Twitterrific, or Safari. Neither the Scriptable-based shortcut or any of the others have this limitation.⁴

If there were a way for Shortcuts to know which app was active when it was called from a Share Sheet, we could add code to the Pythonista-based shortcut to build a dictionary of apps and their URL schemes and then pass the appropriate URL scheme to Pythonista along with the tweet text. Sadly, this doesn’t seem possible with Shortcuts now. (Of course, it would be even better to get a Shortcuts-aware update to Pythonista. It’s been about two years since the last release.)

There’s one other deficiency in the Pythonista-based shortcut: after you dismiss the alert with the translated text, you’re left in the Share Sheet, which you have to dismiss by hand. All the other shortcuts dismiss the Share Sheet automatically.

Thanks to mikeymikey for showing me the Pythonista workarounds.

Before leaving this category, I should mention that if you really wanted a ROT13 translator that was based on Pythonista or Scriptable, you probably wouldn’t bother with Shortcuts at all. Both Python and JavaScript themselves have ways to download HTML, extract text, handle HTML entities, and present results. I did these in Shortcuts to explore how the scripting languages can be used within Shortcuts and what the advantages and disadvantages of doing so are.

Finally, we come to doing the ROT13 translation in Shortcuts itself. Conceptually, the approach is the same as what I used in Line 11 of the JavaScript code:

1. Split the string into a list of characters.
2. Use a dictionary to replace all ASCII characters with their ROT13 translation. Leave the other characters alone.
3. Join the list of characters back into a string.

The difference between doing this in Shortcuts and doing it in JavaScript is that making the translation dictionary with 52 entries by hand is a pain in the ass and the split-translate-join steps take more time to write and use up more code space. But it can be done. Here’s the complete shortcut:

The tedious dictionary creation is in the text block that contains this JSON object

{"A":"N","B":"O","C":"P","D":"Q","E":"R","F":"S",
"G":"T","H":"U","I":"V","J":"W","K":"X","L":"Y",
"M":"Z","N":"A","O":"B","P":"C","Q":"D","R":"E",
"S":"F","T":"G","U":"H","V":"I","W":"J","X":"K",
"Y":"L","Z":"M","a":"n","b":"o","c":"p","d":"q",
"e":"r","f":"s","g":"t","h":"u","i":"v","j":"w",
"k":"x","l":"y","m":"z","n":"a","o":"b","p":"c",
"q":"d","r":"e","s":"f","t":"g","u":"h","v":"i",
"w":"j","x":"k","y":"l","z":"m",".":"."}

and the “Get dictionary” step after that.

To avoid typing in all that nonsense, I edited the JavaScript code above to output the trans variable and pasted that into the text block. The only addition I had to make was the weird period-to-period mapping at the end. For some reason, without that addition, Shortcuts would choke whenever it encountered a period in the input.

The split-translate-join steps are reproduced here:

Compare this with Line 11 of the JavaScript code and you can understand John Gruber’s distain for Shortcuts. And this JavaScript is clumsy when compared to Perl/Python/Ruby, which don’t need to convert the string to and from a list of characters. But this is a way to avoid any outside dependencies.

Thanks to David Anson and Leon Cowle for their tenacity in working on and sharing their Shortcuts-only ROT13 code.

For me, the “right” solution is the one that uses Text Case. It’s relatively compact and leverages an app I already have. But it’s good sometimes to stretch your scripting muscles and learn new things that aren’t immediately productive. I definitely learned some new things putting these shortcuts together and it sure wasn’t immediately productive.

---

ROT13: A Twitter Odyssey

December 24, 2019 at 10:15 AM by Dr. Drang

This started with Chuck Wendig’s recent tweet storm about the new Star Wars movie, the first of which is this:

To avoid inadvertent spoilers to his followers, he encoded the tweets using the old ROT13 system.

Serenity Caldwell linked to the tweet storm and then mentioned that she read it using Matthew Cassinelli’s shortcut for ROT13 decoding, first written two years ago. Jason Snell then mentioned that he’d written his own ROT13-decoding shortcut that used the Text Case app.

Yeah, he says “Title Case,” but that’s probably because converting to title case is his most frequent use of the app. Text Case does all sorts of text manipulations and, as Jason found very handy, provides a Shortcuts action that you can use to access its features (including ROT13) from your own shortcuts.

Matt’s shortcut doesn’t rely on Text Case, but it does go off-device via a link to decode.org. I don’t think there’s good way to do ROT13 within Shortcuts itself; you either follow Jason’s route by using another app or Matt’s by using an online service.¹

I installed both Matt’s and Jason’s shortcuts and tried them out. They both work well, but I found some weird output due to Twitter’s oddball handling of certain characters.

For example, as I used them on Chuck Wendig’s spoiler tweets, I found that Twitter uses the ' HTML entity to depict straight apostrophes. It’s weird that they use an entity for a perfectly legal HTML character, but I guess they do it to avoid single quotes within single quotes in their code. The upshot is that both shortcuts translate

srryf yvxr vg'f

into

feels like it's

which is certainly readable, but not ideal.

As I looked further into Twitter’s output (see below), I also saw " for straight double quotes (why they weren’t done as " is beyond me) and 
 for newlines. Jason’s shortcut handles newlines, but neither display straight double quotes the way you’d like.

I decided to steal bits of both shortcuts and try to adjust the way they handle entities to get output that looks more normal. The first step is to get the tweet’s text from its URL. Jason does it with these steps:

Matt does it with these:

They both do a GET on the tweet’s URL, treat the contents returned as HTML and then extract the tweet text from within the HTML. But they get it from different places. After looking at the HTML of a handful of tweets, I’ve found different variations on the tweet text in five places. How you process the text depends on where you get it from.

The first place you find the text of the tweet is where Jason’s shortcut extracts it: in the <title> tag. Here’s the <title> of Jason’s tweet above:

<title>Jason Snell on Twitter: &quot;(Unhelpfully, 
@Twitterrific seems to only pass URLs in the share 
sheet? So I load the twitter web page, snip out the 
title text, then use Title Case to unrot13 it.)… 
https://t.co/ACJHPAYJNk&quot;</title>

As you can see, this includes a preamble, an ellipsis after the actual text, and also a URL. In this case, the URL points to Jason’s tweet itself, which is a little weird. In other tweets I looked at, there is no trailing URL, so I’m not sure what the criteria are for including it. And, frankly, I’m not all that interested in finding out. I just know that I’d prefer to use one of the other versions of the tweet text, if possible.

The second place you find it is in an attribute of a <link> tag:

<link rel="alternate" type="application/json+oembed" 
href="https://publish.twitter.com/oembed?url=https://
twitter.com/jsnell/status/1208431148721770496" title=
"Jason Snell on Twitter: &quot;(Unhelpfully, 
@Twitterrific seems to only pass URLs in the share 
sheet? So I load the twitter web page, snip out the 
title text, then use Title Case to unrot13 it.)… 
https://t.co/ACJHPAYJNk&quot;">

This also includes the preamble, ellipsis, and URL, so let’s move on.

The third place you find the tweet text is as an attribute of a <meta> tag:

<meta  property="og:description" content="“@settern 
@mattcassinelli (Unhelpfully, @Twitterrific seems to only 
pass URLs in the share sheet? So I load the twitter web 
page, snip out the title text, then use Title Case to 
unrot13 it.)”">

This looks like what I want. It has no preamble or trailing cruft and it includes the handles of the people being replied to. Basically, it’s exactly what you think of as “the tweet text” when you read a tweet.

The fourth place is the content of a <p> tag. This is where Matt’s shortcut extracts the text.

<p class="TweetTextSize TweetTextSize--jumbo js-tweet-text 
tweet-text" lang="en" data-aria-label-part="0">(Unhelpfully, 
<a href="/Twitterrific" class="twitter-atreply pretty-link 
js-nav" dir="ltr" data-mentioned-user-id="643443" ><s>@</s>
<b>Twitterrific</b></a> seems to only pass URLs in the share 
sheet? So I load the twitter web page, snip out the title 
text, then use Title Case to unrot13 it.)</p>

I don’t like this one because it presents a new problem: the @Twitterrific part is presented as the HTML of a link to the Twitterrific user profile page. I suppose I could work out a way to strip out all the anchor tag crap, but why bother when there’s already a string with exactly what I want?

The final spot for the tweet text is in some absurdly long HTML-encoded JSON string embedded in an <input type="hidden"> tag. And when I say “absurdly long,” I mean over 300 kilobytes. Here’s just the bit near the tweet text:

"title":"Jason Snell on Twitter: \"
(Unhelpfully, @Twitterrific seems to only pass URLs in 
the share sheet? So I load the twitter web page, snip out 
the title text, then use Title Case to unrot13 it.)\u2026 
https:\/\/t.co\/ACJHPAYJNk\"

I’m rejecting this out of hand because it’s in such a long tag.

If you’re wondering why I bothered looking at tweets that weren’t already ROT13 encoded, it’s because there are more examples of non-encoded tweets than encoded ones. Looking at non-encoded tweets gave me a wider variety of input to test. Also, encoding is in the eye of the beholder, and the ROT13 algorithm doesn’t care whether the characters look like English text or not.²

After deciding on which version of the tweet text to extract and playing around with different ways to handle the HTML entities, this is the shortcut I came up with:

The first four steps are just like Matt’s and Jason’s, except I extract the text from a different part of the HTML.

The best way to handle most HTML entities is to convert the HTML to rich text, which is done in the 6th step. Unfortunately, this doesn’t preserve the newlines; it flattens them into a single space. So the 5th step converts every 
 into a series of ten equals signs prior to the conversion to rich text. Then after Text Case does the ROT13 encoding in Step 7, Steps 8 and 9 covert the equals sign string into a newline. (The text in Step 8 is just a blank line. I would have preferred to use \n as the replacement text in Step 9, but that doesn’t work, even if you tell Shortcuts to use regular expressions in the Show More part of that step.)

I decided to use Text Case for the ROT13 encoding instead of decode.org for a few reasons:

1. I already own Text Case, so using it doesn’t cost me anything.
2. It’s faster to do the conversion on-device than calling out to a website.
3. In some of my tests, Matt’s shortcut returned an explanation of ROT13 instead of the decoded text. I suspect decode.org doesn’t have the most robust API in the world.

If you want to use this shortcut without copying all the regex crap, you can download it from iCloud. This is what it looks like after running:

---

Given that I see very few ROT13 tweets, I can’t imagine using this shortcut too often. On strictly economic terms, it wasn’t worth the effort. But it was fun to go through the logic of Matt’s and Jason’s shortcuts, dissect the structure of the HTML returned by a Twitter URL, and then figure out how to handle the edge cases (of which I’m sure there are more).

During a checkup a year or so ago, my doctor asked if I do puzzles and things like that. It was a more clear indication of my advancing years than a prostate exam or colonoscopy. I’m not sure how definitive the research is on the connection between keeping your brain active and fending off Alzheimer’s, but I guess he thought of it as “it couldn’t hurt” advice. He’d probably approve of this shortcut.

---

Another small Amazon thing

December 15, 2019 at 10:15 PM by Dr. Drang

A brief followup to my last post.

Let’s say you’re using your iPhone or iPad and want an Amazon Associates link to the item you’re looking at in Safari. You don’t want to assemble a bunch of text about the product—the link itself is enough. There’s no need to mess around with the Amazon Product Advertising API, you can just use the regex matching that’s built into Shortcuts to extract the product’s ASIN from the page’s URL and use it to make a new link with your affiliate tag. Like this:

You will, of course, have to edit the second-to-last step, replacing “my-tag” with whatever your Amazon tag is.

This shortcut is meant to be used from the Share Sheet. When it’s run, it pulls the product ASIN (that’s Amazon’s product ID string) from the URL of the page you’re looking at and generates a new URL (usually simpler than the original) with your affiliate tag appended to the end. The new URL is then put on the clipboard, ready to be pasted wherever you need it.

I wouldn’t be surprised to find that Amazon has more ways to put the ASIN into a URL than the three that are covered by my regex. But this regex has always worked for me and can be extended if you find another format.

---

Older posts

Site search

Meta

• drdrang at leancrew
• Blog archive
• RSS feed
• JSON feed
• Twitter stream
• GitHub repositories

Recent posts

Credits

This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License.