And now it’s all this

I just said what I said and it was wrong
Or was taken wrong

Quicker linking to apps in Drafts

June 3, 2020 at 1:29 PM by Dr. Drang

Today on the Automators forum, there was a question about using Shortcuts to get the current version of an app in the App Store. When I read the answer, from Stephen Millard (@sylumer), I immediately knew I could modify his shortcut to speed up linking to apps, something I do quite often here.

Up until now, this is how I’ve been linking to an app:

1. Open the App Store app.
2. Search for the app I want to link to.
3. Get the app’s URL from the Share Sheet.
4. Return to Drafts, which is where I’m writing the post, select the text I want to turn into a link (typically the name of the app), and run the Markdown Reference Link action from @pdavisonreiber.

Because the App Store doesn’t open in search mode, this can take half a minute to a minute to do. Even worse is the context shift as I switch from Drafts to the App Store and back. By stealing Stephen’s shortcut and making a Drafts action that glues it to @pdavisonreiber’s action, my workflow has been reduced to

1. Type the name of the app and select it.
2. Run the App Store Link action.

which runs in just a few seconds, requires no intervention on my part, and has no context shift other than me watching the screen change from Drafts to Shortcuts and back.

Here’s my adaptation of Stephen’s shortcut. It grabs the App Store URL of the given app and puts it on the clipboard

Step	Action	Comment
0		I won’t be running this from the Share Sheet, but I have to set it up as if I will so it accepts text.
1		Search the App Store for the given text. We have to have some faith that the search will work.
2		We also have to have some faith that the app we’re looking for is at the top of the found list.
3		Get the URL of the app.
4		Put it on the clipboard so it’s ready to be used in the next step of the Drafts action.

The App Store Link action consists of two steps. The first runs the App Store URL shortcut, passing it the selected text.

The second step runs the Markdown Reference Link action, which uses the URL on the clipboard to turn the selected text into a link.

So by doing virtually no work of my own, I have a new Drafts action that will save me upwards of a minute every time I need to link to an app and doesn’t take me out of the flow of writing.

---

Drafts and iCloud Drive

May 30, 2020 at 11:24 AM by Dr. Drang

I didn’t renew my Dropbox Plus account when it expired a month or two ago. For several months I’d had both it and a 2 TB iCloud account active at the same time and had been moving my files and my way of working from one to the other. This was mainly due to my shift over the past four years from being Mac-only to Mac-mainly to iPad-mainly. While the Finder is equally adept at managing files in either cloud, Files is much better with files on iCloud Drive.

My concern was with Drafts. There are programmatic ways to save a draft to a specified file in Dropbox—I use a “filelines” system similar to Vim’s modelines in which the file path is embedded in the text of the draft—but sandboxing prevents that working with file paths in iCloud Drive. Here’s a Drafts forum thread in which Greg Pierce (agiletortoise) explains the problem.

As it turned out, my concern was overblown. Shortly after writing this post in which I talk about writing almost everything in Drafts, I started write almost everything that needs to be saved as a file in Textastic. Textastic is file-based and can read and write anywhere in iCloud Drive (or anyplace Files can access) without continual user approval.

Still, I did occasionally miss being able to use filelines to save drafts directly to iCloud Drive. It wasn’t until this past week that I realized there was a way around it by using Scriptable.

Scriptable has a feature called “bookmarks,” which are ways to access files (and folders) outside of the Scriptable sandbox. You create them by choosing File Bookmarks in Settings.

Then tap the + button in the upper right corner and choose Pick Folder from the popup menu.

You’re now presented with the usual Files-style list of file providers. Choose iCloud Drive.

Finally, you’re asked to choose a name for this bookmark. I chose “iCloud.”

You can now use this bookmark and Scriptable’s FileManager library to access any folder or file in iCloud Drive. The great thing about this is that you’re not limited to just the top level of iCloud Drive—any file or folder at any level in the hierarchy below iCloud Drive is accessible.

With this bookmark defined, I can create a system that looks for an iCloud fileline, a line in a draft that contains

icloud:/path/to/subfolder/filename.txt

and saves the contents of the draft to a file in the given location. Typically, this line will be in a comment, so it will be

# icloud:/path/to/subfolder/filename.py

for a Python file or

% icloud:/path/to/subfolder/filename.tex

for a LaTeX file.

The system consists of two parts:

1. A Drafts action that gets the fileline, extracts the path, and sends the path and the full draft contents to a shorcut.
2. A shortcut that takes the information from the previous step and writes the contents of the draft to the given file.

The Drafts action (which you can download), starts with this JavaScript step,

javascript:
 1:  // Get the folder and filename from the dbox line.
 2:  var d = draft.content;
 3:  
 4:  // Regex for fileline.
 5:  var fileRE = /icloud:\s*(.+)\s*$/m;
 6:  if (fileRE.test(d)) {
 7:    // Get the path. Add a slash to the front if I forgot.
 8:    var path = d.match(fileRE)[1];
 9:    if (path[0] != '/') {
10:      path = '/' + path;
11:    }
12:  
13:    // Create a JSON template to pass to Scriptable via Shortcuts
14:     var fileinfo = {"path":path, "text":d};
15:    
16:    // Create a template tag from the dictionary.
17:    draft.setTemplateTag("fileinfo", JSON.stringify(fileinfo))
18:   
19:  } else {
20:    alert("No icloud: line");
21:    context.fail("No icloud: line");
22:  }

which extracts the path from the iCloud fileline. If there is no fileline, it stops the action with an alert. If there is a fileline, it creates a JSON dictionary with two entries: the path from the fileline and the full text of the draft. The dictionary is saved to a template tag in Line 17 so it can be passed to the next step of the action.

The second and final step in the Drafts action is this Run Shortcut step,

which passes the dictionary created in the first step to a shortcut entitled “Save to iCloud.”

Here’s the “Save to iCloud” shortcut:

Step	Action	Comment
0		Even though we won’t be using it from the Share Sheet, we define it this way so it can accept the text passed to it from Drafts.
1		The inline JavaScript, shown in full below, extracts the path and the file contents from the JSON dictionary and writes the draft contents to the given file. The “Run in App” setting has to be on for the bookmark to work.

Here’s the inline JavaScript:

javascript:
1:  // The argument passed in from Shortcuts should be a
2:  // JSON dictionary with "path" and "text" entries.
3:  var d = args.shortcutParameter;
4:  
5:  // Set up connection to the root level of iCloud Drive
6:  // through a Scriptable bookmark and save the text.
7:  var myFM = FileManager.iCloud();
8:  var iCloud = myFM.bookmarkedPath('iCloud');
9:  myFM.writeString(iCloud + d['path'], d['text']);

Line 3 gets the argument passed in and saves it to the JavaScript dictionary, d. The rest of the script gets the iCloud bookmark, appends the filepath to it, and writes the text of the draft to that file.

If you don’t feel like writing it yourself, you can download the shortcut.

My understanding (based on a post from its developer) is that Toolbox Pro is beta testing a similiar bookmarking system. If that gets released, it may simplify the shortcut.

There is a hitch in this system that’s unlikely to affect me but might affect you. Saving a draft from my iPad to iCloud drive works perfectly, and every time I edit the draft and resave it, the file is overwritten with the latest version of the draft, just as you would expect. But if I edit that draft on my iPhone and save it, a new file gets created in iCloud Drive. If the original file saved from the iPad was

/path/to/subfolder/filename.txt

the file saved from the iPhone will be

/path/to/subfolder/filename 2.txt

even though the fileline is still

icloud:/path/to/subfolder/filename.txt

I assume this has something to do with Scriptable bookmark on the iPhone being somehow different from the bookmark on the iPad, despite them both pointing to the same iCloud Drive location.

As I say, I don’t think this will affect me, as the writing I do on my phone tends to be ephemeral stuff. But it is one of those things that shows how iCloud Drive isn’t quite on the level as Dropbox (or Box or Google Drive or…) when it comes to working with Drafts.

---

COVID antibody testing and conditional probability

May 27, 2020 at 10:52 PM by Dr. Drang

In a Slack channel that I frequent and he infrequents, podcasting magnate Lex Friedman linked to this CNN article on the effectiveness of COVID-19 antibody tests:

The concern is that people who’ve had a positive result from an antibody (serologic) test may be fooling themselves into thinking they’re immune. Unfortunately, there’s a numerical problem with this:

The CDC explains why testing can be wrong so often. A lot has to do with how common the virus is in the population being tested. “For example, in a population where the prevalence is 5%, a test with 90% sensitivity and 95% specificity will yield a positive predictive value of 49%. In other words, less than half of those testing positive will truly have antibodies,” the CDC said.

Although CNN doesn’t link to it, the quote comes from this CDC page.

Let’s go through the numbers and see how 90% turns into 49%.

First, there are a couple of terms of art we need to understand: sensitivity and specificity. Sensitivity is the accuracy of positive test results; it is the percentage of people who actually have the condition that will test positive for it. Specificity is the accuracy of negative test results; it is the percentage of people who actually don’t have the condition that will test negative for it.

With those definitions in mind, we’ll create a 2×2 contigency table for a population of 100,000 people in which the prevalence, sensitivity, and specificity are as given in the quote.

                  Have      Don't have
               antibodies   antibodies
-------------------------------------------------
Test positive    4,500        4,750    |   9,250
Test negative      500       90,250    |  90,750
-------------------------------------------------
                 5,000       95,000    | 100,000

5,000 people actually have the antibodies (5% of 100,000) and 95,000 do not. Of the 5,000 with antibodies, 4,500 (90% of 5,000) will test positive for them and the remaining 500 will test negative for them. Of the 95,000 without antibodies, 90,250 (95% of 95,000) will test negative for them and the remaining 4,750 will test positive for them.

So of the 9,250 people who test positive for the antibodies, only 4,500 actually have them. That’s where the 49% figure comes from in the quote.

This is a very common sort of problem to spring on students in a probability class who are learning about conditional probability and Bayes’s Theorem. The key thing is to realize that the probability that you have antibodies given that you had a positive test is definitely not the same as the probability that you had a positive test given that you have antibodies. When you hear about a test’s “accuracy,” it’s usually the latter that’s presented even though you as a patient are only interested in the former. After all, if you already knew you had the condition, you wouldn’t need to have the test.

Although the CNN article makes it sound as if antibody testing is useless, there are ways of using multiple tests (of different types) to achieve a high predictive value:

Algorithms can be designed to maximize overall specificity while retaining maximum sensitivity. For example, in the example above with a population prevalence of 5%, a positive predictive value of 95% can be achieved if samples initially positive are tested with a second different orthogonal assay that also has 90% sensitivity and 95% specificity.

We can see how this works by putting the 9,250 population of positive tests through a second round of independent testing. Here’s the contingency table for the second test:

                  Have      Don't have
               antibodies   antibodies
-------------------------------------------------
Test positive    4,050          238    |   4,288
Test negative      450        4,512    |   4,962
-------------------------------------------------
                 4,500        4,750    |   9,250

Here we see that of the 4,288 people who test positive in the second test, 4,050 (94%) actually have the antibodies. The second test works so well because it is performed on a population that has a much higher prevalence of people with antibodies. It benefits from the screening done by the first test

---

Turning bookmarks into Markdown reference links

May 25, 2020 at 3:29 PM by Dr. Drang

For some time now, I’ve been writing almost all of my blog posts in Drafts on an iPad. I usually have Drafts and Safari set up in Split View, and I copy the URL of the current Safari page whenever I need to add a link to a post. I’ve always preferred Markdown’s reference format for links, and this action inserts reference links in Drafts the same way an AppleScript I wrote a long time ago does it in BBEdit.¹

With a URL on the clipboard, I select the text that’s going to be the link and run the action. It surrounds the selected text with brackets, puts a numbered reference after it,

after reading [this John Voorhees review][3] in MacStories.

and then adds the reference and URL at the bottom of the draft:

[3]: https://www.macstories.net/reviews/highlights-for-iphone-and-ipad-an-excellent-companion-for-researchers/

The reference number is incremented each time.

Sometimes, though, I have a bunch of websites open in tabs in Safari, and because I know I’ll be linking to each of them, I’d like to add all the references to the bottom of the draft in a single step. I can do this via AppleScript on the Mac, but Shortcuts currently isn’t smart enough to do this sort of thing. So I decided to use the Mac to get around iPadOS’s limitations.

First, on the iPad, I bookmark all the tabs to a “Blogging” sublist I’ve set up in my Favorites (which is the Bookmarks bar on the Mac). This can be done in a single step by long-pressing the bookmark button and choosing the Add Bookmarks for # Tabs command from the popup menu.

Because iCloud syncs my bookmarks, all of these links are now saved in the ~/Library/Safari/Bookmarks.plist file on my Mac. If I have a script on my Mac that can extract them from that file, I can run it from my iPad via the Shortcuts Run Script Over SSH command.

Such a script isn’t that hard to write in Python, as the standard library has a plistlib module that knows how to read and write plist files.

Here’s the mdbookmarks script that pulls out all the URLs in the “Blogging” sublist and prints them in Markdown reference format:

python:
 1:  #!/usr/bin/env python
 2:  
 3:  import plistlib
 4:  import os
 5:  from docopt import docopt
 6:  
 7:  # Handle the command line options
 8:  usage = '''Usage:
 9:    mdbookmarks [options]
10:  
11:  Options:
12:    -n NNN  Starting number for reference links [default: 1]
13:    -h      Show this help message
14:  
15:  Return a set of lines with Markdown reference links to all the
16:  bookmarks in the "Blogging" section of Safari's Bookmarks bar.'''
17:  
18:  args = docopt(usage)
19:  n = int(args['-n'])
20:  
21:  # Load the Safari Bookmarks plist
22:  fn = os.environ['HOME'] + '/Library/Safari/Bookmarks.plist'
23:  with open(fn, 'rb') as f:
24:    bm = plistlib.load(f)
25:  
26:  # Work through the plist and assemble all the URLs in the
27:  # "Blogging" section of the Bookmarks bar into a list
28:  blist = []
29:  for a in bm['Children']:
30:    if a['Title'] == 'BookmarksBar':
31:      for b in a['Children']:
32:        try:
33:          if b['Title'] == 'Blogging':
34:            for c in b['Children']:
35:              blist.append(f"[{n}]: {c['URLString']}")
36:              n += 1
37:        except KeyError:
38:          pass
39:  
40:  # Print the list, one per line
41:  print('\n'.join(blist), end='')

As usual, I use the nonstandard docopt module to handle the command-line switches.

The trickiest part of this script was figuring out where the “Blogging” bookmarks are kept within the labyrinth of Bookmarks.plist. Lines 28–38 were built up through trial and error. I dug through the layers of the plist, testing each layer using type and keys to eventually find my way to the URLs.

Although this script is written for Python 3.7, it shouldn’t be too hard to get it to run on the Python 2.7 that comes preinstalled on the Mac. I think you can just change the f-string in Line 35 to a format call.

I ran mdbookmarks on my Mac to test it, but it’s meant to be run via a shortcut on my iPad. The shortcut that does it is called “Blogging Bookmarks,” and it has only two steps:

Step	Action	Comment
1		Get the starting number from the user.
2		Log into my Mac and run the mdbookmarks script. Use the starting number from the previous step.

You’ll note that this shortcut doesn’t do anything with the output of mdbookmarks. That’s because it’s meant to be run from a simple Drafts action that can be downloaded from here. There are only two steps in the action:²

1. Run the “Blogging Bookmarks” shortcut. This makes the output of mdbookmarks script available in the [[shortcut_result]] template.
2. Insert [[shortcut_result]] at the current cursor point. The intention is to run this action with the cursor at the bottom of the draft.

With all three pieces in place, adding a bunch of references to blog post is fairly easy. I just add bookmarks to the “Blogging” sublist as I do my research, then run the “Blogging Bookmarks” action in Drafts to insert all the references when I start writing the post.

I can make this system sound ridiculously complex: It’s a Drafts action that runs a shortcut that connects to my Mac over SSH and runs a Python script that reads a plist file and extracts information that then works its way back along the chain to insert text with that information into Drafts. But each link in the chain is straightforward. Only mdbookmarks took more than a couple of minutes to write, and that was because the Bookmarks.plist file is a nested mess.

The biggest limitation of this system is that I have to clean out the “Blogging” sublist whenever I’m done with a post. Otherwise, links from older posts will be inserted into newer posts.

I’m pleased I was able to build this system, but I wish I didn’t have to. I’d be much happier if Apple made all of this obsolete by adding features to Shortcuts that, among other things, allow us to get all of Safari’s tab links directly.

---

Older posts

Site search

Meta

• drdrang at leancrew
• Blog archive
• RSS feed
• JSON feed
• GitHub repositories

Recent posts

Credits

This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License.