Drafts and Dropbox

Drafts has long had an easy way to save a draft to Dropbox: You use the first line of your draft as the file name and make a action with the Dropbox step set up something like this:

Simple Dropbox save step

Drafts knows, through its template tag system, that safe_title is the first line of the draft with troublesome characters removed (you can also use the title tag, which takes the first line as-is). The Dropbox path is the slash-separated list of subfolders to your main Dropbox folder. Here, I’ve shown it as a fixed, literal path, but you can also use tags to choose a subfolder by date, another line in the draft, or even the expansion of a TextExpander snippet.

But what if you don’t want the first line of your file to be the title? For example, when I write a blog post, the system of scripts that do the publishing expect the header lines of the Markdown source code to look like this,

Title: A line-numbering action for Drafts
Keywords: drafts
Summary: A script/action for numbering lines of code in a style that works on this blog.
Date: 2018-05-05 18:31:02
Slug: a-line-numbering-action-for-drafts

and they expect the source to be a file with the name <slug>.md in the yyyy/mm subfolder of my blog’s source directory, where the slug is taken from the Slug line and the year and month are taken from the Date line. To publish from my iPad, I need to save the draft to that spot with that file name.

I do this with a two-step action. The first step is a script that gets the required information from the draft:

 1:  // Set template tags for the Dropbox folder and filename
 2:  // of a blog post.
 4:  var d = draft.content;
 6:  // Set up regexes for header info.
 7:  var dateRE = /^Date: (\d\d\d\d)-(\d\d)-\d\d \d\d:\d\d:\d\d$/m;
 8:  var slugRE = /^Slug: (.+)$/m;
10:  // Get the year and month and set the path.
11:  var date = d.match(dateRE);
12:  var year = date[1];
13:  var month = date[2];
14:  var path = '/blog/all-this/source/' + year + '/' + month + '/';
16:  // Get the filename from the slug.
17:  slug = d.match(slugRE)[1];
19:  // Set tags for use in other action steps.
20:  draft.setTemplateTag('blog_path', path);
21:  draft.setTemplateTag('blog_slug', slug);

Line 4 pulls in the entire contents of the current draft. Lines 7 and 8 establish regular expressions for extracting the year, month, and slug from the header lines. Note the capturing parentheses in Line 7 collect the year and the month, and the capturing parentheses in Line 8 collect the slug. The m flag at the end of each regex stands for “multiline,” which means the ^ and $ represent the end of a line, not the end of the entire text string being searched.

Lines 11–14 run the date regex on the text, pull out the year and month, and construct the path to the subdirectory where the file will be saved. Line 17 extracts the slug.

Finally, Lines 20 and 21 create the blog_path and blog_slug template tags, which we’ll use in the next step of the action. The setTemplateTag function is described in the Drafts JavaScript documentation.

The second step in the action is similar to the simple Dropbox step shown above. This time, though, we’re using the tags defined in the script.

Blog post saving Dropbox step

Now I can write my blog posts with the same format I’ve used for years and can upload them to Dropbox in a single step when I’m ready to publish.

This action is very handy, but it’s specific to one use case. I wanted a more general solution for saving other types of files—LaTeX files, Python scripts, whatever—to Dropbox. In particular, I wanted flexibility in where the path and file name could be located within the file, and I wanted them to be easy to write and easy to identify when reading.

I decided to steal the idea of modelines from Vim and adapt it to file names and paths. Somewhere in the draft, I put a line—which I call a fileline—that contains


and that provides both the path and the file name for where the draft should be saved. Typically, these will be inside comments, so the draft of a Python file will start with

#!/usr/bin/env python
# dbox:/path/to/subfolder/filename.py

and a LaTeX file will have

% dbox:/path/to/subfolder/filename.tex

somewhere in the header.

Like my blog post saving action, the action that uses these “filelines” to determine where to save the draft consists of two steps. First, the script step that creates the template tags,

 1:  // Set template tags for Dropbox path and file name
 2:  // from dbox: file line in draft.
 4:  var d = draft.content;
 6:  // Regex for fileline.
 7:  var fileRE = /dbox:(.+)\/(.+)/m;
 9:  // Get the year and month and set the path.
10:  var fileline = d.match(fileRE);
11:  var path = fileline[1] + '/';
12:  var filename = fileline[2];
13:  draft.setTemplateTag('dbox_path', path);
14:  draft.setTemplateTag('dbox_filename', filename);   

and then the Dropbox step that uses those tags for saving,

Fileline Dropbox step

The script follows the same outline as the blog post saver, but the regex in Line 7 is worth an extra look. It takes advantage of the fact that the + quantifier is “greedy.” So the dbox:(.+)\/ part of the regex captures every character after “dbox:” and before the last slash on the fileline. So no matter how many subdirectories deep I want to save this draft, they’re all captured in fileline[1]. Only the file name is captured in fileline[2].

This action is available to download and install from the Drafts Action Directory.

There are ways to do this without using scripting. You could, for example, make a header with the file name in the top line, the directory in the second line, and a blank line separating the header from the rest of the contents. With the proper template tags, you could set up a single-step action that saves only the remainder of the draft, not its entire contents.

Header Save Dropbox step

I’ve put this in the Drafts Action Directory, too, calling it Header Save. Although it’s simple and easy to understand, because the header isn’t saved to Dropbox, there’s a difference between the contents of the draft and the contents of the saved file. I use the fileline solution because I prefer a system in which the draft and the saved file are the same.

A line-numbering action for Drafts

Among my many deficiencies as a human is my failure to write a review of Drafts 5. It will come eventually, but in the meantime, I’ll put out a few posts on how I’m scripting and configuring Drafts to be my primary text editor on iOS for both short snippets (which is how I’ve used it for years) and longer pieces like reports and blog posts. If you want to see a real review of Drafts 5, I’ll suggest

Because so many of my blog posts include source code with line numbers, I need my text editor to have a line-numbering command. Drafts 5 has built-in a way of displaying line numbers,1 but I need the numbers to part of the actual text. The action I made for doing this is called, with the great imagination I’m known for, Number Lines, and you can get it from the Drafts Action Directory.

The action consists of a single step, which is this script:

 1:  // Number the selected lines (or all lines) with colon and
 2:  // two spaces after each line number.
 5:  // Function for right-justifying text str in width n.
 6:  function rjust(str, n) {
 7:    var strLen = str.length;
 8:    if (n > strLen) {
 9:      var prefix = ' '.repeat(n - strLen)
10:      return prefix + str;
11:    } else {
12:      return str;
13:    }
14:  }
16:  // Get either the current selection or the entire draft.
17:  var sel = editor.getSelectedText();
18:  if (!sel || sel.length==0) {
19:    editor.setSelectedRange(0, editor.getText().length);
20:    sel = editor.getSelectedText();
21:    }
23:  // Break the text into lines and number them.
24:  // Right-justify the numbers and put a colon and
25:  // two spaces after the line number.
26:  var lines = sel.split('\n');
27:  var numLines = lines.length;
28:  var width = Math.floor(Math.log(numLines)*Math.LOG10E) + 1;
29:  var numbered = [];
30:  var lNum;
31:  for (var i=0; i<numLines; i++) {
32:    lNum = i + 1;
33:    numbered.push(rjust(lNum.toString(), width) + ':  ' + lines[i]);
34:  }
36:  // Replace the original text with the line-numbered text.
37:  editor.setSelectedText(numbered.join('\n'));

This is longer than I think it should be, mainly because JavaScript doesn’t have Python’s (or Perl’s or Ruby’s) text-handling capabilities. It doesn’t even have a C-style sprintf. So the rjust function in Lines 6–14 is there just to right justify the line numbers.

Update May 5, 2018 9:56 PM
This tweet from Tim Tepaße let me know that recent versions of JavaScript include a padStart method that will handle the right justification of strings. So the rjust method wasn’t necessary, and line in which it’s used can be changed to

numbered.push(lNum.toString().padStart(width) + ':  ' + lines[i]);

I’ve left the original code here in the blog post, but the Drafts Action Directory has been updated.

This tells me a couple of things:

  1. Code in other actions I’ve written that zero-pads numbers to ensure two digits for month and day numbers can be simplified with padStart (it has an optional parameter for the padding character).
  2. I need to update the JavaScript references I use. The websites that Google steers me to don’t know about padStart, nor does my copy of David Flanagan’s JavaScript: The Definitive Guide. To be fair, I just noticed my copy of Flanagan was published in 2006.

Lines 17–21 get either the selected text or—if no text is selected—the entire draft. This is the text that will have line numbers added to it.

Line 26 splits the text into an array of lines so we can loop through and add line numbers to each one. Line 27 gets the length of the array, numLines, which we’ll use as a limit in the for loop that starts on Line 31.

Before we start the loop, though, we need to know how wide (in characters) the line numbers will be. I could do this by turning numLines into a string and getting its length, but there’s no mathematical fun in that. Instead, in Line 28, I took the integer portion of the base-10 logarithm of numLines and added 1 to it. And because JavaScript doesn’t have a base-10 log function, I had to use the natural log of the numLines and convert it to base 10 by multiplying it by the natural log of 10. Who says junior high math doesn’t come in handy?

Inside the loop, we prefix each line with the right-justified line number, a colon, and two spaces (we’ll get to the colon and the spaces in a minute), and push the resulting string onto the end of the numbered array, which we initialized as empty in Line 29.

With the loop finished, we join up the elements of numbered with a newline character and replace the selection in the draft with the numbered lines.

So why the colon and the two spaces after the line number? Well, partly because I think it looks nice in a plain text file, but mostly because I have a JavaScript/jQuery function here on the blog that takes line-numbered source code formatted that way and styles it so the line numbers are visible but less intrusive.2 The setup is described in this post from way back in 2010.

I’ve been using Drafts 5 for almost two months now (I came in late to the beta), and I’ve been slowing building up and refining a set of actions to help me use it for nearly all of my iOS writing. Some of them are so customized for my way of working that they won’t be immediately useful to anyone else. But I’ll still post them, as they give a sense of what Drafts 5 is capable of, and they can be the starting point for your own scripts.

  1. Strictly speaking, it’s paragraph numbering, but for source code that’s the same as line numbering. 

  2. If you’re reading the RSS feed of this post, you’ll see the line numbers and colons without any styling. You’ll have to go to the actual post to see what I’m talking about. 

A small Apple surprise

It’s been a while since I’ve operated this here blog, so let’s get back into the swing of things with a post that nearly writes itself.

Apple announced its quarterly sales today. You can get loads of charts on all the data from the usual sources. Revenue was up significantly, but I’ve always been more interested in unit sales because it’s the number of Macs and iOS devices out there that provide the market for third-party developers who make the software I need to use these machines.

Here are the unit sales for the three products Apple reports separately. As always, the dots show the actual quarterly sales and the line is the four-quarter moving average. Also, I plot real dates along the horizontal axis, not Apple’s goofy fiscal quarters.

Apple sales

This doesn’t show the old iPhone sales growth, but it’s steady. Certainly not the precipitous dropoff that was being predicted some weeks ago from “supply channel sources.”

Because the iPhone dominates unit sales, the chart above isn’t particularly good at showing what’s going on with the iPad and the Mac. Here’s the iPad by itself:

iPad sales

The iPad has managed to pull off an entire year of growth after bottoming out this time last year. As with the iPhone, it’s not stellar growth, but it’s welcome after the previous three years. It would be nice to see it always over the 10 million threshold again.

(You’ll note that since becoming an iPad user, I’ve stopped being snarky about its no-longer-world-beating figures and have started rooting for it. That’s what happens when you have a dog in the fight.)

Here’s the Mac:

Mac sales

Bleh. While iMacs seem to be in good shape, the MacBook line, which has usually driven sales, has little to recommend it. The MacBook Air has wheezy old specs that make many prospective buyers fear they’d be getting instant obsolescence. The MacBook Pro has a keyboard with serious reliability problems and a feature, the TouchBar, that neither Apple nor third-party developers have done much with. The unsuffixed MacBook is, I think, a nice machine, but I’m not sure many users are willing to pay a premium for an extra-light notebook (that’s somewhat underpowered) when they can get an iPad.

The real news of Apple’s growth is coming from services and the elusive “other” category of products, and that’s been the case for a while. These categories don’t have unit sales figures, so I’ve kept away from them. Maybe I should rethink that.

The sunrise plot

Since I almost never make a graph without showing the code for it, here’s how the sunrise/sunset plots in yesterday’s post were made.

Chicago sunrise and sunset

I started with the US Naval Observatory’s 2018 sunrise/sunset data for Chicago, which is a plain text table (i.e, monospaced font using space characters to align columns) that looks like this:

USNO data for Chicago

I copied the table, pasted it into BBEdit, and did some editing to get it into this form:

2018-01-01  0718 1631
2018-01-02  0718 1632
2018-01-03  0718 1632
2018-01-04  0718 1633
2018-01-05  0718 1634
2018-01-06  0718 1635
2018-01-07  0718 1636
2018-01-08  0718 1637
2018-01-09  0718 1638
2018-01-10  0718 1639
2018-01-11  0717 1640
2018-01-12  0717 1642
2018-01-13  0717 1643
2018-01-14  0716 1644
2018-01-15  0716 1645
2018-01-16  0715 1646
2018-01-17  0715 1647
2018-01-18  0714 1649
2018-01-19  0714 1650
2018-01-20  0713 1651

Most of the editing consisted of selecting columns for February through December and pasting them under the January data. Then I prepended the year and month (with hyphens) in front of the the days. That left me with a file, called chicago-riseset.txt, with 365 lines and three columns. If I were going to do this sort of thing on a regular basis, I’d write a script to handle this editing, but for a one-off I just did it “by hand.”

The script that parsed the data and created the graphs is this:

 1:  #!/usr/bin/env python
 3:  from fileinput import input
 4:  from dateutil.parser import parse
 5:  from datetime import datetime
 6:  import numpy as np
 7:  from matplotlib import pyplot as plt
 8:  import matplotlib.dates as mdates
 9:  from matplotlib.ticker import MultipleLocator, FormatStrFormatter
11:  # Read in the sunrise and sunset data in CST
12:  # and convert to floating point hours
13:  days = []
14:  rises = []
15:  sets = []
16:  for line in input():
17:    d, r, s = line.split()
18:    days.append(parse(d))
19:    hr, min = int(r[:2]), int(r[-2:])
20:    rises.append(hr + min/60)
21:    hr, min = int(s[:2]), int(s[-2:])
22:    sets.append(hr + min/60)
24:  # Daylight lengths
25:  lengths = np.array(sets) - np.array(rises)
27:  # Get the portion of the year that uses CDT
28:  cdtStart = days.index(datetime(2018, 3, 11))
29:  cstStart = days.index(datetime(2018, 11, 4))
30:  cdtdays = days[cdtStart:cstStart]
31:  cstrises = rises[cdtStart:cstStart]
32:  cdtrises = [ x + 1 for x in cstrises ]
33:  cstsets = sets[cdtStart:cstStart]
34:  cdtsets = [ x + 1 for x in cstsets ]
36:  # Plot the data
37:  fig, ax =plt.subplots(figsize=(10,6))
38:  plt.fill_between(days, rises, sets, facecolor='yellow', alpha=.5)
39:  plt.fill_between(days, 0, rises, facecolor='black', alpha=.25)
40:  plt.fill_between(days, sets, 24, facecolor='black', alpha=.25)
41:  plt.fill_between(cdtdays, cstsets, cdtsets, facecolor='yellow', alpha=.5)
42:  plt.fill_between(cdtdays, cdtrises, cstrises, facecolor='black', alpha=.1)
43:  plt.plot(days, rises, color='k')
44:  plt.plot(days, sets, color='k')
45:  plt.plot(cdtdays, cdtrises, color='k')
46:  plt.plot(cdtdays, cdtsets, color='k')
47:  plt.plot(days, lengths, color='#aa00aa', linestyle='--', lw=2)
49:  # Add annotations
50:  ax.text(datetime(2018,8,16), 4.25, 'Sunrise', fontsize=12, color='black', ha='center', rotation=9)
51:  ax.text(datetime(2018,8,16), 18, 'Sunset', fontsize=12, color='black', ha='center', rotation=-10)
52:  ax.text(datetime(2018,3,16), 13, 'Daylight', fontsize=12, color='#aa00aa', ha='center', rotation=22)
54:  # Background grids
55:  ax.grid(linewidth=1, which='major', color='#cccccc', linestyle='-', lw=.5)
56:  ax.grid(linewidth=1, which='minor', color='#cccccc', linestyle=':', lw=.5)
58:  # Horizontal axis
59:  ax.tick_params(axis='both', which='major', labelsize=12)
60:  plt.xlim(datetime(2018, 1, 1), datetime(2018, 12, 31))
61:  m = mdates.MonthLocator(bymonthday=1)
62:  mfmt = mdates.DateFormatter('              %b')
63:  ax.xaxis.set_major_locator(m)
64:  ax.xaxis.set_major_formatter(mfmt)
66:  # Vertical axis
67:  plt.ylim(0, 24)
68:  ymajor = MultipleLocator(4)
69:  yminor = MultipleLocator(1)
70:  tfmt = FormatStrFormatter('%d:00')
71:  ax.yaxis.set_major_locator(ymajor)
72:  ax.yaxis.set_minor_locator(yminor)
73:  ax.yaxis.set_major_formatter(tfmt)
75:  # Tighten up the white border and save
76:  fig.set_tight_layout({'pad': 1.5})
77:  plt.savefig('riseset.png', format='png', dpi=150)

After all the imports in Lines 3–9, the script begins by using the fileinput module to read and parse the lines of the data file, one by one. Each line is split into date, rise time, and set time in Line 17. Line 18 parses the date using the dateutil library, returning a datetime object. Lines 19–20 then split the sunrise time into the hour and minute parts and convert them into a single floating point number. Lines 21–22 do the same thing to the sunset time. The dates and times are accumulated into the days, rises, and sets lists.

The duration of daylight is calculated by subtracting the rise time from the set time in Line 25. I could have done this within the loop of Lines 16–22, but chose to do it through NumPy array arithmetic instead.

Lines 28–34 handle the daylight saving time stuff. The USNO data are given in standard time. To convert to DST, I needed to create new date and time lists that extend only over the duration of DST—from March 11 through November 3—and add an hour to the sunrise and sunset times. Lines 28–29 get the indices necessary to slice the lists, Line 30 slices the list of dates, Lines 31 and 33 slice the lists of rise and set times, and Lines 32 and 34 add the DST hour to the rise and set times. It sounds more complicated than it is.

Then we start using Matplotlib to make our graph. Lines 37–47 create the plot and add all of the various lines and areas. The fill_between function creates the areas, and the plot function draws the lines. I use the alpha parameter in the fill_between calls to get the shading I wanted and to allow the gridlines to show through the filled-in areas. There was a bit of trial and error to get alphas that made the two DST zones look about the same.

The parts of the plot were labeled in Lines 50–52. The text function puts the given text (third parameter) at the given x- and y-coordinates (first and second parameters). The neat thing about this function is that the coordinates are given in the same units as the data, which is why you see the x-coordinate given as a datetime. The ha parameter is short for “horizontal alignment,” which allowed me to specify the center of the text so it would fall between the vertical gridlines. The rotation values were chosen through trial and error to get the text tilted to match (by eye) their curves.

Lines 55–56 display the gridlines. I set the linewidth to .5 points, but thin lines like that are better for PDF output than PNG.

After setting the font size for all the tick labels in Line 59, the horizontal axis is formatted in Lines 60–64. The horizontal limits were set to run the length of the year, and the tick marks delineate the month boundaries. The date format in Line 62 uses the standard strftime system, and has a bunch of space characters at the beginning to get the month labels to be (more or less) centered in the middle of each month instead of centered under the tick mark at the beginning of the month. There should be a better way to do this, but I haven’t found it.

The vertical axis is formatted in Lines 67–73. Lines 68 and 69 set the major and minor tick marks to be 4 hours and 1 hour apart, respectively. Recall that the rise and set times are just numbers—we can’t use the strftime system Line 70 formats the time labels, which are just numbers, to look like like hours.

Finally, Line 76 gets rid of a lot of the whitespace border that would otherwise surround the plot, and Line 77 saves it to a PNG file. The 10″ by 6″ plot size set back in Line 37, combined with the dpi=150 setting in Line 77, gives us an image that’s 1500×900 pixels.

I did make one small change to the script. As I mentioned in yesterday’s post, I thought the curve and tick labels were too small for the size of the graph as it appears in the blog. I bumped the font size up from 10 to 12 to make the text more legible. Not a huge difference, but a definite improvement.