A small thing

So often it’s the little things. A few months ago, I made a small change in how I pack for overnight trips, and it’s eliminated a nagging doubt I used to have.

I use the Clear Care system to disinfect my contact lenses. Every night, I put the lenses in a plastic holder which then goes into a cup of hydrogen peroxide solution.

Lens holder and cup

The holder is integral with the screw-on cap. The left lens goes in the holder on the white side, the right lens on the blue side. Before putting the holder into the cup and screwing it tight, I fill the cup with solution up to the embossed line. And that’s where the doubt came in.

Clear Care’s travel bottles contain 3 ounces of solution, the maximum allowed by the TSA. The problem is that the bottles are opaque, so after I’d used one a few times, I wouldn’t be sure how much was left in the bottle. I could shake the bottle and kind of tell if there was more or less than half left, but beyond that I had no idea. So I’d often find myself standing in my bathroom while packing for a trip, trying to decide whether there was likely to be enough in the bottle to make it through however many nights I’d be away. More often than not, I’d pack a new bottle just to be sure. The solution wouldn’t go to waste—I’d use the travel bottle at home until it was empty—the waste was of my time and attention.

Eventually, I realized that this was both stupid and easily solved. One day, when I had just replaced an old cup and holder with a new one, I took the old cup into work with me to do some measuring.

One of the advantages of working in a company with a chemist is that she keeps a variety of graduated cylinders in the lab cabinets. I didn’t take long to learn that the contacts cup is 9.6 ml up to the embossed line. As I said, the travel bottle is 3 oz or 90 ml, so in theory, I could get 9 nights of use from the travel bottle. But because I sometimes fill the cup beyond the line—and I really don’t want to find myself at a hotel late at night without enough cleaner—I limit myself to 8 nights per bottle.

What sophisticated technological system do I use to keep track of how much solution I’ve used? I write “8” in on the side of each travel bottle and make a tally mark every time I use it. I keep a blue Sharpie in my shaving kit for this.

Clear Care travel bottle

As long as I can subtract, I’ll have peace of mind.

Apple quarterly sales

I was traveling for business yesterday, so I’m even later than usual with my take on Apple’s sales. By now, I’m sure you’ve already seen the many charts at MacStories and Six Colors, but I still like to post my own. It gives me a chance to try out new ways of showing the data.

I started out plotting only the four-quarter moving averages because I wanted to show how trends in jumpy data can be better shown through smoothing. Later, I included the raw sales figures but tried to keep them from overwhelming the moving averages. This time, I’ve added a faint dashed line to point out the year-over-year changes in the raw sales data. I’m trying to squeeze more useful information into the graphs without making them so cluttered that the main point is lost.

Here are the unit sales for the devices Apple breaks out individually:

Apple sales

The iPhone dominates this graph, especially since 2013, when the iPad peaked and started its three-year slide. To better see what’s going on with the iPad and the Mac, we need to plot them by themselves.

iPad sales

The iPad is still showing modest gains after bottoming out a little over a year ago. As for the Mac…

Mac sales

Ugh. Everyone has pointed out that this quarter had the worst Mac unit sales since 2010. This is true, but as you can see, the June quarter of 2013 was about as bad—3.754 million units compared to the most recent quarter’s 3.740 million. “Worst quarter since waaay back in 2010” makes for a better story.

I don’t think there’s much mystery to the Mac’s poor sales. As Marco Arment said in a series of tweets today, Apple let its notebook line (which drives Mac sales) go fallow for too long, and it’s caught up with them. I see it as being similar to, albeit less dramatic than, what happened to the iPad during its slide. With the iPad, Apple’s neglect was primarily on the software side; with Mac notebooks, there’s been neglect on both the software and hardware sides.

Even if the quality of the Mac is now improving, and there are good reasons to think it is, sales are unlikely to bounce back for few quarters. These are big investments and it takes time for word to spread. Recall that the iPad sales continued to decline for about a year even after the big improvements that came with the iPad Pro and iOS 9. Sales tend to be a trailing indicator of product quality. This is especially true after your indifference to a product line has trained your customers to be skeptical.

Python and platforms

After my last post on the scripts that generate my Apple sales graphs (which I’ll need again when the new quarterly results are posted tomorrow), I was shamed by Rosemary Orchard and Nathan Grigg into rewriting them to run on both macOS and iOS (the latter via Pythonista). It turned out to be pretty easy, with only a couple of additions needed.

I won’t bore you with the entire script, I’ll just bore you with the new stuff that lets the script run on both platforms. There were two changes.

First, since there is no OptiPNG for iOS, this line in the original script,

subprocess.run(['optipng', pngFile])

which processed the just-saved file on my Mac, had to go. The fix, though, wasn’t too much different: just run OptiPNG on the server after the file was uploaded. I already had an SSH connection to the server, so all I had to do was add this line,

sshIn, sshOut, sshErr = ssh.exec_command("optipng '{}'".format(pngDest))

before closing the connection. The exec_command function is part of the paramiko SSH library, and pngDest is a variable that contains the full path on the server to the newly uploaded PNG file.

The second change was really more of an addition than a change. I decided I wanted the script to put the uploaded PNG’s URL on the clipboard to make it easy to paste into a blog post. This is easy to do on the Mac, where you just use subprocess to run the pbcopy command;1 and it’s even easier to do on iOS, where Pythonista includes the handy clipboard library. The problem comes in figuring out which device the script is running on.

(It’s possible this problem has already been solved and is sitting in GitHub repository, but I didn’t find it. Al Sweigart’s pyperclip is a cross-platform library for moving text to and from the clipboard, but the platforms it crosses are the traditional Windows, Mac, and Linux—no iOS.)

At first, I thought calling os.name would give me what I needed, but it returns posix on both macOS and iOS. After a bit of searching, I hit upon the platform library, which I’d never used before. In particular, the platform.platform() function returns the string


on my 2012 iMac at home,


on my 2017 iMac at work,


on my iPad Pro, and


on my iPhone 6s. I don’t understand why the minor Darwin number is one less on the home iMac than it is everywhere else, but it doesn’t matter for what I need here. The key is that Macs have “x86” in the string (at least for now) and it’s a safe bet that iOS devices never will.

Thus, the following section of code at the end of my script:

if 'x86' in platform.platform():
  import subprocess
  subprocess.Popen('pbcopy', stdin=subprocess.PIPE).communicate(pngURL.encode())
  import clipboard

As its name implies, pngURL is a string that contains the URL of the just-uploaded and optimized PNG file. Applying encode to it before sending it to communicate is necessary because communicate takes only bytes.

Oh, there is one more thing. I used to keep the Apple sales scripts and data files in a Dropbox folder, but now I have them in a folder on iCloud Drive. Pythonista’s external files feature can link to individual Dropbox files, but I couldn’t get it to link to a folder, and I kept getting errors whenever a script on Dropbox tried to save a file. Both of these problems disappeared when I moved everything to iCloud.2

So thanks to Rosemary and Nathan for the nudge. I didn’t think I wanted an iOS-resident solution, but now that I have it, it seems pretty neat.

  1. That link should go to Apple’s online man page for the pbcopy command, but Apple has decided either to delete its man pages or move them where neither I nor Google can find them. Have I complained about this before? Yes, and I’m complaining about it again. 

  2. I’ve been thinking about switching entirely from Dropbox to iCloud, but that’s a topic for another post. 

Plotting my Apple sales plots

Last week, Jason Snell wrote a nice post on how he automated his system for producing and uploading the many charts he makes whenever Apple posts its quarterly results. It’s a Mac-centric system, based on Numbers, Image Magick, SCP1, and Automator. A few days later, in a heroic Twitter thread, Jason took up the challenge of creating an iOS workflow that did the same thing. I’ve been expecting a blog post with full writeup, but there’s been nothing so far. Maybe he’s cleaning it up around the edges.2

I am not heroic. The creation of my charts is automated automatically (so to speak) because they come from Python scripts. And I can generate them from iOS through the simple expedient of using Prompt to log into my Mac from my iPad or iPhone and then typing the name of the necessary script at the command line.

What’s that? You don’t think logging onto a Mac and running a Terminal command is “generat[ing] them from iOS”? Well, I’ve already told you I’m not heroic; you shouldn’t be surprised that I’m also a cheater.

Cheater or not, I was inspired by Jason to add a few things to my Apple chart-making scripts. Two of them, reducing the size of the graphic via OptiPNG and uploading the resulting file through STFP, were things I used to do at the command line after making a chart. Another was analytic: to the raw sales and four-quarter moving average, I added a companion line that tracks the year-over-year sales associated with the current quarter. People seem to like year-over-year data, and it was easy to include. Finally, I thought the branding joke of putting my giant snowman head in a prominent place of every graph had worn thin, so I made the head much smaller and tucked it into a less conspicuous spot.

The result, still using the results reported three months ago, looks like this:

Apple sales

The year-over-year tracking is done by making those raw sales dots slightly bigger than the others and connecting them with a thin, faint, and dashed line. My goal was to maintain the prominence of the moving average while making easier to see the year-over-year changes. One thing I’d never noticed before was that the Jan-Mar quarter used to have above average iPhone sales but hasn’t since 2015.

The plot above was made to compare the three product lines. Because the iPhone sets the plot’s scale, it also serves as a decent plot of the iPhone itself. But it’s terrible at showing the evolution of iPad and Mac sales, so I also make individual plots for them.

iPad sales

Mac sales

The data is kept in files that look like this,

2016-Q1 2015-12-26  74.779
2016-Q2 2016-03-26  51.193
2016-Q3 2016-06-25  40.399
2016-Q4 2016-09-24  45.513
2017-Q1 2016-12-31  78.290
2017-Q2 2017-04-01  50.763
2017-Q3 2017-07-01  41.026
2017-Q4 2017-09-30  46.677
2018-Q1 2017-12-30  77.316
2018-Q2 2018-03-31  52.217

with one line per quarter, with the quarter’s name, end date, and sales separated by whitespace. There’s a file like this for each of the devices.

The script I use to make the first plot is this:

  1:  #!/usr/bin/env python
  3:  from datetime import date, datetime
  4:  from sys import stdin, argv, exit
  5:  import numpy as np
  6:  import matplotlib.pyplot as plt
  7:  import matplotlib.dates as mdates
  8:  from matplotlib.ticker import MultipleLocator
  9:  from PIL import Image
 10:  import paramiko
 11:  import subprocess
 13:  # Initialize
 14:  phoneFile = 'iphone-sales.txt'
 15:  padFile = 'ipad-sales.txt'
 16:  macFile = 'mac-sales.txt'
 17:  firstYear = 2010
 18:  today = date.today()
 19:  baseFile = today.strftime('%Y%m%d-Apple sales')
 20:  pngFile = baseFile + '.png'
 21:  pdfFile = baseFile + '.pdf'
 22:  dest = '/path/to/images{}/{}'.format(today.strftime('%Y'), pngFile)
 24:  # Read the given data file and return the series.
 25:  def getSeries(fname):  
 26:    global lastYear, lastMonth
 27:    dates = []
 28:    sales = []
 29:    for line in open(fname):
 30:      if line[0] == '#':
 31:        continue
 32:      quarter, edate, units = line.strip().split('\t')
 33:      units = float(units)
 34:      qend = datetime.strptime(edate, '%Y-%m-%d')
 35:      dates.append(qend)
 36:      sales.append(units)
 37:    ma = [0]*len(sales)
 38:    for i in range(len(sales)):
 39:      lower = max(0, i-3)
 40:      chunk = sales[lower:i+1]
 41:      ma[i] = sum(chunk)/len(chunk)
 42:    return dates, sales, ma
 44:  # Make new series with just the latest quarter for every year.
 45:  def getYoY(d, s):
 46:    dyoy = list(reversed(d[::-4]))
 47:    syoy = list(reversed(s[::-4]))
 48:    return dyoy, syoy
 50:  # Read in the data
 51:  phoneDates, phoneRaw, phoneMA = getSeries(phoneFile)
 52:  padDates, padRaw, padMA = getSeries(padFile)
 53:  macDates, macRaw, macMA = getSeries(macFile)
 54:  phoneDatesYoY, phoneRawYoY = getYoY(phoneDates, phoneRaw)
 55:  padDatesYoY, padRawYoY = getYoY(padDates, padRaw)
 56:  macDatesYoY, macRawYoY = getYoY(macDates, macRaw)
 58:  # Tick marks and tick labels
 59:  y = mdates.YearLocator()
 60:  m = mdates.MonthLocator(bymonth=[1,
 61:   4, 7, 10])
 62:  yFmt = mdates.DateFormatter('                %Y')
 63:  ymajor = MultipleLocator(10)
 64:  yminor = MultipleLocator(2)
 66:  # Plot the raw sales data and moving averages.
 67:  # Connect the year-over-year raw data.
 68:  fig, ax = plt.subplots(figsize=(8,6))
 69:  ax.plot(phoneDates, phoneMA, '-', color='#7570b3', linewidth=3, label='iPhone')
 70:  ax.plot(phoneDates, phoneRaw, '.', color='#7570b3')
 71:  ax.plot(phoneDatesYoY, phoneRawYoY, '.', color='#7570b3', markersize=8)
 72:  ax.plot(phoneDatesYoY, phoneRawYoY, '--', color='#7570b3', linewidth=1, alpha=.25)
 73:  ax.plot(padDates, padMA, '-', color='#d95f02', linewidth=3, label='iPad')
 74:  ax.plot(padDates, padRaw, '.', color='#d95f02')
 75:  ax.plot(padDatesYoY, padRawYoY, '.', color='#d95f02', markersize=8)
 76:  ax.plot(padDatesYoY, padRawYoY, '--', color='#d95f02', linewidth=1, alpha=.25)
 77:  ax.plot(macDates, macMA, '-', color='#1b9e77', linewidth=3, label='Mac')
 78:  ax.plot(macDates, macRaw, '.', color='#1b9e77')
 79:  ax.plot(macDatesYoY, macRawYoY, '.', color='#1b9e77', markersize=8)
 80:  ax.plot(macDatesYoY, macRawYoY, '--', color='#1b9e77', linewidth=1, alpha=.25)
 82:  # Add a grid.
 83:  ax.grid(linewidth=1, which='major', color='#dddddd', linestyle='-')
 85:  # Set the upper and lower limits to show all of the last year in the data set.
 86:  # Add a year if the sales are for the last calendar quarter.
 87:  lastYear = macDates[-1].year
 88:  lastMonth = macDates[-1].month
 89:  if lastMonth == 12:
 90:    lastYear += 1
 91:  plt.xlim(xmin=date(firstYear, 1, 1), xmax=date(lastYear, 12, 31))
 92:  plt.ylim(ymin=0, ymax=80)
 94:  # Set the labels
 95:  plt.ylabel('Unit sales (millions)')
 96:  plt.xlabel('Calendar year')
 97:  t = plt.title('Raw sales and four-quarter moving averages')
 98:  t.set_y(1.03)
 99:  ax.xaxis.set_major_locator(y)
100:  ax.xaxis.set_minor_locator(m)
101:  ax.xaxis.set_major_formatter(yFmt)
102:  ax.yaxis.set_minor_locator(yminor)
103:  ax.yaxis.set_major_locator(ymajor)
104:  ax.set_axisbelow(True)
105:  plt.legend(loc=(.08, .72), borderpad=.8, fontsize=12)
106:  fig.set_tight_layout({'pad': 1.5})
108:  # Save the plot file as a PNG and as a PDF.
109:  plt.savefig(pngFile, format='png', dpi=200)
110:  plt.savefig(pdfFile, format='pdf')
112:  # Add the logo to the PNG and optimize it.
113:  plot = Image.open(pngFile)
114:  head = Image.open('snowman-head.jpg')
115:  smallhead = head.resize((60, 60), Image.ANTIALIAS)
116:  plot.paste(smallhead, (1496, 26))
117:  plot.save(pngFile)
118:  subprocess.run(['optipng', pngFile])
120:  # Upload the PNG
121:  ssh = paramiko.SSHClient()
122:  ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
123:  ssh.connect(hostname='host.com', username='user', password='password', port=6789)
124:  sftp = ssh.open_sftp()
125:  sftp.put(pngFile, dest)

Much of the script has been explained in that earlier post. Here, I’ll just discuss the new stuff.

The getYoY function in Lines 45–48 uses slices to create the lists of dates and sales for the year-over-year subset of the full data. The slice itself, [::-4], works backward from the end of the list, so I included the reversed function to put the year-over-year lists in chronological order. This isn’t necessary for plotting, but I figured future me would expect all of the lists to be in the same order if he was going to do something else with them.

Lines 71, 75, and 79 plot the year-over-year quarters with a slightly larger dot. This dot goes over and hides the raw data dots that are plotted in Lines 70, 74, and 78. Lines 72, 76, and 80 plot thin (linewidth=1), faint (alpha=.25), dashed ('--') lines connecting the year-over-year dots.

After the plot is saved as a PNG (Line 101), I use the Python Imaging Library to add my head near the upper right corner (Lines 113–117). Then I run the file through OptiPNG via the subprocess library.

Finally, in Lines 121–125, I use the paramiko library to establish an SSH connection to the server and upload the file via SFTP. The file name uses the current date as a prefix (Line 19), and the destination directory on the server is named according to the year: imagesyyyy. This path is set in Line 22.

You’ll note that in addition to the PNG file, the script also generates a PDF. I almost never use the PDF, but if I want to annotate the plot before publishing it, I get better results if I annotate the PDF in OmniGraffle and then export the OG file as a PNG.

As I said, this script produces the plot with sales of all three devices. The individual iPad and Mac plots are produced with similar scripts that import only one device’s sales. I should also mention that this script has evolved over the past three years, and it’s likely to contain lines of code that no longer do anything but haven’t yet been deleted. In addition to being a coward and a cheat, I’m also lazy.

The results for the Apr-Jun quarter are going to be posted next Tuesday. I’ll be traveling that day and probably won’t be posting updated plots until the day after. Until then, you’ll just have to get by with charts from Jason and the other usual suspects.

  1. If you’re wondering why I’m linking to a generic Unix man page site instead of to Apple, it’s because Apple has either taken its man pages offline or changed their fucking URLs again

  2. Sneakily, Jason didn’t write a new post, he updated the original with the iOS stuff. Scroll down to the bottom to see it.