# Keeping up with IP number changes

Last weekend I was at a hotel working on my iPad and launched Prompt to connect to my iMac at home and run a command. It wouldn’t connect. I launched Transmit (which still works well, despite its zombie status) and couldn’t connect there, either. Obviously, my service provider had changed my home IP number. This happens rarely,1 but I didn’t want to be caught like this again.

My first thought was to write a script that would check the IP number periodically and save it in a file on the leancrew.com server. But a bit of searching led me to this script by Charlie Hawker, which checks for the IP number and sends an email if it’s changed, a much better idea.2

There were, however, two things about the script I didn’t like:

1. It’s written in PHP. I don’t disklike PHP—I use it to build this blog—but it’s not a language I’m deeply familiar with, and I’d prefer a Python script so I can easily make changes if necessary.
2. It gets the IP number by loading a website (ip6.me) and scraping it. I wanted something a little more direct.

The solution to the second problem was to use dig, the domain information groper (a program that was undoubtedly named by a man). Here’s a command that does it by returning information from OpenDNS3

dig +short myip.opendns.com @resolver1.opendns.com


With that command in mind, the script, checkip, was easy to write:

python:
1:  #!/usr/bin/python
2:
3:  import smtplib
4:  import subprocess as sb
5:  import os
6:  import sys
7:
8:  # Parameters
9:  ipFile = '{}/.ipnumber'.format(os.environ['HOME'])
10:  mailFrom = 'user@gmail.com'
11:  mailTo = 'user@gmail.com'
12:  gmailUser = 'user'
14:  cmd = '/usr/bin/dig +short myip.opendns.com @resolver1.opendns.com'
15:  msgTemplate = '''From: {}
16:  To: {}
17:  Subject: Home IP number has changed
18:  Content-Type: text/plain
19:
20:  {}
21:  '''
22:
23:  # Get the current IP number and the filed IP number
24:  dig = sb.check_output(cmd.split(), universal_newlines=True)
25:  currentIP = dig.strip()
26:  with open(ipFile, 'r') as ip:
28:
29:  # Update the file and send an email if they differ
30:  if currentIP != oldIP:
31:    with open(ipFile, 'w') as ip:
32:      ip.write(currentIP)
33:
34:    msg = msgTemplate.format(mailFrom, mailTo, currentIP)
35:    smtp = smtplib.SMTP_SSL('smtp.gmail.com', 465)
36:    smtp.ehlo()
38:    smtp.sendmail(mailFrom, mailTo, msg)


Lines 9–21 set the parameters that are used later in the script. As you can see from Line 9, I save the IP number in the hidden file .ipnumber in my home directory. Line 14 is the dig command we just talked about. Lines 15–21 is the template for the email that the script sends when the IP number change, and Lines 10–13 are the personal settings needed to send the email.

Lines 24–25 run the dig command4 and store its result (after stripping the trailing newline) in the currentIP variable. Lines 26–27 read the .ipnumber file, which contains the IP number from the last time the script was run, and store the contents in oldIP.

Line 30 then compares the two numbers and continues the script only if they differ. Lines 31–32 puts the new IP number into the .ipnumber file, and Lines 34–38 build an email message and send it via GMail. This script works only if you have a GMail account—if you don’t, you’ll have to tweak these lines to get your email service to send the message.

The email it sends is nothing fancy: a subject line telling me the IP number has changed and a body with the new number (which I’ve obscured in this screenshot).

To avoid an error the first time this script is run, create the .ipnumber file and put any old IP number in it. I seeded mine with an incorrect value, 11.11.11.11, so I could make sure the file changed and the email was sent when the script ran.

I decided to have the script run every hour. If I were using Linux, I’d set this up through cron, but the (more complicated) Mac way is with launchd. Here’s the plist file that controls the running of the script. It’s called com.leancrew.checkip.plist and is saved in my ~/Library/LaunchAgents directory:

xml:
1:  <?xml version="1.0" encoding="UTF-8"?>
2:  <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
3:  <plist version="1.0">
4:  <dict>
5:    <key>Label</key>
6:    <string>com.leancrew.checkip</string>
7:    <key>ProgramArguments</key>
8:    <array>
9:      <string>/Users/drang/Dropbox/bin/checkip</string>
10:    </array>
11:    <key>StartCalendarInterval</key>
12:    <array>
13:      <dict>
14:        <key>Minute</key>
15:        <integer>12</integer>
16:      </dict>
17:    </array>
18:  </dict>
19:  </plist>


The key entries here are Line 9, which gives the full path to the checkip script, and Lines 11–17, which tell launchd to run the script at 12 minutes past the hour.5 Because there are no Hour, Day, or Month entries in this section, it runs every hour of every day of every month.

Because of where it’s stored, this Launch Agent gets loaded into the launchd system every time I log in. To get it to load without logging out and logging back it, I ran these two commands:6

launchctl load ~/Library/LaunchAgents/com.leancrew.checkip.plist
launchctl start com.leancrew.checkip


With it in the system, the agent will continue to run as long as I’m logged in. For me, this is essentially forever, as I keep this computer running all the time specifically so I can log into it whenever I need to.

I’ve thought of improving the script by adding an error-handling section to alert me when the dig command fails. This would let me know, for example, if OpenDNS has stopped running its service. But the possibility of being inundated by false positive emails has kept me from trying that out.

1. I managed to get the command run by connecting to my work computer, where the IP number is fixed.

2. Yes, I know there are services that will do this sort of thing. I don’t want to rely on them.

3. This does rely on OpenDNS, but if it ever shuts down or curtails this service, there will always be another similar way to get my IP number. I’ll just change my script to use that instead.

4. Note: I typically write scripts in Python 3 nowadays, where the subprocess module is considerably different. This script was easier to do in the Python 2 that comes with macOS because it gets run by the launchd system, which uses an environment that doesn’t know where my installation of Python 3 is. Rather than telling it, I decided to just write it for Python 2. You could call this the $PATH of least resistance. 5. Why 12 minutes? No particular reason. Any number under 60 would work just as well. 6. The load and start subcommands are now considered legacy features of launchctl and will probably be removed in the future. But for now they still work, so I’ll worry about learning the new subcommands later. # Building a tweet in Shortcuts I occasionally tweet links to Kindle books that are on sale. Although I do include my Amazon Associate tag in the link so I get a commission on sales, this is not what you would call a significant money maker. The commission on Kindle books is 4%, which amounts to 8¢ on the typical$2 sale price. I do it because I think the people who follow me might be interested in the book.1

In the past, these tweets were purely functional: such-and-such book is on sale for \$2—here’s a link to it. I’d use the Associate app—actually its Share Sheet extension—to get the link. A couple of days ago, I decided it would be fun to jazz up the tweet with an image of the book. I already had some Python code that could use Amazon’s product advertising API to get the URL of a book image; I figured it wouldn’t be too hard to jigger with it to pull out the author and title, too, and combine it with a Shortcuts to build a tweet.

Here’s how it works. While I’m on the web page for the book, I invoke Shortcuts from the Share Sheet and choose the Tweet Kindle Book shortcut. This launches Pythonista to extract the relevant information from the Amazon API. At this point, the shortcut gets stuck with my screen showing Pythonista. To get it unstuck, I tap the ◀︎Safari link in the top left corner.

This little kick to get the shortcut going again is needed because Pythonista is not a particularly good iOS automation citizen. It has a URL scheme, which is what allows Shortcuts to invoke it, but it doesn’t implement callbacks, so it won’t automatically return to the app that called it. Pythonista does have ways to jump to other apps, but I haven’t figured out a way to get it to automatically continue this shortcut.

Anyway, after unsticking the shortcut, it assembles the book information into a tweet that I can edit and send.

Here are the steps of the shortcut.

The Python script (which we’ll get to in minute) takes the URL of the page being shared as input and puts a JSON dictionary of the book information on the clipboard. For this example, the JSON is

{"title": "Provenance",
"author": "Ann Leckie",
"imgURL": "https://images-na.ssl-images-amazon.com/images/I/51EiO-mWOtL.jpg"


The Get Dictionary from Input step parses the JSON and turns it into a Shortcuts dictionary, from which we can extract the values. The first thing we do is get the URL of the book cover image and use Get Contents of URL to get the image itself. The image is saved in the variable Tweet.

Next, we assemble the text of the tweet from the other parts of the dictionary and a few linking words. This text is added to the Tweet variable (yes, you can add text to a variable that contains an image—I was surprised at that, too) and the whole thing is finally sent off to the Tweet step. You’ll need to have the official Twitter app installed for the Tweet step to be available.

The Python script, amazon-tweet-info.py, is this:

python:
1:  import requests
2:  from datetime import datetime
3:  import base64, hashlib, hmac
4:  import urllib.parse
5:  from bs4 import BeautifulSoup
6:  import clipboard
7:  import sys
8:  import re
9:  import json
10:
11:  # Get the Amazon URL from the first argument and extract the ASIN
12:  amzURL = sys.argv[1]
13:  itemASIN = re.search(r'(dp/|gp/product/)([^/?%]+)', amzURL).group(2)
14:
15:  # Date and time
16:  t = datetime.utcnow()
17:  timeStamp = urllib.parse.quote(t.strftime('%Y-%m-%dT%H:%M:%SZ'))
18:
19:  # Parameters
20:  associateTag = 'yourtag'
21:  accessKey = 'youraccesskey'
22:  secretKey = b'yourlongsecretkey'
23:  parameters = ['Service=AWSECommerceService',
24:     'Operation=ItemLookup',
25:     'ResponseGroup=Large',
26:     'ItemId={}'.format(itemASIN),
27:     'AWSAccessKeyId={}'.format(accessKey),
28:     'AssociateTag={}'.format(associateTag),
29:     'Timestamp={}'.format(timeStamp)]
30:  parameters.sort()
31:  paramString = '&'.join(parameters)
32:
33:  # Generate signature from parameters and secret key
34:  unsignedString = '''GET
35:  webservices.amazon.com
36:  /onca/xml
37:  {}'''.format(paramString)
38:  signedString = hmac.new(secretKey, msg=unsignedString.encode(), digestmod=hashlib.sha256).digest()
39:  sig = urllib.parse.quote(base64.b64encode(signedString))
40:
41:  # Generate URL from parameters and signature
42:  url = 'http://webservices.amazon.com/onca/xml?{}&Signature={}'.format(paramString, sig)
43:
44:  # Get image information
45:  resp = requests.get(url)
46:  xml = resp.text
47:
48:  # Extract the information from from the XML
49:  # response and build a dictionary
50:  soup = BeautifulSoup(xml, 'html5lib')
51:  info = {}
52:  info['imgURL'] = soup.item.largeimage.url.text
53:  info['author'] = soup.item.itemattributes.author.text
54:  info['title'] = soup.item.itemattributes.title.text
56:
57:  clipboard.set(json.dumps(info))


The first item given to it on the “command line” (which in Shortcuts is what flows into the Run Script step) is expected to be the URL of the web page for the book. Lines 12–13 slurp it in and extract its unique Amazon ID, the ASIN.

Lines 16–17 get the current date and time and format it for later use with the API. Lines 20–31 put together all the data needed to make an API request. The associateTag is what you get from Amazon when you sign up for the Associates program. It’s attached to the link URLs that earn you a commission. The accessKey and secretKey are ID strings you get when you sign up for the Product Advertising API. The secret key is defined as a byte string rather than a normal Python 3 string because the encoding that’s part of the signing process (see below) requires a byte string.

Your request to the API has to be “signed” by encoding the request with your secret key. That’s done in Lines 34–39. The details of this are a little hairy, but Amazon explains them pretty well. Line 42 puts everything together into a URL to send to the API.

An important part of the request is the ResponseGroup parameter. There are many response groups available for plucking out different bits of information on a product. As best I could tell from the documentation, the only one that provides all the pieces I want is the Large group. That’s why you see it on Line 25.

Lines 45–46 use the Requests module to send the request URL to the API and get the response, which is in XML format. Lines 50–54 use the Beautiful Soup module to parse the XML and build a Python dictionary with the author, title, and image URL. Line 55 uses the ASIN and the associate tag to create the link entry in the dictionary.

Finally, Line 57 uses the JSON module to convert the Python info dictionary into JSON format and put it on the clipboard. This is the result of the first step of the Tweet Kindle Book shortcut.

This was a relatively simple shortcut to put together, mainly because I already had a Python script written to get the image URL. I just had to figure out which response group also included the author and title data and where they fit within the XML hierarchy.

I’m still a little annoyed about the pause at the Pythonista step. Maybe there’s a trick I don’t know about to get it to return to the shortcut. Or maybe I should consider rewriting that step in JavaScript so I can use Scriptable, which I think is more suited for use as a Shortcuts step.

1. If I’m buying the linked book myself because I don’t already own it, it takes 25 book sales for me to break even. Given how cheap frugal discerning my Twitter followers are, that many sales is rare.

# Photo location followup

There’s always a better way to do it. Shortly after I posted my Photo Coordinates shortcut, professional automator Rosemary Orchard tweeted me a link to a cleaner shortcut that uses the Get Details of Images step.

As you can see, after extracting the Location property of Get Details of Images, you can further extract the Latitude and Longitude properties. This got me thinking that I could reduce the shortcut to a single step by treating the Shortcut Input magic variable as a Location, not as Photo Media, and then extracting the latitude and longitude from that.

This is a great idea, except it doesn’t work. Presumably, an image can’t be treated like a location because a location is a property of an image—and you can’t jump two levels down the hierarchy in one step.

But it does show I could have written my original shortcut more quickly by taking the Location property of the image instead of the Metadata Dictionary, and then extracting the Latitude and Longitude.

This is no shorter than my original, but if I’d done it this way I wouldn’t have had to screw around with the nested JSON in the Metadata Dictionary. Rosemary’s is still shorter, of course, and easier to read because it doesn’t create an unnecessary variable.

On the other hand, looking into the Metadata Dictionary taught me some things about the GPS metadata in phone images that I’ve never seen in the metadata on standard cameras and that I wouldn’t have learned if I’d done it the easy way right from the start. At some point, I may make use of that. In particular, the direction the camera was pointing when the photo was taken seems like it would be nice to know in some situations.

# Drawing the quarters

As you probably heard on Twitter, or from the usual sources, Apple announced that today’s quarterly figures will be the last to include unit sales for the Mac, iPhone, and iPad. I’ve been posting plots of units sales since April of 2015, when I shook up the staid world of Apple punditry by introducing it to the four-quarter moving average. It’s a sad day here at Drang Global Headquarters as we post the last of its kind:

The dots are the raw sales figures, the thick solid lines are the four-quarter moving averages, and the thin dashed lines connect the same quarters in consecutive years, giving a sense of the year-over-year changes. You can see why Apple isn’t particularly interested in showing these figures anymore—you have to get out a magnifying glass to see any recent growth.

Oddly enough, back in August, after the last set of quarterly results came out, I decided it was time to start plotting revenue. Not because I had any inkling that Apple would stop providing unit sales, but because I wanted to compare these three products to Services and Other Products, the two remaining categories in Apple’s quarterly reports. So I dug back through the reports and added revenue to my data files for the Mac, iPhone, and iPad. This data update and a few small edits to my plotting script allowed me to quickly produce a new version of the chart above, but with this one focused on revenue:

You can see why Apple prefers to talk revenue.

The dominance of the iPhone makes it hard to see the Mac and the iPad down there in the mud, so let’s show them all by themselves:

(Stephen Hackett, feel free to use this the next time your co-hosts talk about the fading Mac platform. Do it quickly, though, as I suspect the iPad will rebound next quarter.)

Of course, in true best-laid-plans style, I didn’t get around to making data files for Services and Other Products, so I can’t make the plots I intended to. Next time.