Next post Previous post

Blogging from BBEdit, redux

June 22, 2014 at 10:41 PM by Dr. Drang

I’ve been having blog problems the past couple of weeks. My script for publishing from BBEdit has been working fine when publishing new posts, but it’s been throwing errors when updating posts after editing. I’m not sure what has changed. I did update my WordPress installation recently, but I’m also suspicious of new configurations my webhost may have implemented. Whatever the cause of the problem, I decided to rewrite the script to change how it used the XML-RPC MetaWeblog API. It seems to be working consistently now.

The errors using the old script (initial version described here, slight improvement described here) traced back to the server returning an HTTP 302 code when accessed through Python’s xmlrpclib module. I wondered if the problem was due to the underlying urllib calls. I decided to do my own XML-RPC calls via Kenneth Reitz’s requests module, which I knew would follow redirects.

It took a bit of messing around, but with the help of Dave Winer’s MetaWeblog RFC, this article at OpenGroupware, and this one at O’Reilly, I finally pieced together all the parts and got the script working. Despite being written at a distinctly lower level than the original,1 it’s only about 20 lines longer. It’s still called Publish Post, and I still have it saved as a BBEdit Text Filter, accessible through the Text‣Apply Text Filter‣Publish Post menu item. Here’s the source code, anonymized in the places you’d expect:

python:
  1:  #!/usr/bin/python
  2:  # -*- coding: utf-8 -*-
  3:  
  4:  import xmlrpclib
  5:  import sys
  6:  from datetime import datetime, timedelta
  7:  import time
  8:  import pytz
  9:  import keyring
 10:  import subprocess
 11:  import tweepy
 12:  import requests
 13:  
 14:  '''
 15:  Take text from standard input in the format
 16:  
 17:    Title: Blog post title
 18:    Keywords: key1, key2, etc
 19:  
 20:    Body of post after the first blank line.
 21:  
 22:  and publish it to my WordPress blog. Return in standard output
 23:  the same post after publishing. It will then have more header
 24:  fields (see hFields for the list) and can be edited and re-
 25:  published again and again.
 26:  
 27:  The goal is to work the same way TextMate's Blogging Bundle does
 28:  but with fewer initial headers.
 29:  '''
 30:  
 31:  
 32:  ####################### Parameters ########################
 33:  
 34:  # The blog's XMLRPC URL and username.
 35:  url = 'http://leancrew.com/path/to/xmlrpc.php'
 36:  user = 'username'
 37:  
 38:  # Time zones. WP is trustworthy only in UTC.
 39:  utc = pytz.utc
 40:  myTZ = pytz.timezone('US/Central')
 41:  
 42:  # The header fields and their metaWeblog synonyms.
 43:  hFields = [ 'Title', 'Keywords', 'Date', 'Post',
 44:              'Slug', 'Link', 'Status', 'Comments' ]
 45:  wpFields = [ 'title', 'mt_keywords', 'date_created_gmt',  'postid',
 46:               'wp_slug', 'link', 'post_status', 'mt_allow_comments' ]
 47:  h2wp = dict(zip(hFields, wpFields))
 48:  
 49:  
 50:  ######################## Functions ########################
 51:  
 52:  def makeContent(header, body):
 53:    "Make the content dict from the header dict."
 54:    content = {}
 55:    for k, v in header.items():
 56:      content.update({h2wp[k]: v})
 57:    content.update(description=body)
 58:    return content
 59:  
 60:  def xmlrpc(url, call, params):
 61:    "Send an XMLRPC request and return the response as a dictionary."
 62:    payload = xmlrpclib.dumps(params, call)
 63:    resp = requests.post(url, data=payload)
 64:    return xmlrpclib.loads(resp.content)[0][0]
 65:  
 66:  def tweetlink(text, url):
 67:    "Tweet a link to the given URL. Return the URL to the tweet."
 68:  
 69:    # Authorize Twitter and establish a connection.
 70:    auth = tweepy.OAuthHandler('theAPIKey',
 71:             'theAPISecret')
 72:    auth.set_access_token('theAccessToken',
 73:             'theAccessTokenSecret')
 74:    api = tweepy.API(auth)
 75:  
 76:    # How long will the shortened URL to the post be?
 77:    short_length = api.configuration()['short_url_length']
 78:    
 79:    # Don't include the snowman if the tweet is about Afghanistan.
 80:    if "Afghanistan" in text:
 81:      prefix = u''
 82:    else:
 83:      prefix = u'⛄ '
 84:  
 85:    # Construct the tweet.
 86:    max_text = 140 - short_length - (len(prefix) + 1)
 87:    if len(text) > max_text:
 88:      text = prefix + text[:max_text-1] + u'…'
 89:    else:
 90:      text = prefix + text
 91:    tweet = '''{}
 92:  {}'''.format(text.encode('utf-8'), url)
 93:  
 94:    # Send the tweet.
 95:    out = api.update_status(tweet)
 96:    return 'https://twitter.com/drdrang/status/%s' % out.id_str
 97:  
 98:  
 99:  ###################### Main program #######################
100:  
101:  # Read and parse the source.
102:  source = sys.stdin.read()
103:  header, body = source.split('\n\n', 1)
104:  header = dict( [ x.split(': ', 1) for x in header.split('\n') ])
105:  
106:  # The publication date may or may not be in the header.
107:  if 'Date' in header:
108:    # Get the date from the string in the header.
109:    dt = datetime.strptime(header['Date'], "%Y-%m-%d %H:%M:%S")
110:    dt = myTZ.localize(dt)
111:    header['Date'] = xmlrpclib.DateTime(dt.astimezone(utc))
112:    # Articles for later posting shouldn't be tweeted.
113:    tweetit = False
114:  else:
115:    # Use the current date and time.
116:    dt = myTZ.localize(datetime.now())
117:    header.update({'Date': xmlrpclib.DateTime(dt.astimezone(utc))})
118:    # Articles for posting now should be tweeted.
119:    tweetit = True
120:  
121:  # Get the password from Keychain.
122:  pw = keyring.get_password(url, user)
123:  
124:  # Connect and upload the post.
125:  blog = xmlrpclib.Server(url)
126:  
127:  # It's either a new post or an old one that's been revised.
128:  if 'Post' in header:
129:    # Revising an old post.
130:    postID = int(header['Post'])
131:    del header['Post']
132:    content = makeContent(header, body)
133:    
134:    # Upload the edited post.
135:    params = (postID, user, pw, content, True)
136:    unused = xmlrpc(url, 'metaWeblog.editPost', params)
137:    
138:    # Download the edited post.
139:    params = (postID, user, pw)
140:    post = xmlrpc(url, 'metaWeblog.getPost', params)
141:    
142:  else:
143:    # Publishing a new post.
144:    content = makeContent(header, body)
145:    
146:    # Upload the new post and get its ID.
147:    params = (0, user, pw, content, True)
148:    postID = xmlrpc(url, 'metaWeblog.newPost', params)
149:    postID = int(postID)
150:    
151:    # Download the new post.
152:    params = (postID, user, pw)
153:    post = xmlrpc(url, 'metaWeblog.getPost', params)
154:    
155:    if tweetit:
156:      tweetURL = tweetlink(post[h2wp['Title']], post[h2wp['Link']])
157:  
158:  # Print the header and body.
159:  header = ''
160:  for f in hFields:
161:    if f == 'Date':
162:      # Change the date from UTC to local and from DateTime to string.
163:      dt = datetime.strptime(post[h2wp[f]].value, "%Y%m%dT%H:%M:%S")
164:      dt = utc.localize(dt).astimezone(myTZ)
165:      header += "%s: %s\n" % (f, dt.strftime("%Y-%m-%d %H:%M:%S"))
166:    else:
167:      header += "%s: %s\n" % (f, post[h2wp[f]])
168:  print header.encode('utf8')
169:  print
170:  print post['description'].encode('utf8')
171:  
172:  # Open the published post in the default browser after a delay.
173:  time.sleep(3)
174:  subprocess.call(['open', post['link']])

I’m not going to explain how I use the script or most of its internal workings. That was covered in the two earlier blog posts. What I do want to talk about is the new xmlrpc function defined in Lines 60–64 and used four times in Lines 128–156.

The three arguments to xmlrpc are

  1. The URL of the blog’s xmlrpc.php file. This is defined in Line 35.
  2. The name of the MetaWeblog API call. This will be either metaWeblog.newPost, metaWeblog.editPost, or metaWeblog.getPost, depending on where we are in the script.
  3. A tuple of parameters that make up the payload of information that’s POSTed in the MetaWeblog API call.

What xmlrpc does is

  1. Wrap the parameters and the API call up in an XML structure using the xmlrpclib.dumps convenience function.
  2. POST the XML structure to the WordPress xmlrpc.php using requests.
  3. Unwrap the content of the XML response into a Python dictionary using the xmlrpclib.loads convenience function. This is what xmlrpc returns.

This is, I believe, pretty much what method calls to an xmlrpclib.ServerProxy object do, except that this function takes advantage of the magic Reitz built into requests. Once I had xmlrpc written, it was easy to replace the ServerProxy method calls that had been triggering the 302 errors.2

It’s been a week of rewriting old scripts and workflows: photo mapping, expense reports, invoicing emails, Markdown table formatting, and now blog posting. Maintenance is important, but I’d like to move on to something new.

  1. You might say it’s closer to the MetaWeblog. ↩

  2. I am by no means a web programmer, but here I am playing one on the internet. Frightening. ↩

Next post Previous post