Next post Previous post

Blogging from BBEdit, redux

June 22, 2014 at 10:41 PM by Dr. Drang

I’ve been having blog problems the past couple of weeks. My script for publishing from BBEdit has been working fine when publishing new posts, but it’s been throwing errors when updating posts after editing. I’m not sure what has changed. I did update my WordPress installation recently, but I’m also suspicious of new configurations my webhost may have implemented. Whatever the cause of the problem, I decided to rewrite the script to change how it used the XML-RPC MetaWeblog API. It seems to be working consistently now.

The errors using the old script (initial version described here, slight improvement described here) traced back to the server returning an HTTP 302 code when accessed through Python’s xmlrpclib module. I wondered if the problem was due to the underlying urllib calls. I decided to do my own XML-RPC calls via Kenneth Reitz’s requests module, which I knew would follow redirects.

It took a bit of messing around, but with the help of Dave Winer’s MetaWeblog RFC, this article at OpenGroupware, and this one at O’Reilly, I finally pieced together all the parts and got the script working. Despite being written at a distinctly lower level than the original,

1 it’s only about 20 lines longer. It’s still called Publish Post, and I still have it saved as a BBEdit Text Filter, accessible through the Text‣Apply Text Filter‣Publish Post menu item. Here’s the source code, anonymized in the places you’d expect:

  1  #!/usr/bin/python
  2  # -*- coding: utf-8 -*-
  3  
  4  import xmlrpclib
  5  import sys
  6  from datetime import datetime, timedelta
  7  import time
  8  import pytz
  9  import keyring
 10  import subprocess
 11  import tweepy
 12  import requests
 13  
 14  '''
 15  Take text from standard input in the format
 16  
 17    Title: Blog post title
 18    Keywords: key1, key2, etc
 19  
 20    Body of post after the first blank line.
 21  
 22  and publish it to my WordPress blog. Return in standard output
 23  the same post after publishing. It will then have more header
 24  fields (see hFields for the list) and can be edited and re-
 25  published again and again.
 26  
 27  The goal is to work the same way TextMate's Blogging Bundle does
 28  but with fewer initial headers.
 29  '''
 30  
 31  
 32  ####################### Parameters ########################
 33  
 34  # The blog's XMLRPC URL and username.
 35  url = 'http://leancrew.com/path/to/xmlrpc.php'
 36  user = 'username'
 37  
 38  # Time zones. WP is trustworthy only in UTC.
 39  utc = pytz.utc
 40  myTZ = pytz.timezone('US/Central')
 41  
 42  # The header fields and their metaWeblog synonyms.
 43  hFields = [ 'Title', 'Keywords', 'Date', 'Post',
 44              'Slug', 'Link', 'Status', 'Comments' ]
 45  wpFields = [ 'title', 'mt_keywords', 'date_created_gmt',  'postid',
 46               'wp_slug', 'link', 'post_status', 'mt_allow_comments' ]
 47  h2wp = dict(zip(hFields, wpFields))
 48  
 49  
 50  ######################## Functions ########################
 51  
 52  def makeContent(header, body):
 53    "Make the content dict from the header dict."
 54    content = {}
 55    for k, v in header.items():
 56      content.update({h2wp[k]: v})
 57    content.update(description=body)
 58    return content
 59  
 60  def xmlrpc(url, call, params):
 61    "Send an XMLRPC request and return the response as a dictionary."
 62    payload = xmlrpclib.dumps(params, call)
 63    resp = requests.post(url, data=payload)
 64    return xmlrpclib.loads(resp.content)[0][0]
 65  
 66  def tweetlink(text, url):
 67    "Tweet a link to the given URL. Return the URL to the tweet."
 68  
 69    # Authorize Twitter and establish a connection.
 70    auth = tweepy.OAuthHandler('theAPIKey',
 71             'theAPISecret')
 72    auth.set_access_token('theAccessToken',
 73             'theAccessTokenSecret')
 74    api = tweepy.API(auth)
 75  
 76    # How long will the shortened URL to the post be?
 77    short_length = api.configuration()['short_url_length']
 78    
 79    # Don't include the snowman if the tweet is about Afghanistan.
 80    if "Afghanistan" in text:
 81      prefix = u''
 82    else:
 83      prefix = u'⛄ '
 84  
 85    # Construct the tweet.
 86    max_text = 140 - short_length - (len(prefix) + 1)
 87    if len(text) > max_text:
 88      text = prefix + text[:max_text-1] + u'…'
 89    else:
 90      text = prefix + text
 91    tweet = '''{}
 92  {}'''.format(text.encode('utf-8'), url)
 93  
 94    # Send the tweet.
 95    out = api.update_status(tweet)
 96    return 'https://twitter.com/drdrang/status/%s' % out.id_str
 97  
 98  
 99  ###################### Main program #######################
100  
101  # Read and parse the source.
102  source = sys.stdin.read()
103  header, body = source.split('\n\n', 1)
104  header = dict( [ x.split(': ', 1) for x in header.split('\n') ])
105  
106  # The publication date may or may not be in the header.
107  if 'Date' in header:
108    # Get the date from the string in the header.
109    dt = datetime.strptime(header['Date'], "%Y-%m-%d %H:%M:%S")
110    dt = myTZ.localize(dt)
111    header['Date'] = xmlrpclib.DateTime(dt.astimezone(utc))
112    # Articles for later posting shouldn't be tweeted.
113    tweetit = False
114  else:
115    # Use the current date and time.
116    dt = myTZ.localize(datetime.now())
117    header.update({'Date': xmlrpclib.DateTime(dt.astimezone(utc))})
118    # Articles for posting now should be tweeted.
119    tweetit = True
120  
121  # Get the password from Keychain.
122  pw = keyring.get_password(url, user)
123  
124  # Connect and upload the post.
125  blog = xmlrpclib.Server(url)
126  
127  # It's either a new post or an old one that's been revised.
128  if 'Post' in header:
129    # Revising an old post.
130    postID = int(header['Post'])
131    del header['Post']
132    content = makeContent(header, body)
133    
134    # Upload the edited post.
135    params = (postID, user, pw, content, True)
136    unused = xmlrpc(url, 'metaWeblog.editPost', params)
137    
138    # Download the edited post.
139    params = (postID, user, pw)
140    post = xmlrpc(url, 'metaWeblog.getPost', params)
141    
142  else:
143    # Publishing a new post.
144    content = makeContent(header, body)
145    
146    # Upload the new post and get its ID.
147    params = (0, user, pw, content, True)
148    postID = xmlrpc(url, 'metaWeblog.newPost', params)
149    postID = int(postID)
150    
151    # Download the new post.
152    params = (postID, user, pw)
153    post = xmlrpc(url, 'metaWeblog.getPost', params)
154    
155    if tweetit:
156      tweetURL = tweetlink(post[h2wp['Title']], post[h2wp['Link']])
157  
158  # Print the header and body.
159  header = ''
160  for f in hFields:
161    if f == 'Date':
162      # Change the date from UTC to local and from DateTime to string.
163      dt = datetime.strptime(post[h2wp[f]].value, "%Y%m%dT%H:%M:%S")
164      dt = utc.localize(dt).astimezone(myTZ)
165      header += "%s: %s\n" % (f, dt.strftime("%Y-%m-%d %H:%M:%S"))
166    else:
167      header += "%s: %s\n" % (f, post[h2wp[f]])
168  print header.encode('utf8')
169  print
170  print post['description'].encode('utf8')
171  
172  # Open the published post in the default browser after a delay.
173  time.sleep(3)
174  subprocess.call(['open', post['link']])

I’m not going to explain how I use the script or most of its internal workings. That was covered in the two earlier blog posts. What I do want to talk about is the new xmlrpc function defined in Lines 60–64 and used four times in Lines 128–156.

The three arguments to xmlrpc are

  1. The URL of the blog’s xmlrpc.php file. This is defined in Line 35.
  2. The name of the MetaWeblog API call. This will be either metaWeblog.newPost, metaWeblog.editPost, or metaWeblog.getPost, depending on where we are in the script.
  3. A tuple of parameters that make up the payload of information that’s POSTed in the MetaWeblog API call.

What xmlrpc does is

  1. Wrap the parameters and the API call up in an XML structure using the xmlrpclib.dumps convenience function.
  2. POST the XML structure to the WordPress xmlrpc.php using requests.
  3. Unwrap the content of the XML response into a Python dictionary using the xmlrpclib.loads convenience function. This is what xmlrpc returns.

This is, I believe, pretty much what method calls to an xmlrpclib.ServerProxy object do, except that this function takes advantage of the magic Reitz built into requests. Once I had xmlrpc written, it was easy to replace the ServerProxy method calls that had been triggering the 302 errors.

2

It’s been a week of rewriting old scripts and workflows: photo mapping, expense reports, invoicing emails, Markdown table formatting, and now blog posting. Maintenance is important, but I’d like to move on to something new.

  1. You might say it’s closer to the MetaWeblog. ↩

  2. I am by no means a web programmer, but here I am playing one on the internet. Frightening. ↩

Next post Previous post