Next post Previous post

Photo management via the Finder

October 5, 2013 at 9:12 AM by Dr. Drang

I was in my car listening to last week’s episode of The Prompt when the segment on photo management with Bradley Chambers started. Figuring this was going to be yet another list of tips for iPhoto/Aperture/Lightroom, I wanted to fast forward past it, but my phone was in my pocket, so I kept listening. And was pleasantly surprised.

Bradley’s approach to the photo management, which he’s written up in a short book available in iTunes or as a PDF, is very similar to one I started using when I began shooting digital photos about twelve years ago: just use the file system, organizing the photos in a folder/subfolder structure according to the year and month they were taken. He even suggested breaking out special events that have a lot of photos associated with them, like vacations or birthday parties, into their own folders, something I also used to do but lapsed from.

Actually, I’ve lapsed from just about all photo management, which I blame on Apple and iPhoto. Some years ago, in a misguided attempt to make finding photos “easier” for my wife, I poured all our photos into iPhoto. It’s slow, bloated, and generally a piece of shit that I regret ever having anything to do with. Even worse, she and I now have iPhones that are piling up photos that I’ve refused to put into iPhoto but haven’t organized otherwise. It’s clear that I need to go back to my old system, but the thought of taking on such a monumental task has made me blanche, and I just keep putting it off.

Dropbox has been something of an enabler in this procrastination. I use its Camera Upload feature to copy photos from my iPhone to my computer and back them up. This doesn’t organize the photos—they’re all kept in a single flat folder—but it does rename them according to when they were taken (something I’ve done using several scripts over the years). So I’m not in any significant danger of losing my photos, but I’m not in a position to find them easily, either.

Clearly, I needed to set up a system that would get me back to where I was before I drank the iPhoto Kool-Aid. That would involve one script (or set of scripts) that would fix the mess I’d allowed myself to get into and another to manage photos on an ongoing basis. I knew, in broad terms, how I wanted to do the former, and I’d been thinking about buying Hazel to handle the latter.

By the way, if you’re wondering why anyone would use the Finder for photo management, it’s because the Finder, if you use it right, is pretty damned good at it. The icons give you previews of the photos, which you can resize on the fly with that little slider in the bottom right corner of the window. If the largest icon size isn’t big enough, you can always use Quick Look or Preview. In fact, you can easily open an entire folder of photos in Preview if you need more detailed views.

I’m not a big fan of tagging, but the Finder and Spotlight give you that, too, and it’ll be even easier to tag photos in Mavericks. The primary advantage, though, of using the file system directly as your photo management tool is what Bradley said in the podcast: it’s future-proof. Apple could decide to drop iPhoto tomorrow, or it could make changes to it that you find so distasteful you can’t stand to use it anymore. Files and folders will always be there.1

As the podcast proceded, I soon learned that I wasn’t the only one with a photo mess on his hands. Myke Hurley has apparently never organized his photos. He has gigabytes and gigabytes of photos just sitting on his phone. Backed up to iCloud, yes, if you consider that a backup, but with no structure. Bradley began an intervention.

I agreed with much of what was suggested: bringing the photos onto Myke’s Mac through Image Capture and setting up the year/month folders. But Bradley then suggested Myke move his photos into the folder structure by hand, doing maybe fifty a day for the thousands of photos Myke has. This is madness. It’s using a human to serve the computer rather than the other way around. Like me, Myke needs an automated solution.

So I decided to use Myke’s plight as the kick in the pants to get me to finish the scripts I’d been planning to write. As I suspected, it didn’t take very long. Imagining poor little Myke dragging files for weeks on end was just the motivation I needed to sit down and do it.

The first script is called make-photo-folders, and it does just what you’d expect from the name.

python:
1:  #!/usr/bin/python
2:  
3:  import os
4:  
5:  os.chdir(os.environ['HOME'] + '/Pictures')
6:  for year in range(2007, 2021):
7:    for month in range(1, 13):
8:      os.makedirs('iPhone/%04d/%02d' % (year, month))

It assumes that you want your photos organized by year and month in a folder called “iPhone” in the Pictures folder of your home directory. There are a few opportunities for customization:

  1. Most likely, you don’t want that folder called “iPhone.” That can be changed by altering the word on Line 8.
  2. If you don’t want the photos to be in your Pictures folder, change that part of Line 5. Dropbox might be a good choice if you have enough space.
  3. If your photos go back earlier than 2007, you’ll want to change the first argument to range in Line 6 to the year of your earliest photos.

  4. If you want to follow Bradley’s system to the letter, you’ll want the names of the month folders to include the year, e.g., “2013-10” instead of just “10.” I understand why Bradley likes this—your Finder window’s title bar will show more information—but I don’t use it because, as we’ll see in a bit, I rename the photos themselves to include the year. Anyway, if you use Bradley’s system, Line 8 should be changed to

    os.makedirs('iPhone/%04d/%04d-%02d' % (year, year, month))

After running this script, you’ll have a complete set of year/month folders through 2020.

Now comes the trickier script. The goal of this one is to go through a folder and copy all the JPEGs into the appropriate year/month folder. In the process of copying, the file names—which are likely to have names like IMG_1776.JPG—will be changed to reflect the date and time on which they were taken, e.g.

2130-09-25 20.05.43.jpg

This is the format Dropbox uses in its Camera Upload. I’m not a big fan of the space between the date and the time, but other characters I’ve tried look ugly.2 If you’re wondering why I don’t use colons as the hour/minute/second separator, you probably haven’t been around Macs very long. In the pre-OS X days, Apple used colons to separate folder names. AppleScript still uses them and colons are still illegal—create a new folder in the Finder and see what happens when you try to give it a name with a colon. Thus, periods.

You may also be wondering what happens if two photos were taken at the same time. This is not just a theoretical concern. The new iPhones have a burst mode that will almost certainly result in photos with identical timestamps. To handle this, the script names the first photo as above and appends lower case letters to the subsequent ones:

2013-09-25 20.05.43.jpg
2013-09-25 20.05.43a.jpg
2013-09-25 20.05.43b.jpg
2013-09-25 20.05.43c.jpg

Here’s the script, which is called organize-photos:

python:
 1:  #!/usr/bin/python
 2:  
 3:  import os, shutil
 4:  import subprocess
 5:  import os.path
 6:  from datetime import datetime
 7:  
 8:  ######################## Functions #########################
 9:  
10:  def photoDate(f):
11:    "Return the date/time on which the given photo was taken."
12:  
13:    cDate = subprocess.check_output(['sips', '-g', 'creation', f])
14:    cDate = cDate.split('\n')[1].lstrip().split(': ')[1]
15:    return datetime.strptime(cDate, "%Y:%m:%d %H:%M:%S")
16:  
17:  
18:  ###################### Main program ########################
19:  
20:  # Where the photos are and where they're going.
21:  sourceDir = os.environ['HOME'] + '/Desktop/Imports'
22:  destDir = os.environ['HOME'] + '/Pictures/iPhone'
23:  
24:  # The format for the new file names.
25:  fmt = "%Y-%m-%d %H.%M.%S"
26:  
27:  # The problem files.
28:  problems = []
29:  
30:  # Get all the JPEGs in the source folder.
31:  photos = os.listdir(sourceDir)
32:  photos = [ x for x in photos if x[-4:] == '.jpg' or x[-4:] == '.JPG' ]
33:  
34:  # Copy photos into year and month subfolders. Name the copies according to
35:  # their timestamps. If more than one photo has the same timestamp, add
36:  # suffixes 'a', 'b', etc. to the names. 
37:  for photo in photos:
38:    original = sourceDir + '/' + photo
39:    suffix = 'a'
40:    try:
41:      pDate = photoDate(original)
42:      yr = pDate.year
43:      mo = pDate.month
44:      newname = pDate.strftime(fmt)
45:      duplicate = destDir + '/%04d/%02d/%s.jpg' % (yr, mo, newname)
46:      while os.path.exists(duplicate):
47:        newname = pDate.strftime(fmt) + suffix
48:        duplicate = destDir + '/%04d/%02d/%s.jpg' % (yr, mo, newname)
49:        suffix = chr(ord(suffix) + 1)
50:      shutil.copy2(original, duplicate)
51:    except ValueError:
52:      problems.append(photo)
53:  
54:  # Report the problem files, if any.
55:  if len(problems) > 0:
56:    print "Problem files:"
57:    print "\n".join(problems)

There’s plenty of opportunity for customization here, too.

  1. The script assumes the unorganized photos are in a folder called “Imports” on your Desktop. If that’s not where they are, change the sourceDir definition in Line 21.
  2. As in make-photo-folders, the organized year/month folders are assumed to be in a folder called “iPhone” in your Pictures folder. Change the definition of destDir in Line 22 to whatever you changed make-photo-folders to.

  3. If you decided to use Bradley’s system for naming the month folders, change Line 45 to

    duplicate = destDir + '/%04d/%04d-%02d/%s.jpg' % (yr, yr, mo, newname)
  4. If you have a different idea for naming the photo files themselves, change the definition of fmt in Line 25. The various strftime codes Python uses for printing dates are given here.

When organize-photos is done running, the photos will be archived and the script will print out a list of photos it couldn’t archive. If your photos are from your iPhone, these “problem files” will probably be JPEGs you copied from the web into your Camera Roll. Unlike the photos you took with the camera, these images don’t have creation dates in their metadata, so the script didn’t know what to do with them. You’ll have to copy them by hand if you want them in your new archive.

I should mention also that organize-photos does no more than what its name implies. It doesn’t organize any screenshots—which are saved as PNGs—that might have been copied over from your phone, nor does it organize your movies. I don’t feel bad about the screenshots, but I do wish I knew how to handle movies. Unfortunately, Apple’s sips command, which I use to get the creation date, doesn’t work on movies, so you’ll have to copy them by hand. An easy way to find all the movies in your old archive is to open a Finder window to that folder and sort by Kind.

The scripts are non-destructive. No files are deleted from your old archive. Having said that, make sure you have a backup before running these scripts.

These scripts have worked well for organizing my iPhone photos, but they’re obviously inadequate for pulling files out of an iPhoto library. To do that I’ll have to use something like os.path.walk to dig through iPhoto’s directory structure. That’s a topic for another day.

Another improvement I’ve thought of is to have the source directory passed in as an argument to organize-photos instead of having it built into the script itself. That would make it more flexible if you have your photos distributed across a handful of directories. This is a pretty simple change to define sourceDir in terms of sys.argv, an exercise I’ll leave to the reader.

Myke mentioned these scripts on the most recent Prompt, and he seems a little nervous about using them—probably because the customization looks daunting. If you don’t like the scripts’ defaults but are afraid to edit source code, you could try this:

  1. Move all your unorganized photos into a folder called “Imports” on your Desktop.
  2. Run the make-photo-folders script.
  3. Run the organize-photos script.
  4. Your photos are now organized in a folder called “iPhone” in your Pictures folder. Move/copy/rename this folder as you see fit.

You’ll still be stuck with my choices for naming the photos and the month folders, but your photos will be where you want them.

Update 10/6/13
Many helpful comments, questions, and suggestions:

  1. Most popular by far was “What about just using Hazel?” My simple answer is that if Hazel can do everything you need, go for it. I mentioned Hazel in the body of the post because I’ve considered buying it, but I didn’t know it could do all the things this script does. I prefer to write scripts instead of using tools like Automator, Hazel, and Keyboard Maestro (a topic for a future post), but my preferences shouldn’t keep you from using a tool you like.
  2. Exiftool has been suggested by John Johnston as a way to get the creation date of movies. It works, although the date is stored in a format that differs from that of photos, so you have to be careful with time zones. Expanding the script to handle movies is a good idea, but I didn’t want to use an external program that wasn’t shipped with OS X. Expecting Myke to install another command line tool to get my command line tool to work seemed like asking too much.
  3. Casey Liss stopped mooning over BMWs long enough to make some pretty significant improvements to the organize-photos script. You can see the script on GitHub and an explanation of it on his blog. An important technical improvement he made was to eliminate the “bare” except I originally had in Line 51. That was a serious mistake on my part, as it kept the script running under a multitude of errors when all I really wanted to do was keep it running when the JPEG didn’t have a creation date in its metadata. I’ve changed Line 51 to what it should have been in the first place. Casey handles exceptions in a somewhat more forgiving way—you should check out his script to see our different approaches.
  4. And there were the usual set of typos and missing words that I can’t seem to eliminate. Thanks to everyone who pointed those out.

  1. Yes, Apple is downplaying the directory structure, but it’s still there and it’s going to stay because that’s how computers organize their files. Even if you do end up moving to some other system in the future, a folder/subfolder structure will be a good place to start. ↩

  2. Although I’m a big fan of Underscore David Smith, I’m not a big fan of the underscore character. ↩

Next post Previous post