A couple of days ago, Numberphile released a video on YouTube in which Tom Crawford explained why parabolic mirrors reflect incoming light to the focal point. His explanation was, I thought, needlessly complicated, so I decided to write up a simpler one. No implicit differentiation, no vector algebra, no messing around with square roots of polynomials.

\nHere’s Tom’s explanation:

\n\nFirst, I don’t think Tom does his audience any favors by starting with a parabola that opens to the right. Most people are used to parabolas that open upward and have this simple equation

\n\n\\[y = bx^2\\]\n\nwhere *b* controls the spread of the parabola. Using this equation is especially helpful when you want to get the slope,

because this is one of the first derivatives you learn in calculus.

\nSometimes, doing the unexpected can lead to nice simplifications, but that doesn’t happen in Tom’s analysis, so let’s make our parabola follow the equation above.

\n\nAs you can see, I’ve added the focal point (or focus), which is on the y-axis at a distance \\(1/4b\\) above the vertex. If it’s been a while since you took analytic geometry, you may have forgotten where the focus is, but that’s the spot. (If you’re wondering about the directrix, it’s parallel to the x-axis and \\(1/4b\\) below it, but we won’t need that in our analysis.)

\nThe figure also includes a line tangent to the parabola at some general point, \\((x, bx^2)\\) and some signal (a light or sound wave) that comes in parallel to the y-axis, bounces off the parabola and goes through the focal point. Physics says the incident and reflected beams are at the same angle, \\(\\theta\\), to the tangent line. We get the other \\(\\theta\\), the one on the lower right, from geometry: it’s on the opposite side of intersecting lines from the upper \\(\\theta\\).

\nOur job is to prove that the above construction is true, that the reflected beam passes through the focal point for any point on the parabola. To do that, let’s redraw the triangles from the above diagram a little bigger and identify some other dimensions and angles.

\n\nHere, \\(\\alpha\\) is the angle of the line from the reflection point to the focus and \\(\\beta\\) is the angle of the tangent line through the reflection point. As you can see from the larger triangle,

\n\n\\[2 \\theta = 90^\\circ - \\alpha\\]\n\nand from the smaller triangle,

\n\n\\[\\theta = 90^\\circ - \\beta\\]\n\nCombining these equations gives us this relationship between \\(\\alpha\\) and \\(\\beta\\):

\n\n\\[2\\beta = 90^\\circ + \\alpha\\]\n\nIf we prove that this is true, we’ve shown that the reflected beam goes through the focus.

\nWe know that the tangent of \\(\\beta\\) is the derivative of the parabola, \\(2bx\\), and we can work out the tangent of \\(\\alpha\\) from the dimensions given in the figure. So let’s take the tangent of both sides of the above equation and see what happens.

\nFor the left side, we use this double angle formula for tangent:

\n\n\\[\\tan (2\\beta) = \\frac{2 \\tan \\beta}{1 - \\tan^2 \\beta} = \\frac{2 (2bx)}{1 - (2bx)^2} = \\frac{4 b x}{1 - 4 b^2 x^2}\\]\n\nFor the right side, we use this trig identity,

\n\n\\[\\tan (90^\\circ + \\alpha) = - \\cot \\alpha\\]\n\nand the lengths given for the sides of the larger triangle to get

\n\n\\[-\\cot \\alpha = -\\frac{x}{b x^2 - \\frac{1}{4b}} = -\\frac{4bx}{4b^2x^2 - 1} = \\frac{4 b x}{1 - 4 b^2 x^2}\\]\n\nThis proves that

\n\n\\[\\tan (2\\beta) = \\tan (90^\\circ + \\alpha )\\]\n\nand therefore

\n\n\\[2\\beta = 90^\\circ + \\alpha\\]\n\nand that’s what we set out to prove. Because this is true for any value of *x*, any vertical beam hitting the parabola will be reflected to the focal point.

When Tom turned his lines into vectors and started taking dot products, I thought he was doing something clever to make his later algebra easier. But as you saw from his messing around with square roots of polynomials, it didn’t make the algebra easier, it made it harder.

\nShould I point out that Tom’s flashlight probably doesn’t work the way he says? That the part behind the LED is milky white and not a shiny reflector? That its beam clearly doesn’t come out parallel to the axis but spreads quite widely? No, I’m sure you already noticed all that.

\n

[If there are equations in this post, you will see them rendered properly in the original article.]

"}, {"title": "ImageOptim", "url": "https://leancrew.com/all-this/2023/03/imageoptim/", "author": {"name": "Dr. Drang"}, "summary": "Stephen Hackett somehow reminds me of an app I use all the time.", "date_published": "2023-03-22T14:45:06+00:00", "id": "https://leancrew.com/all-this/2023/03/imageoptim/", "content_html": "Yesterday, I listened to the most recent Mac Power Users episode, “20 Mac Apps Under $20.” Although MPU is probably best known for its deep dive episodes, I always like these more rapid-fire discussions. As usual, this one has a good mix of apps that are new to me and those I already know about.^{1} One of Stephen’s picks is ImageOptim (discussion starts at 1:00:50), an app I use all the time but haven’t talked about here.

As suggested by its name, ImageOptim optimizes images by reducing their file size. It doesn’t resize an image in the sense of changing its width or height in pixels. What ImageOptim leaves you with is an image that takes up less space on your disk; it doesn’t take up less space on your screen.

\nI’m sure you already know that a JPEG’s file size can be reduced by lowering its “quality.” ImageOptim can do this kind of optimization, but so can lots of apps. what ImageOptim really excels at is reducing the file size *without* changing the image quality. And it can do this for PNGs, which is the format I use for the screenshots I post here. Generally, I get 25–40% smaller files after running them through ImageOptim.

Fundamentally, ImageOptim is a front end for a set of open source image optimization programs. You can see and choose the programs in either the Preferences or the Tools menu.

\n\nThe given image is run through each of the checked programs appropriate for its file type, and the one that provides the smallest file is selected. The image file is overwritten with that optimized version and you’re told how much smaller the file became. You’ll note that I don’t have my images run through Zopfli or PNGOUT, as these programs run slower than the others, and I don’t think the extra time is worth slight extra reduction in file size.

\nThis is all very well and good, but I wouldn’t think as highly of ImageOptim if it weren’t for one very useful feature: it can be run as a command line program. Which means that my Keyboard Maestro macro for taking screenshots and uploading them to the blog’s server can reduce their file size during the process.

\n(I used to use OptiPNG for this, but at some point a couple of years ago it started running *very* slowly, and the delay was disrupting even the slow pace of my writing. I just ran a few tests, and OptiPNG seems to be running fine on my Macs now, but I’m sticking with ImageOptim.)

My screenshot-and-upload macro, SnapSCP, looks like this:

\n\nThe Python script in Step 1 is this:

\n`python:\n 1: #!/usr/bin/env python3\n 2: \n 3: import Pashua\n 4: import tempfile\n 5: from PIL import Image\n 6: import sys, os, os.path\n 7: import subprocess\n 8: import urllib.parse\n 9: from datetime import date\n10: import sys\n11: \n12: # Parameters\n13: dstring = date.today().strftime('%Y%m%d')\n14: year = date.today().strftime('%Y')\n15: type = \"png\"\n16: localdir = os.environ['HOME'] + \"/Pictures/Screenshots\"\n17: tf, tfname = tempfile.mkstemp(suffix='.'+type, dir=localdir)\n18: bgcolor = (61, 101, 156)\n19: border = 16\n20: optimizer = '/Applications/ImageOptim.app/Contents/MacOS/ImageOptim'\n21: server = f'user@leancrew.com/path/to/images{year}/'\n22: port = '9876'\n23: \n24: # Dialog box configuration\n25: conf = b'''\n26: # Window properties\n27: *.title = Snapshot\n28: \n29: # File name text field properties\n30: fn.type = textfield\n31: fn.default = Snapshot\n32: fn.width = 264\n33: fn.x = 50\n34: fn.y = 40\n35: fnl.type = text\n36: fnl.default = Name:\n37: fnl.x = 0\n38: fnl.y = 42\n39: \n40: # Border checkbox properties\n41: bd.type = checkbox\n42: bd.label = Add background\n43: bd.x = 10\n44: bd.y = 0\n45: \n46: # Default button\n47: db.type = defaultbutton\n48: db.label = Save\n49: \n50: # Cancel button\n51: cb.type = cancelbutton\n52: '''\n53: \n54: # Capture a portion of the screen and save it to a temporary file.\n55: status = subprocess.run([\"screencapture\", \"-io\", \"-t\", type, tfname])\n56: \n57: # Exit if the user canceled the screencapture.\n58: if not status.returncode == 0:\n59: os.remove(tfname)\n60: print(\"Canceled\")\n61: sys.exit()\n62: \n63: # Open the dialog box and get the input.\n64: dialog = Pashua.run(conf)\n65: if dialog['cb'] == '1':\n66: os.remove(tfname)\n67: print(\"Canceled\")\n68: sys.exit()\n69: \n70: # Add a desktop background border if asked for.\n71: snap = Image.open(tfname)\n72: if dialog['bd'] == '1':\n73: # Make a solid-colored background bigger than the screenshot.\n74: snapsize = tuple([ x + 2*border for x in snap.size ])\n75: bg = Image.new('RGBA', snapsize, bgcolor)\n76: bg.alpha_composite(snap, dest= (border, border))\n77: bg.save(tfname)\n78: \n79: # Rename the temporary file using today's date (yyyymmdd) and the\n80: # name provided by the user.\n81: name = dialog['fn'].strip()\n82: fname = f'{localdir}/{dstring}-{name}.{type}'\n83: os.rename(tfname, fname)\n84: bname = os.path.basename(fname)\n85: \n86: # Optimize the PNG.\n87: subprocess.run([optimizer, fname], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)\n88: \n89: # Upload the file via scp.\n90: subprocess.run(['scp', '-P', port, fname, server])\n91: \n92: # Generate the URL of the uploaded image.\n93: bname = urllib.parse.quote(bname)\n94: print(f'https://leancrew.com/all-this/images{year}/{bname}', end='')\n`

\nHow this script uses Pashua, `screencapture`

, and `scp`

are described in an older post. The only thing that’s different now is the use of ImageOptim. The full path to the executable is given in Line 20,

`python:\noptimizer = '/Applications/ImageOptim.app/Contents/MacOS/ImageOptim'\n`

\nand it’s run via the `subprocess`

library in Line 87,

`python:\nsubprocess.run([optimizer, fname], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)\n`

\nWhen the optimized screenshot is done uploading and its URL is on the clipboard, the macro plays the Glass sound to alert me that I can paste the URL wherever I need it. It typically takes 3–5 seconds.

\nSo ImageOptim gives you the best of both worlds: a nice app for optimizing images through the GUI and a command-line tool for doing the same thing in scripts and macros. It’s even better than Stephen said it was.

\n\n

\n

\n

- \n
- \n
What’s good about a discussion of apps I’m familiar with? Well, I’m usually not as familiar with them as I think. There’s often a feature or two I never knew about. ↩

\n \n

\n

[If there are equations in this post, you will see them rendered properly in the original article.]

"}, {"title": "Feet, inches, and averaging", "url": "https://leancrew.com/all-this/2023/03/feet-inches-and-averaging/", "author": {"name": "Dr. Drang"}, "summary": "A mistake by Slate gives me a chance to do some fancy averaging and make fun of Purdue.", "date_published": "2023-03-19T20:56:18+00:00", "id": "https://leancrew.com/all-this/2023/03/feet-inches-and-averaging/", "content_html": "After Purdue’s historically embarrassing loss to Fairleigh Dickinson on Friday night, many sportswriters awakened to Purdue’s longstanding poor performance in the NCAA tournament and wrote articles like this one in *Slate*. As I read that article, I saw one error that should embarrass *Slate* and then saw another statement that seemed like an error but wasn’t. I figured they were worth a short blog post.

The embarrassing error concerns the height difference between Purdue and FDU. Purdue’s top player, Zach Edey, is 7′ 4″. According to Slate, FDU has “nobody taller than 6-foot-6.” *Slate* apparently doesn’t know how to use foot and inch marks, but I don’t consider that an error. And even though a quick perusal of FDU’s roster clearly shows a 6′ 7″ player, I’m going to give them a pass on that, too, as Pier-Olivier Racine doesn’t get much court time and didn’t play at all in the Purdue game. No, the error comes later when, after establishing that Edey is 7′ 4″ and the tallest players on FDU are 6′ 6″, the article says

\n\nEdey is 8 inches taller than anyone who guarded him.

\n

My guess is that the writer, Alex Kirshner, did the calculation in his head this way,

\n74 – 66 = 8

\nand everyone at *Slate* who read the article before publication just went along with it. I’m also pretty sure that all my non-American readers are nodding their heads and congratulating themselves on not using those awful and confusing US customary units. But honestly, is it really that hard to see that 7′ 4″ is actually 10″ taller than 6′ 6″? I bet even Purdue’s engineering students would get that one right.

Here’s the thing that seemed like a mistake but wasn’t:

\n\n\nThere are 363 Division I men’s basketball teams. FDU is the shortest one of them all, according to Ken Pomeroy’s data, with an average height of 6-foot-1.

\n

This wasn’t the first time I’d heard that Fairleigh Dickinson’s team had an average height of 6′ 1″—it’s in several of the Purdue-excoriating articles. And it wasn’t the first time I thought *that can’t be right.* I don’t need KenPom to tell me the team’s average height (and a good thing, too, because it’s behind a paywall), all the heights are on the afore-linked roster.

Scrolling through the list of players, there are only two under 6′ 1″, one who is 6′ 1″, and ten others who are taller. Even without doing a single calculation, I knew the average had to be over 6′ 1″. And it is. To the nearest half-inch,^{1} the average height of the thirteen players on FDU’s roster is 6′ 3″.

The thing is, while I can believe that *Slate* would make a mistake in calculating a team’s average height, I can’t believe Ken Pomeroy would. So how did he come up with 6′ 1″? The answer is on his site, and it’s a good one:

\n\nOverall average height is computed by taking the average listed height of every player on the team, weighted by minutes played. Players that have played less than 10% of their team’s minutes are not included.

\n

Pomeroy wants to give his readers the average height of the team *on the court*, and weighting the players’ heights by minutes played is probably the best way to do that. And dropping out those who play in “garbage time” makes sense, too.^{2}

So I made a little spreadsheet with the FDU players, their heights, and the minutes they played in the Purdue game. Here’s a screenshot:

\n\nAs you can see, the team’s shorter players, Roberts and Singleton, also got the most minutes, so the time-weighted average, rounded to the nearest half-inch, for this particular game was 6′ 1½″. Close enough. I wouldn’t be surprised to learn that FDU had its taller players on the court more than usual in this game.

\nThe spreadsheet also includes the naive average of 6′ 3″ and the 6′ 2″ average of the nine FDU players who played in the Purdue game.

\nBy the way, I didn’t enter the heights in feet and inches as shown in the second column. I entered the heights in inches in the third column and converted to feet and inches for presentation with this formula for the player heights

\n`FIXED(FLOOR(C2÷12,1), 0) & \"′ \" &\nFIXED(MOD(C2, 12), 0) & \"″\"\n`

\nand this formula for the average heights.

\n`FIXED(FLOOR(C16÷12,1), 0) & \"′ \" &\nFIXED(MROUND(MOD(C16, 12), 0.5), 1) & \"″\"\n`

\nI needed the `MROUND`

function in the latter formula to get the results rounded to the nearest half-inch.

The “played average” was calculated this way:

\n`AVERAGEIF($D$2:$D$14,\">0\",C$2:C$14)\n`

\nI’m not a big spreadsheet user, and I think this is the first time I ever used `MROUND`

or `AVERAGEIF`

. So thanks to *Slate* and KenPom for that.

\n

\n\n

\n\n

\n

[If there are equations in this post, you will see them rendered properly in the original article.]

"}, {"title": "Better MathJax equations", "url": "https://leancrew.com/all-this/2023/03/better-mathjax-equations/", "author": {"name": "Dr. Drang"}, "summary": "MathJax settings to improve the rendering of equations.", "date_published": "2023-03-15T17:34:20+00:00", "id": "https://leancrew.com/all-this/2023/03/better-mathjax-equations/", "content_html": "If some of the equations in my posts look a little off—poorly spaced or, more likely, with a wobbly baseline—there may be something you can do about it. I learned this on Mastodon through a combination of Andy Napper and Mimmo.

\nTo some extent, the formatting of equations depends on the browser you use, but I’m not going to tell you switch from your favorite browser just to read this blog. Instead, you can change how MathJax renders equations in whatever browser you use by tweaking a setting or two.

\nHere’s an example of an uneven baseline from a recent post:

\n\nThis is how Safari renders the equation when MathJax is using the “CHTML” (CSS and HTML) renderer. But if you right-click on the equation and work your way through the MathJax settings, you can switch to the SVG renderer

\n\nto get a more even baseline

\n\nThe nice thing about this is that even though you change the setting on one equation, it applies to all the equations. Even better, it’s persistent—you don’t have to keep changing it every time you open a different post with equations.

\nIf you find yourself zooming in on equations often, you might want to set up a “Zooom Trigger” to show a magnified view of any equation you click on:

\n\nThe problem with the Zoom Trigger is that it shows the magnified equation in a box that might not be big enough to show the whole equation at once:

\n\nEven though you can scroll around within the box to see the rest of the equation, I never use this feature. I find it much better to zoom the entire web page by either spreading two fingers on the trackpad^{1} or pressing ⌘+ a few times.

It’s been a long time since I reviewed ANIAT’s MathJax configuration, and I’m sure there are things I can do to improve the rendering for everyone. Until I get around to that, have a look through all of MathJax’s settings to see if any of its customizations strike your fancy.

\n\n

\n

\n

- \n
- \n
I refuse to call this “pinching,” no matter what Apple and everyone else does. ↩

\n \n

\n

[If there are equations in this post, you will see them rendered properly in the original article.]

"}, {"title": "Concrete and pi", "url": "https://leancrew.com/all-this/2023/03/concrete-and-pi/", "author": {"name": "Dr. Drang"}, "summary": "Possibly the only contribution I will ever make to Pi Day.", "date_published": "2023-03-14T17:08:34+00:00", "id": "https://leancrew.com/all-this/2023/03/concrete-and-pi/", "content_html": "Back in either 1980 or ’81, I took course called Properties and Behavior of Concrete, which had the course number CE 214 and this description:

\n\n\nEngineering properties of plain concrete; influence of cement, aggregates, water, and admixtures on the properties of fresh and hardened concrete; microstructure of cement paste and concrete; mix design; handling of fresh concrete; and behavior under various types of loading and environments. Laboratory practice is an important part of the course.

\n

The University of Illinois’s civil engineering department has undergone course number inflation over the past 40 years. That course is now CEE 401, but the description is nearly identical:

\n\n\nExamination of the influence of constituent materials (cements, water, aggregates and admixtures) on the properties of fresh and hardened concrete, concrete mix design, handling and placement of concrete, and behavior of concrete under various types of loading and environment. Laboratory exercises utilize standard concrete test methods. Field trips are held during some scheduled laboratory sessions.

\n

When I took the course, it was taught by J. Francis Young, whose book on concrete was still being copyedited (we students got photocopies of proofs with his handwritten corrections as handouts). Friends of mine who took it the following semester were taught by Clyde Kesler, the father of the concrete canoe, shortly before he retired.

\nThe differences between Young and Kesler were striking. Kesler was a local boy, born and raised within a few miles of the university he got his degrees from and then spent his entire career at. Young was born in New Zealand, got his Ph.D. in London, and then had a long career in the middle of Illinois. They were brought together by concrete and academics.

\nWhat does this have to do with pi? During the part of the class on mix design, we learned how to proportion the water, cement, large aggregate (gravel), small aggregate (sand), and admixtures to get certain target properties in the concrete. During the mix design process, you do some calculations using the specific gravities of these constituents.

\nThe specific gravity of portland cement is generally taken to be 3.15, but it’s not an exact figure. Prof. Young told us to just tap the π button on our calculators when we needed to enter it. This tip didn’t save us much time, but it was a great way to teach us the specific gravity of cement. In the 40+ years since taking that class, I’ve had to go through the mix design process only a few times, but every time I remembered to press the π button.

\n

[If there are equations in this post, you will see them rendered properly in the original article.]

"}, {"title": "Physics and units again", "url": "https://leancrew.com/all-this/2023/03/physics-and-units-again/", "author": {"name": "Dr. Drang"}, "summary": "Another commentary on a Matt Parker video.", "date_published": "2023-03-12T19:54:50+00:00", "id": "https://leancrew.com/all-this/2023/03/physics-and-units-again/", "content_html": "I don’t want this place to turn into a Matt Parker followup blog, but there are things in his recent Pi Day video that fit in with comments I made on his Runge video, specifically the part about engineers embedding unit conversions within formulas. So go ahead and watch the video if you haven’t already.

\n\nThe formula of interest is the one used to calculate the speed of a car during a turning skid:

\n\n\\[v = \\sqrt{15\\: r f}\\]\n\nwhere *r* is the radius of the car’s motion during the skid, in feet, and *f* is the dynamic coefficient of friction between the tires and the road. The left-hand-side, *v*, is the speed of the car in miles per hour.

Traffic accident reconstructionists, like Matt’s guide Eric in the video, are able to use this formula because they can measure the radius of the skidmark on the pavement and can either measure the friction coefficient directly, as they do in the video, or just estimate it from known values. As you can see in the video, Eric expected the value to be 0.7, and they measured it as 0.68.

\nThe 15 in the formula is not a pure number, it’s a combination of the acceleration due to gravity, *g*, and a conversion from feet per second to miles per hour. The pure mechanics version of the formula is

which Matt derives in the video. Here in the US, we would measure *r* in feet and use a *g* of \\(32.2 \\: \\mathrm{ft/s^2}\\). That gives us a *v* in feet per second, which isn’t how vehicle speeds are typically presented. To get the speed in miles per hour, we need to convert feet to miles and seconds to hours, which is done like this:

Since \\(3600/5280 = 0.682\\), we can bring the conversion fraction inside the square root like this,

\n\n\\[v = \\sqrt{ (32.2)(0.682)^2\\: r f } = \\sqrt{ 14.97\\: r f }\\]\n\nand 15 is close enough to 14.97 justify the substitution. The 15 is the acceleration due to gravity in these rather weird units:

\n\n\\[\\mathrm{\\frac{{mile}^2}{{ft} \\cdot {hr}^2}}\\]\n\n \nAlthough I do accident reconstruction professionally, I’ve never delved much into *traffic* accident reconstruction. But when I have, I’ve noticed that the formulas reconstructionists use are often in this form, where the formula includes a hidden unit conversion and only works if the inputs and outputs are in a particular set of units. When I saw this video, I realized I could have used one of these formulas in my discussion last month of formulas with embedded units.

(I also thought of a friend of mine who passed away last year. He always said the hardest part of traffic accident reconstruction was the back-and-forth conversion between feet per second and miles per hour.)

\nBy the way, no one I know uses 5280 and 3600 to convert between fps and mph. They always use 88 and 60 because

\n\n\\[88\\: \\mathrm{\\frac{ft}{s}} \\left( \\mathrm{\\frac{1\\: mile}{5280 \\: ft}} \\right) \\left( \\mathrm{\\frac{3600 \\: s}{1 \\: hr}} \\right) = 60\\: \\mathrm{\\frac{mile}{hr}}\\]\n\nOf course, you could reduce the fraction further and say that 22 fps is 15 mph, but 60 mph is such a useful speed (a mile a minute) that people usually remember the conversion that way.

\nOne last thing: Matt makes a small disparaging comment about having to use US customary units in the video, but even if he were working in metric, I’m pretty sure he’d have to do a unit conversion. Car speeds in the more enlightened parts of the world are typically given in kilometers per hour, not meters per second.

\n\n\\[v = \\sqrt{ (9.81 \\: \\mathrm{m/s^2})\\: r f} \\left( \\mathrm{\\frac{1\\: km}{1000\\: m}} \\right) \\left( \\mathrm{\\frac{3600\\: s}{1\\: hr}} \\right) = \\sqrt{127\\: r f}\\]\n\nYou could probably use 130 without much worry. After all, you’re going to reduce the error in *v* when you take the square root.

\n

[If there are equations in this post, you will see them rendered properly in the original article.]

"}, {"title": "Learning (sort of) from ChatGPT", "url": "https://leancrew.com/all-this/2023/03/learning-sort-of-from-chatgpt/", "author": {"name": "Dr. Drang"}, "summary": "Simon Willison's understandable refusal to learn AppleScript ends up teaching me some new command syntax.", "date_published": "2023-03-09T22:01:14+00:00", "id": "https://leancrew.com/all-this/2023/03/learning-sort-of-from-chatgpt/", "content_html": "Simon Willison, the primary developer of the Datasette exploratory data analysis tool, has a strong interest in ChatGPT and similar AI toys.^{1} He recently linked on Mastodon to this dialog with ChatGPT to write some AppleScript. In that dialog, we see the good and bad of using ChatGPT to help you write programs. Although I generally don’t think much of programming that way, I did learn something from Willison’s exploration.

Willison asked ChatGPT to write him an AppleScript that output the contents of all of his Apple Notes. He then asked ChatGPT to turn that into a shell script. The result was

\n` 1: #!/bin/zsh\n 2: \n 3: osascript -e 'tell application \"Notes\"\n 4: repeat with eachNote in every note\n 5: set noteTitle to the name of eachNote\n 6: set noteBody to the body of eachNote\n 7: set output to noteTitle & \"\\n\" & noteBody & \"\\n\"\n 8: display dialog output\n 9: log output\n10: end repeat\n11: end tell'\n`

\nThat Willison was willing to run this script is proof of his assertion that he’s steadfastly refused to learn AppleScript. Putting `display dialog`

(Line 8) into a loop that could run dozens or hundreds of times, depending on how many notes you have, is insane. So although this was perfectly legal AppleScript (wrapped in a shell command) that solved the main problem of looping through all the notes, it has a defect (“it spammed my screen with dialog boxes”) that no AppleScript coder would include.

Two other aspects that I think most AppleScript programmers would avoid:

\n- \n
- Using
`\"\\n\"`

in Line 7. That’s a perfectly normal way to add a linefeed in most languages, but AppleScript has`linefeed`

for that, and idiomatic AppleScript would use it. \n - Using
`log`

in Line 9. This is a more subtle mistake.`log`

is meant to be used for debugging. When used inside an`osascript`

call, it writes to standard error. But the contents of the`output`

variable are the whole point of this script—it should be going to standard output, not standard error. \n

Willison and ChatGPT eventually got to a script that uses AppleScript’s `write`

command to write his output to a file. He then wrote a short Python script that reads this file and saves the notes into an SQLite database:

`python:\n 1: import sqlite_utils\n 2: split = b\"------------------------\\n\"\n 3: s = open(\"/tmp/notes.txt\", \"rb\").read()\n 4: notes = [n.decode(\"mac_roman\") for n in s.split(split) if n]\n 5: \n 6: cleaned_notes = [{\n 7: \"id\": n.split(\"\\n\")[0],\n 8: \"title\": n.split(\"\\n\")[1],\n 9: \"body\": \"\\n\".join(n.split(\"\\n\")[2:]).strip()\n10: } for n in notes]\n11: \n12: db = sqlite_utils.Database(\"/tmp/notes.db\")\n13: db[\"notes\"].insert_all(cleaned_notes)\n`

\nWhat struck me about this script was Line 4. Why is he decoding the contents of the file using Mac OS Roman? I have plenty of code that mixes AppleScript and Python, and I’ve never needed to use `decode(\"macroman\")`

. Surely AppleScript doesn’t write text files in Mac OS Roman anymore?

Surely it does. Here’s Apple’s documentation on the `write`

command and the formats it can generate through the `as`

clause:

\n\n\n

`as class`

\n\nWrite the data as this class. The most common ones control the use of three different text encodings:

\n\n

\n- \n
`text`

or`string`

- The primary text encoding, as determined by the user’s language preferences set in the International preference panel. (For example, Mac OS Roman for English, MacJapanese for Japanese, and so on.)
\n

\n- \n
`Unicode text`

- UTF-16.
\n

\n- \n
`«class utf8»`

- UTF-8.
\nAny other class is possible, for example date or list, but is typically only useful if the data will be read using a read statement specifying the same value for the as parameter.

\n\n

Default Value:

\nThe class of the supplied data. See Special Considerations.

Despite it being 2023, and despite Apple talking big about AppleScript being “entirely Unicode-based” over 15 years ago, this says AppleScript still writes text files, by default, in the Mac OS Roman encoding.

\nMaybe the documentation is wrong. I created an Apple Note specifically to see how non-ASCII characters are encoded under different circumstances.

\n\nThe contents of the note are

\n`¿Thîs filé hås “Unicode” characters—doesn’t it?\n`

\nI then wrote this AppleScript as a test:

\n`applescript:\n 1: set fRef to open for access \"/Users/drang/Desktop/notes-test.txt\" with write permission\n 2: set eof of fRef to 0\n 3: \n 4: tell application \"Notes\"\n 5: repeat with thisNote in every note\n 6: tell thisNote\n 7: if name contains \"UTF-8 Test\" then\n 8: write plaintext & linefeed to fRef\n 9: close access fRef\n10: set the clipboard to plaintext & linefeed\n11: exit repeat\n12: end if\n13: end tell\n14: end repeat\n15: end tell\n`

\nIt gets the contents of the note shown above (`plaintext`

is the content without any of the HTML) and does two things with it:

- \n
- Writes it to a file on my Desktop. \n
- Puts it on the clipboard. \n

Let’s see the difference between the two. Running `xxd`

, the hex dump utility, on the saved file gives

`00000000: 5554 462d 3820 5465 7374 0a0a c054 6894 UTF-8 Test...Th.\n00000010: 7320 6669 6c8e 2068 8c73 20d2 556e 6963 s fil. h.s .Unic\n00000020: 6f64 65d3 2063 6861 7261 6374 6572 73d1 ode. characters.\n00000030: 646f 6573 6ed5 7420 6974 3f0a doesn.t it?.\n`

\nRunning it on the clipboard (via `pbpaste | xxd`

) gives

`00000000: 5554 462d 3820 5465 7374 0a0a c2bf 5468 UTF-8 Test....Th\n00000010: c3ae 7320 6669 6cc3 a920 68c3 a573 20e2 ..s fil.. h..s .\n00000020: 809c 556e 6963 6f64 65e2 809d 2063 6861 ..Unicode... cha\n00000030: 7261 6374 6572 73e2 8094 646f 6573 6ee2 racters...doesn.\n00000040: 8099 7420 6974 3f0a ..t it?.\n`

\nAs you can see, there are more bytes in the clipboard than there are in the file, which shows the encodings are different. It’s easy enough to figure out that the file is encoded in Mac OS Roman and the clipboard is encoded in UTF-8.

\nSo the encoding depends on how you handle the output, which is one of those—how should I put this?—*unintuitive* aspects of AppleScript, one that I don’t recall running into before now. I suppose it’s because I seldom (never?) write text files directly from AppleScript. Not files with non-ASCII characters, anyway.

As the documentation says, the way to get UTF-8 output written to the file is to change Line 8 to

\n`applescript:\nwrite plaintext & linefeed to fRef as «class utf8»\n`

\nWhy the weird `«»`

syntax? I suppose it’s because Apple had already used `as Unicode text`

for UTF-16 and just couldn’t be bothered to extend the syntax to handle UTF-8 gracefully.

You know how I talked a couple of days ago about updating a process so I wouldn’t be embarrassed by my description of it? Apple may have music in its DNA, but it apparently lacks the gene for embarrassment. That would also explain a lot about Shortcuts.

\nI’m going to give ChatGPT some credit for introducing me to `as «class utf8»`

, even though it didn’t put it in its code. Most of the credit, though, goes to Simon Willison for documenting his ChatGPT dialog, writing clear Python code, and leading me to look up some AppleScript syntax I didn’t know about.

\n

\n

\n

- \n
- \n
Is “toys” too dismissive? Time will tell. ↩

\n \n

\n

[If there are equations in this post, you will see them rendered properly in the original article.]

"}, {"title": "Automation via embarrassment", "url": "https://leancrew.com/all-this/2023/03/automation-via-embarrassment/", "author": {"name": "Dr. Drang"}, "summary": "I wrote a new macro today to avoid the embarrassment of explaining how I did the process by hand.", "date_published": "2023-03-06T19:30:14+00:00", "id": "https://leancrew.com/all-this/2023/03/automation-via-embarrassment/", "content_html": "This morning I was adding an entry to my homemade wiki when I became acutely embarrassed by how convoluted the entry was becoming. I stopped writing, created an automation that made the process I was describing much simpler, and rewrote the entry feeling much better about myself.

\nMy homemade wiki—or personal knowledge management system, as they are often called nowadays—is just a set of Markdown files that gets turned into a static set of web pages. It’s called Hints, and I’ve described it this way:

\n\n\nFirst, you should know that I have a very specific use for this wiki. It tells me how to do certain things on my computer. These are typically programming hints—techniques and code excerpts that have helped me solve problems that I expect to run into again—but some are little configuration hacks that make using my computer easier. They’re mostly the culmination of long stretches of research and trial-and-error testing, put into my own words with links back to the original sources. Despite my calling it a wiki, there are very few internal links.

\n

The entry I was writing today described how I take a table of items in LaTeX format and turn them into a one-item-per-line list that I can process in a variety of ways that we won’t get into here. Here’s a made-up example, a list of randomly chosen apartment numbers in a 20×3 table:

\n`\\begin{table}[htbp]\n\\begin{center}\n\\begin{tabular}{\n @{\\hspace*{5pt}}\n ccc\n @{\\hspace*{5pt}}\n}\n\\toprule\nApts 1--20 & Apts 21--40 & Apts 41--60 \\\\\n\\midrule\n3603 & 303 & 1704 \\\\\n1701 & 3205 & 3206 \\\\\n203 & 603 & 4001 \\\\\n1107 & 2708 & 1308 \\\\\n601 & 2705 & 808 \\\\\n\\addlinespace\n701 & 4407 & 902 \\\\\n2001 & 4207 & 1008 \\\\\n1104 & 3303 & 3501 \\\\\n2001 & 4401 & 303 \\\\\n2702 & 405 & 2707 \\\\\n\\addlinespace\n2403 & 3107 & 4301 \\\\\n3203 & 2407 & 404 \\\\\n903 & 3708 & 3708 \\\\\n4008 & 703 & 4205 \\\\\n3301 & 4402 & 1107 \\\\\n\\addlinespace\n3805 & 3305 & 3405 \\\\\n4108 & 4102 & 2304 \\\\\n1406 & 4306 & 502 \\\\\n4002 & 3502 & 305 \\\\\n607 & 2607 & 4101 \\\\\n\\bottomrule\n\\end{tabular}\n\\caption{Random selection of apartments.}\n\\label{apt-list}\n\\end{center}\n\\end{table}\n`

\nIn a report, the table would look like this:

\n\n(I use the booktabs package to get this formatting.)

\nAs you can see from the header line, the list is laid out column-by-column, and there’s an order to it that needs to be preserved. My old way to get a one-item-per-line list from this would be to copy the LaTeX, paste it into a new BBEdit document, and do a few obvious edits to turn it into a tab-separated-values (TSV) file. I’d then open the TSV file in Numbers, copy/paste the second and third columns below the first, and save it back out, still as TSV.

\nThis was not an especially onerous process, but writing out the details of each step sure made it seem like one. Also, it made me acutely aware of how often I went through these steps and how often I caught myself making mistakes by doing things in the wrong order. Even though my Hints wiki is only for my own use, I couldn’t bring myself to memorialize such a clumsy procedure.^{1}

So I built a simple Keyboard Maestro macro that does the work for me. Here’s what it looks like:

\n\nI select the data in the LaTeX table, run the macro, and my clipboard then has the data in a single column that I can paste into a new BBEdit document. The key step is the shell script, which is a pipeline composed of three steps:

\n`sed -E '/^\\\\/d;s/ +\\& +/\\t/g;s/ *\\\\\\\\$//' |\\\nrs -c -T |\\\nrs 0 1\n`

\nThe `sed`

command deletes all lines that start with a backslash, changes the space-ampersand-space between each column into a tab, and deletes the trailing space and double backslash. For the example table above, it would output

`3603 303 1704\n1701 3205 3206\n203 603 4001\n1107 2708 1308\n601 2705 808\n701 4407 902\n2001 4207 1008\n1104 3303 3501\n2001 4401 303\n2702 405 2707\n2403 3107 4301\n3203 2407 404\n903 3708 3708\n4008 703 4205\n3301 4402 1107\n3805 3305 3405\n4108 4102 2304\n1406 4306 502\n4002 3502 305\n607 2607 4101\n`

\n(I’m showing it here with spaces between the columns to make it look nice. In the macro, it puts tabs between the columns.)

\nThe first `rs`

command transposes (`-T`

) the data from column-by-column to row-by-row. The `-c`

option tells `rs`

to use tabs as column separators in the input. This turns the 20×3 input into 3×20 output. The second `rs`

command takes that and reformats it into a single column.^{2} Keyboard Maestro then puts the single-column output onto the clipboard.

I once read something by one of the early Unix programmers (Dennis Ritchie, I think) who said that an advantage of programmers writing their own man pages was that a full description of the how the code worked (and, in the BUGS section, how it didn’t) was strong incentive to go back and make the code better. No one wanted others to see their ungraceful work. I don’t even want to see it myself.

\n\n

\n

\n

- \n
- \n
You might be wondering why I didn’t create a one-entry-per-line file right off the bat, at the same time I was creating the LaTeX table. It’s because I didn’t know when I was making these tables that I’d ever have a need for a single-column version. ↩

\n \n - \n
I feel certain there’s a way to do this with a single

\n`rs`

command, but I haven’t found it. I keep thinking`rc -c -t 0 1`

should work, but it doesn’t. ↩ \n

\n

[If there are equations in this post, you will see them rendered properly in the original article.]

"}, {"title": "Cleaning and graphing baseball data", "url": "https://leancrew.com/all-this/2023/02/cleaning-and-graphing-baseball-data/", "author": {"name": "Dr. Drang"}, "summary": "My code is simpler than it was a decade ago.", "date_published": "2023-02-28T18:16:44+00:00", "id": "https://leancrew.com/all-this/2023/02/cleaning-and-graphing-baseball-data/", "content_html": "If any of you followed the link in yesterday’s post to my old writeup from a decade ago, you noticed that the scripts I wrote back then used the `csv`

and `cPickle`

libraries to read, filter, and store the baseball game data. Now I find it more natural to use pandas for all of that, so I wrote all new code instead of trying to edit the old stuff. This post goes through the steps.

First, I downloaded all the game data from Retrosheet. Back in 2012, I had to download each year’s data individually; yesterday, I was able get every year in a single zip file called `gl1871_1922.zip`

. Unzipping this gave me a text file for each year, with names like `gl1871.txt`

. Also, there were files for playoff, World Series, and All Star games.

I wanted to look at only regular season games from the “modern era.” The modern era could have lots of definitions, but I chose to start with 1920, the year Babe Ruth began playing with the Yankees. So I deleted all the files except `gl1920.txt`

through `gl2022.txt`

.

These are CSV files without a header line. Each line represents a game, and there are 161 fields in each line; Retrosheet has an index that describes the fields. The files have DOS/Windows (CRLF) linebreaks, so I started off by converting them to Unix (LF) linebreaks with

\n`dos2unix gl*.txt\n`

\nYou can find the `dos2unix`

utility in many places on the internet. I installed it via Homebrew.

Strictly speaking, this conversion wasn’t necessary, as none of my subsequent manipulations required Unix linebreaks, but I did it anyway because it’s safer in general to have Unix-style files when you’re going work in a Unix environment.

\nI concatenated all files together:

\n`cat gl.*.txt > gl.csv\n`

\nThis gave me a 184 MB CSV file with no header line. There are command-line tools for manipulating and filtering such files, but I decided to use pandas because that’s what I’m most familiar with. I opened a Jupyter console session and ran these commands:

\n`python:\nimport pandas as pd\ndf = pd.read_csv('gl.csv', header=None, parse_dates=[0], usecols=[0, 3, 4, 6, 7, 9, 10, 18])\ndf.to_csv('games.csv', index=False)\n`

\nThe second line imported just the fields I wanted into the dataframe. In order, they were the date (0), visiting team (3), visiting team league (4), home team (6), home team league (7), visiting team score (9), home team score (10), and game duration in minutes (18).^{1} If you compare these number to the Retrosheet index, you’ll see that my numbers are one less than the index’s. That’s because pandas starts its numbering at zero instead of one.

With this done, I had a new CSV file called `games.csv`

that was a lot smaller, only 6.1 MB. This file had a header line of

`0,3,4,6,7,9,10,18\n`

\nwhich isn’t very useful. I opened it in BBEdit and edited that line to be

\n`Date,VTeam,VLeague,HTeam,HLeague,VScore,HScore,Time\n`

\nTo get the statistics I wanted to plot, I opened a new Jupyter console session and ran these commands:

\n`python:\n1: import pandas as pd\n2: from scipy.stats import scoreatpercentile\n3: df = pd.read_csv('games.csv', parse_dates=[0])\n4: df['Year'] = df.Date.dt.year\n5: for y in df.Year.unique():\n6: p25 = scoreatpercentile(df.Time[df.Year==y], 25)\n7: p50 = scoreatpercentile(df.Time[df.Year==y], 50)\n8: p75 = scoreatpercentile(df.Time[df.Year==y], 75)\n9: print(f'{y},{p25:.2f},{p50:.2f},{p75:.2f}')\n`

\nThe `dt.year`

property gets the year from the date. The `scoreatpercentile`

function returns the value in the given list at the given percentile. So the loop in Lines 5–9 goes through each year, determines the 25th, 50th, and 75th percentile values for all the game durations of that year, and prints out the result in this form:

`1920,99.00,109.00,120.00\n1921,100.00,111.00,125.00\n1922,100.00,110.50,124.00\n1923,102.00,112.00,125.00\n1924,101.00,112.00,125.00\n1925,102.00,114.00,127.00\n<etc>\n`

\nI copied the output, saved it to a file called `gametimes.csv`

, and added this header line:

`Year,Q1,Median,Q3\n`

\nThis is the data I wanted to plot.

\nI didn’t trust myself to write the plotting code interactively, so I created this `gametime-plot.py`

file:

`python:\n 1: #!/usr/bin/env python3\n 2: \n 3: import pandas as pd\n 4: import matplotlib.pyplot as plt\n 5: \n 6: # Import game time data\n 7: df = pd.read_csv('gametimes.csv')\n 8: \n 9: # Create the plot with a given size in inches\n10: fig, ax = plt.subplots(figsize=(6, 4))\n11: \n12: # Add the interquartile range and the median\n13: plt.fill_between(df.Year, df.Q1, df.Q3, alpha=.25, linewidth=0, color='#0066ff')\n14: ax.plot(df.Year, df.Median, '-', color='black', lw=2)\n15: \n16: # Gridlines and ticks\n17: ax.grid(linewidth=.5, axis='x', which='major', color='#bbbbbb', linestyle='-')\n18: ax.grid(linewidth=.5, axis='y', which='major', color='#bbbbbb', linestyle='-')\n19: ax.tick_params(which='both', width=.5)\n20: \n21: # Title and axis labels\n22: plt.title('Baseball game durations')\n23: plt.xlabel('Year')\n24: plt.ylabel('Minutes per game')\n25: \n26: # Save as PNG\n27: plt.savefig('20230227-Baseball game durations.png', format='png', dpi=200)\n28: \n`

\nI think this is pretty straightforward. Normally, my plotting scripts include commands for setting the tickmark placement, but in this case matplotlib’s defaults were just fine. The main work is done in Lines 13–14, where the `fill_between`

shades in the interquartile range and `plot`

runs a thick black line along the median.

I could’ve combined the code that calculates the percentiles with the code that does the plotting into a single script, but for a one-off like this, I usually prefer to take it one step at a time so I can check the intermediate results. Even if I had combined all the code here into a single script, it would have been simpler and easier to read than what I did back in 2012. That’s all due to pandas.

\n\n

\n

\n

- \n
- \n
Most of these fields were unnecessary for what I was plotting, but I kept them in case I wanted to do some other analyses. ↩

\n \n

\n

[If there are equations in this post, you will see them rendered properly in the original article.]

"}, {"title": "Revisiting baseball game durations", "url": "https://leancrew.com/all-this/2023/02/revisiting-baseball-game-durations/", "author": {"name": "Dr. Drang"}, "summary": "In which Paul Kafasis reminds me that baseball games last way too long.", "date_published": "2023-02-28T03:08:22+00:00", "id": "https://leancrew.com/all-this/2023/02/revisiting-baseball-game-durations/", "content_html": "This morning, I learned from Paul Kafasis that major league baseball will be going to a pitch clock, something they tried out in the minor leagues last year. The idea, of course, is to speed up our interminable national pastime.

\nThe story reminded me of a post I wrote about a decade ago in which I plotted the generally upward trend of baseball game durations over about 90 years. I grabbed a new set of data from Retrosheet and made this updated plot:

\n\nThe last decade has not been kind to fans’ tushies. The black line is the median game length, and the blue zone is the interquartile range, which runs from the 25th percentile to the 75th percentile. Games have gotten 10–12 minutes longer in the past 10 years.

\nPaul quotes a *Washington Post* article about the how the pitch clock

\n\nhas reduced the average [minor league] game time from 3 hours 4 minutes in 2021 to 2:36 in 2022

\n

My first thought was *How can minor league games possibly last as long as major league games? Isn’t that a violation of the Geneva Convention?* But then I remembered that minor league teams put on a lot of non-baseball show between innings. Myron Noodleman (RIP) ate up a lot of time.

I wish baseball luck. A half-hour drop would bring the game back down to the running time it had when I was a fan. I think that’s quite optimistic, but we’ll revisit this graph at the end of the season and see how it went.

\n

[If there are equations in this post, you will see them rendered properly in the original article.]

"}, {"title": "Modulo", "url": "https://leancrew.com/all-this/2023/02/modulo/", "author": {"name": "Dr. Drang"}, "summary": "Ambiguities in modular arithmetic.", "date_published": "2023-02-26T16:26:28+00:00", "id": "https://leancrew.com/all-this/2023/02/modulo/", "content_html": "I left something out of yesterday’s post. Another mathematical aspect of Underscore David Smith’s `unitSquareIntersectionPoint`

function is this set of lines at the top of the function:

`var normalizedDegree = angle.degrees\nwhile normalizedDegree > 360.0 {\n normalizedDegree -= 360.0\n}\nwhile normalizedDegree < 0.0 {\n normalizedDegree += 360.0\n}\n`

\nThere are lots of ways to normalize an angle. Here, UDS wants to take the input `angle`

and turn it into the equivalent angle between 0° and 360°. If `angle`

is already in that range, neither of the while loops is entered and `normalizedDegree`

remains equal to the input `angle`

.

You may recognize this as an example of modular arithmetic extended to include floating point numbers. SSteve did, and UDS included their code that makes that connection more explicit:

\n`var normalizedDegree = angle.degrees % 360;\nnormalizedDegree = normalizedDegree < 0 ? normalizedDegree + 360 : normalizedDegree;\n`

\nUnfortunately, SSteve wasn’t able use Swift’s `%`

operator alone, because it returns a value with a sign that matches the sign of the dividend. Thus the second line with the ternary operator was needed to ensure a positive result.

Sign ambiguity can be a nasty problem in modular arithmetic because different programming languages treat modular arithmetic differently. If I had to normalize the input angle in my `xy`

function to get it between 0° and 360°—which I don’t because the trig functions handle angles outside that range correctly—I’d be able to use

`python:\nnormalizedAngle = angle % 360\n`

\nin Python without further adjustment. Python uses a *floored division* definition of `%`

, so it always returns a value with the same sign as the divisor—in this case, +360.

When I read Underscore’s first post, I didn’t know whether Swift even had a modulo operator or how it worked. I went to the Wikipedia page looking for an answer and found three:

\n- \n
`%`

uses*truncated division*, which, as we’ve seen, returns an answer with the same sign as the dividend. Unfortunately, it’s limited to integers, so it probably shouldn’t be used in USD’s function (unless there’s some typecasting going on in SSteve’s code that I don’t understand). \n`truncatingRemainder(dividingBy:)`

, like`%`

, uses truncated division, but it can be used with floating point numbers. This is probably what should be used in SSteve’s code snippet. \n`remainder(dividingBy:)`

uses*rounded division*, where the sign depends on how close the dividend is to an integer multiple of the divisor. The sign can be either positive or negative, which I hate. \n

So none of these can be used without an adjustment.

\nModular arithmetic doesn’t come up often in my work, but when it does, I get nervous about the sign of the result. I’ve programmed in too many languages over the years to remember what type of division is used for the modulo operator in the language I’m currently writing in. Even the language I use the most, Python, has different ways of doing modulo arithmetic. Many different ways:

\n- \n
- The modulo operator,
`%`

, uses floored division. \n - The
`divmod`

function uses floored division. \n - The
`fmod`

function in the`math`

library uses truncated division. \n - The
`remainder`

function in the`math`

library uses rounded division. \n - The
`mod`

function in the NumPy library uses floored division. \n - The
`fmod`

function in the NumPy library uses truncated division. \n - The
`remainder`

function in the NumPy library uses floored division. \n

This is the sort of thing that makes you think Underscore was right in writing his own modular arithmetic code. It may take up more space, but at least the way it works is obvious.

\n

[If there are equations in this post, you will see them rendered properly in the original article.]

"}, {"title": "Underscore David Smith and linearization for large angles", "url": "https://leancrew.com/all-this/2023/02/underscore-david-smith-and-linearization-for-large-angles/", "author": {"name": "Dr. Drang"}, "summary": "A pretty good trig transformation without trig functions.", "date_published": "2023-02-25T19:46:28+00:00", "id": "https://leancrew.com/all-this/2023/02/underscore-david-smith-and-linearization-for-large-angles/", "content_html": "Underscore David Smith wrote a post earlier this week that I had trouble with. Part of the trouble was that the code he wrote was in Swift, but that was a minor problem, as UDS’s code was clear enough to understand even though I don’t know Swift. The bigger problem was how he managed to transform an angle into a pair of *xy* coordinates without using any trigonometric functions. Once I figured it out, I saw that it was an interesting example of linearization without using the typical small angle approximation. So I decided to write this post.

Unsurprisingly, other people had the same trouble with UDS’s code, and this morning he published an updated post with their “corrections” to his transformation function. In some sense, these truly are corrections in that they provide a more accurate transformation. But UDS’s function is pretty accurate, probably accurate enough for his purposes, so they’re not necessarily correcting a mistake. Still, they presented some interesting ways of performing the transformation, and I’ll comment on them at the end of this post.

\nLet’s summarize Underscore’s problem. He wants to use Swift’s `LinearGradient`

fill style for an arbitrary angle. Unfortunately, `LinearGradient`

won’t accept an angle as its argument. It will, however, take a `UnitPoint`

argument. A `UnitPoint`

is a pair of coordinates on the perimeter of the unit square shown below,

where I’ve also shown the angle that UDS wants to use as his input.

\nGetting \\(x\\) and \\(y\\) from \\(\\theta\\) is a fairly straightforward trig problem, although there’s some annoying bookkeeping necessary to deal with the corners. With the hope of avoiding that bookkeeping, I decided to look into conformal mapping from a disc onto a square. I didn’t have my copy of Churchill’s book handy, so I did some Googling and ran across this nice writeup by Chamberlain Fong. Fong shows several ways of mapping discs onto rectangles. Most important, he shows that conformal mapping (which can be very nasty mathematically) is unnecessary for our problem—we can use what he calls the Simple Stretching transformation.

\n\nAs you can see, the angles in the box are the same as those in the disc, which we’ll need to do what UDS wants. The transformation formulas Fong gives for this mapping (on p. 3 of his paper) work over the entire disc and square. We can specialize them for just the perimeter:

\n\n\\[\\xi = \\mathrm{sgn}(u), \\quad \\eta = \\mathrm{sgn}(u) \\frac{v}{u} \\qquad \\mathrm{for}\\, u^2 \\ge v^2\\]\n\n\\[\\xi = \\mathrm{sgn}(v)\\frac{u}{v}, \\quad \\eta = \\mathrm{sgn}(v) \\qquad \\mathrm{for}\\, u^2 \\lt v^2\\]\n\nHere, \\(\\mathrm{sgn}\\) is the *signum* or *sign* function—it returns \\(+1\\) if the argument is positive, \\(-1\\) if the argument is negative, and zero if the argument is zero. The coordinate systems are shown below:

So what we have now is a series of transformations. First, we go from \\(\\theta\\) to \\((u, v)\\):

\n\n\\[u = \\cos \\theta\\]\n\n\\[v = \\sin \\theta\\]\n\nThen we go from \\((u, v)\\) to \\((\\xi, \\eta)\\) using the formulas above. Finally, we go from \\((\\xi, \\eta)\\) to \\((x, y)\\):

\n\n\\[x = \\frac{1 + \\xi}{2}\\]\n\n\\[y = \\frac{1 - \\eta}{2}\\]\n\nThis last one is, I think, pretty obvious. We flip the vertical coordinate, then squeeze the 2×2 box into 1×1 and move its center from the origin to \\((1/2, 1/2)\\).

\nYou might be looking at this and thinking the bookkeeping I wanted to avoid would be easier than cascading three coordinate transformations like this. I guess it depends on what you’re used to. To me, putting one transformation after another after another is easier to remember than keeping track of which side of the box you’re on.

\nBefore turning this into code, there’s a simplification we can make to the transformation from \\((u, v)\\) to \\((\\xi, \\eta)\\). Note that

\n\n\\[\\mathrm{sgn}(u) = \\frac{u}{|u|}\\]\n\nSo the transformation equations can be changed to

\n\n\\[\\xi = \\frac{u}{|u|}, \\quad \\eta = \\frac{v}{|u|} \\qquad \\mathrm{for}\\, |u| \\ge |v|\\]\n\n\\[\\xi = \\frac{u}{|v|}, \\quad \\eta = \\frac{v}{|v|} \\qquad \\mathrm{for}\\, |u| \\lt |v|\\]\n\nwhere we’ve also noted that the ordering of the squares is the same as the ordering of the absolute values.

\nIn Python (because I don’t know Swift), the three-part transformation from \\(\\theta\\) to \\((x, y)\\) is

\n`python:\n 1: def xy(a):\n 2: # Coords on circle of unit radius with angle a (in degrees)\n 3: u = cosd(a)\n 4: v = sind(a)\n 5: \n 6: # Convert to coords on square from (-1, -1) to (1, 1)\n 7: if abs(u) >= abs(v):\n 8: xi = u/abs(u)\n 9: eta = v/abs(u)\n10: else:\n11: xi = u/abs(v)\n12: eta = v/abs(v)\n13: \n14: # Convert to coords on square from (0, 0) to (1, 1) with flipped\n15: # vertical axis\n16: return (1 + xi)/2, (1 - eta)/2\n`

\nThis is not necessarily the fastest implementation, or the shortest, but it’s pretty simple, and you can see each transformation in turn. Note that if you think of the \\((\\xi, \\eta)\\) box as being divided into 90° sectors like this,

\n\nthe green areas correspond to \\(|u| \\ge |v|\\) and the yellow areas correspond to \\(|u| \\lt |v|\\), which is how we split the `if`

conditional.

So how does this compare with what Underscore does? Here’s his code:

\n`func unitSquareIntersectionPoint(_ angle:Angle) -> UnitPoint {\nvar normalizedDegree = angle.degrees\nwhile normalizedDegree > 360.0 {\n normalizedDegree -= 360.0\n}\nwhile normalizedDegree < 0.0 {\n normalizedDegree += 360.0\n}\nif normalizedDegree < 45.0 || normalizedDegree >= 315 {\n //Right Edge, x = 1.0\n var degreeToConsider = normalizedDegree\n if degreeToConsider < 45.0 {\n degreeToConsider = normalizedDegree + 360.0\n //angle now between 315 & 405\n }\n let degreeProportion = (degreeToConsider - 315.0) / 90.0\n return UnitPoint(x: 1.0, y: 1.0 - degreeProportion)\n} else if normalizedDegree < 135.0 {\n //Top Edge, y = 0.0\n let degreeProportion = (normalizedDegree - 45.0) / 90.0\n return UnitPoint(x: 1.0 - degreeProportion, y: 0.0)\n} else if normalizedDegree < 225.0 {\n //left Edge, x = 0.0\n let degreeProportion = (normalizedDegree - 135) / 90.0\n return UnitPoint(x: 0.0, y: degreeProportion)\n} else if normalizedDegree < 315.0 {\n //Bottom Edge, y = 1.0\n let degreeProportion = (normalizedDegree - 225) / 90.0\n return UnitPoint(x: degreeProportion, y: 1.0)\n}\nreturn .zero\n}\n`

\n(Sorry, I don’t have syntax highlighting set up here for Swift.)

\nAs you can see, he splits his calculations into the four colored sectors—now on the unit square—but he has to treat all four sectors separately

\n\nThe main difference, though, is that he doesn’t use trig functions. For each side of the square, he sets one of the coordinates to 0 or 1, as appropriate, and sets the other as

\n\n\\[\\frac{\\phi - 90}{90}\\]\n\nwhere \\(\\phi\\) is the angle from the closest corner clockwise from \\(\\theta\\). In other words, to get the *y* coordinate along the right edge, \\(\\phi\\) is measured from the bottom right corner; to get the *x* coordinate along the top edge, \\(\\phi\\) is measured from the top right corner; and so on.

In my `xy`

function, the key calculations are

which give us \\(\\pm 1\\) for the first two and either \\(\\pm \\cot \\theta\\) or \\(\\pm \\tan \\theta\\) for the other two. My \\(\\pm 1\\) correspond to UDS’s 0 and 1.^{1} And since

and tangent repeats every 180°, my other calculations always return a result equivalent to the tangent of an angle between -45° and +45°.

\nThe upshot is that UDS’s function is approximating the tangent like this,

\n\nwhere the blue line is the tangent and the orange line is UDS’s approximation. As you can see, they’re fairly close to each other.

\nYou’ll notice I’ve plotted \\(\\theta\\) in radians. That’s because I also wanted to show the usual small angle approximation of tangent,

\n\n\\[\\tan \\theta \\approx \\theta\\]\n\nwhich I’ve plotted as the dotted black line. The small angle approximation works only when the angle is given in radians.

\nAs expected, the small angle linear approximation gets crappy as you get further from 0°. By tilting his approximating line, UDS gets perfect alignment at the corners and midsides of the box (the most important places) and decent alignment elseswhere.

\nHow decent? Let’s plot the angle associated with UDS’s output and compare it to the input angle:

\n\nThe symmetry (actually antisymmetry) means we can restrict ourselves to 0° to 45°. The dashed line is what the plot would look like if Underscore’s function wasn’t doing an approximation.

\nHere’s a plot of the error:

\n\nIt’s not a parabola, nor is it symmetric, but it’s close. The maximum error is 4.07° when the input angle is 23.52°

\nIs a maximum error of 4° acceptable? I suspect it is, especially because of where it occurs. People tend to be better at detecting angular errors near the horizontal, vertical, and 45° than they are at other angles.^{2} UDS’s function is good where it needs to be and less good where it doesn’t.

On the other hand, it could well be argued that this is not an especially time-critical calculation, and you might as well do the trig calculations^{3} instead of estimating through proportions. A lot depends on what sort of calculations you’re most comfortable with.

Which brings us to Underscore’s update and the “corrections” therein. Most of them include code that looks a lot like mine, which isn’t surprising, since they’re doing the same sort of trigonometry I am. Math is math. But there are some interesting variations.

\nDrewFitz does two thing I like:

\n- \n
- They start by changing the sign of the angle, which has the effect of flipping the vertical coordinate without flipping the horizontal coordinate because sine is antisymmetric and cosine is symmetric. \n
- They determine the maximum of \\(|u|\\) and \\(|v|\\), and uses that as his denominator. This eliminates the
`if/else`

clause (it’s still under the hood of the`max`

function, but you don’t see it). \n

Sometimes being clever like this gets you in trouble, but I think the comments in the code explain what’s going on quite well.

\nrobb’s solution is very close to mine and even closer to the equations in Fong’s paper. Like DrewFitz, robb tweaks the angle upon input to handle the flipping of the vertical axis, but in this code the angle is increased by 90° and the sine and cosine are reversed. That tweak was a little harder for me to understand, but it works.

\nLike Underscore, I find Rob Mayoff’s solution hard to understand. Mayoff uses a combination of `max`

, `min`

and `copysign`

(which takes the place of signum) to deal with the sector choice. It works, but I would have a hard time explaining why without going through numerical examples. Also, I think they’re using an opposite sign convention for the angle—UDS’s increases counterclockwise while Rob Mayoff’s seems to increase clockwise—but that’s a minor issue.

Of course, I think my solution is the “best” (followed closely—maybe edged out?—by DrewFitz’s) but that’s because it fits my way of thinking. As we learn more, our ways of thinking change and “best” changes, too. That’s why you’re constantly told to comment your code.

\n\n

\n

\n

- \n
- \n
Remember, at this stage my calculations are done on the 2×2 square. My \\(\\pm 1\\) get squeezed and shifted to 0 and 1 in the last line of the function. ↩

\n \n - \n
This is related to the idea of “banking to 45°” described by Cleveland in his

\n*The Elements of Graphing Data*. ↩ \n - \n
Are trig calculations still expensive compared to regular arithmetic? I was taught that they are, but that was long ago. ↩

\n \n

\n

[If there are equations in this post, you will see them rendered properly in the original article.]

"}], "home_page_url": "https://leancrew.com/all-this/", "version": "https://jsonfeed.org/version/1", "icon": "https://leancrew.com/all-this/resources/snowman-200.png"}