Over-Solving or Solving Problems You Don't Have

Sometimes we call them "Belt and Braces" solutions. As a former suspenders person who switched to belts, the idea of wearing both is a little like over-engineering. In the unlikely event of catastrophic failure of one system, your pants can still remain properly hoist. There's a weird, but defensible reason for that. Most over-engineering lacks a coherent reason. 

Sometimes we call them "Bells and Whistles." The solution has both bells and whistles for signaling. This is usually used in a derogatory sense of useless noisemakers, there for show only. Again, there's a really low-value and dumb, but defensible reason for this. 

While colorful, none of this is helpful for describing over-engineered software. Over-engineered software is often over-engineered for incoherent and indefensible reasons.

Over-engineering generally means trying to solve a problem that no user actually has. This leads to throwing around irrelevant features.

Concrete Example

I lived on a boat. I spent a fair amount of time fretting over navigation. 

There are two big questions: 

1. How far apart are two points, really. 
2. What's the real bearing from one point to another.

These are -- in some cases -- easy to answer.

If you have a printed, paper chart at the right scale, you can use dividers to compute a distance. It's actually a very easy task. Similarly, you can read the bearing off the chart directly. There's a trick to comparing a course to a nearby compass rose, but it's easy to learn and very accurate.

Of course, we don't want to painstakingly copy our notes from a paper chart to a spreadsheet to add them up to get total distance. And then fold in speed to get time and fuel consumption. These summary computations are a pain.

What you want is to do all of this with a computer.

1. Plot the points using a piece of software like OpenCPN (https://opencpn.org).
2. Extract the GPX file.
3. Compute distances, bearings, and durations to create a route.

"So?" you ask.

So. When I did this, I researched the math and got a grip on the haversine formula for doing the spherical geometry computation of distances between points on a sphere.

It's not too bad. The formula are big-ish. But manageable. See http://www.edwilliams.org/avform.htm#Dist for the great circle distance formula.

For airplanes and powered freighters crossing oceans, this is perfect.

For a small sailboat going from Annapolis, Maryland, to the Bahamas, this level of complexity is craziness. While accurate, it doesn't really solve the problem I have. 

I don't actually need that much accuracy. 

I need this much accuracy.

And no more. This is the essential hypotenuse distance using an R-factor to convert the difference between latitudes and the distance between longitudes into pretty-close distances. For nautical miles, R is 60×180÷π. 

This is simpler and it solves the problem I actually have. Up to about 232 miles, the answer is within 1 mile of correct. The error grows quickly. Double the distance and the error seems to jump to 8 miles. A 464 mile sailing journey (at 6 knots) takes 3 days. Wind, weather, tides and currents will introduce more error than the simplifying assumptions.

What's important is this can be put into a spreadsheet without pain. I don't need to write sophisticated Python apps to apply haversine to sequences of way-points. I can do a simpler hypotenuse computation on waypoints converted to radians.

Is there a lesson learned?

I think there is.

There's the haversine a super-general solution. It handles great-circle routes elegantly. 

But it doesn't solve my actual problem. And that makes it over-engineering.

My problem is what we call rhumb-line sailing. Over short-enough distances the world may as well be flat. Over slightly longer distances, errors in the ship's compass and speedometer make a hyper-accurate great circle route moot. 

(Even with several fancy GPS-based navigation computers, a prudent mariner has paper backups. The list of waypoints, estimated times and directions are essential when the boat's GPS reciever fails.)

I don't really need the sophistication (and the potential for bugs) with haversine. It doesn't solve a problem I actually have.

Navigation: Latitude, Longitude, Haversine, and all that

For a few years, I was a tech nomad. See Team Red Cruising for some stories of life on a sailboat. Warning: it's pretty dull.

As a tech nomad, I lived and died (literally) by my ability to navigate. Modern GPS devices make the dying part relatively unlikely. So, let's not oversell the danger aspect of this.

The prudent mariner plans a long voyage with a great deal of respect for the many things which can go wrong. One aspect of this is to create a "Float Plan". Read more about it here: http://floatplancentral.cgaux.org.

The idea is to create a summary of the voyage, provide that summary to trusted shore crew, and then check in periodically so that the shore crew can confirm that you're making progress safely. Failure to check in is an indicator of a problem, and action needs to be taken. We use a SPOT Messenger to check in at noon (and sometimes at waypoints.)

Creating a float plan involved an extract of the waypoints from our navigation software (GPS NavX). I would enrich the list of waypoints with estimated travel time between the points.  Folding in a departure time would lead to a schedule that could be tracked. I also include some navigation hints in the form of a bearing between waypoints so we know which way to steer to find the next point.

The travel time is the distance (in  nautical miles) coupled with an assumption about speed (5 knots.) It's a really simple thing. But the core haversine calculation is not a first-class part of any spreadsheet app. Because of the degrees-to-radians conversions required, and the common practice of annotating degrees with a lot of internal punctuation (38°54ʹ57″ 077°13ʹ36″), it becomes right awkward to simply implement this as a spreadsheet.

Some clever software has a good planning mode. The chartplotter on the boat can do a respectable job of estimating time between waypoints. But. It's not connected to a computer or the internet. So we can't upload that information in the form of a float plan. The idea of copying the data from the chart plotter to a spreadsheet is fraught with errors.

Navtools

Enter navtools. This is a library that I use to transform a route into a .csv with distances and bearings that I can use to create a useful float plan. I can add an estimated arrival time calculation so that a change to departure time creates the entire check-in schedule.

This isn't a sophisticated GUI app. It's just enough software to transform a GPS NavX extract file into a more useful form. The GUI was a spreadsheet (i.e., Numbers.) From this we created a PDF with the details.

Practically, we don't have good connectivity on the boat.  So we would create a number of alternative plans ("leave tomorrow", "leave the day after", "leave next Monday", etc.) we would go ashore, find a coffee shop, and email the various plans to ourselves. They could sit in our inbox, waiting for weather and tide to be favorable.

Then, when the weather and tides were finally aligned, we could forward the relevant details to our trusted shore crew. This was a quick spurt of cell phone connectivity to forward an email. It worked out well. When the scheduled departure time arrived, we'd coax Mr. Lehman to life, raise the anchor and away.

Literate Programming

This is an exercise in literate programming. The code that's executed and the HTML documentation are both derived from source ReStructured Text (RST) documents. The documentation for the navigation module includes the math along with the code that implements the math.

I have to say that I'm enthralled with the intimate connection between requirements, design, and implementation that literate programming embodies.

I'm excited to (finally) publish the thing to GitHub. See https://github.com/slott56/navtools.  I'm looking at some other projects that require the navtools module. What I wind up doing is copying and pasting the navigation calculation module into other projects. I had something like three separate copies on my laptop. It was time to fold all of the features together, delete the clones, and focus on one authoritative copy going forward.

I still have to remove some crufty old code. One step at a time. First, get all the tests to pass. Then expunge the old code. Then make progress on the other projects that leverage the navtools.navigation module.