Tuesday, April 20, 2021

OpenMarine and Signal-K

 I heard about these less than a week ago.

This is very interesting. Very interesting.

I have a partially complete IoT anchor alarm.

The idea of leveraging the boat's other devices through a Signal-K interface is appealing.

The problem is, I don't want the power draw.

I think I want to continue on the path of a small, thrifty, stand-alone device that can be used when the rest of the boat's systems are mostly shut down.

However. Seeing these projects causes me to rethink my use of Arduino and C++. While the Arduino is the thriftiest possible device -- the power consumption is negligible -- I think that a small upgrade to a Python-based device might make the software a tiny bit simpler. 

An Arduino Uno, specifically, is just barely capable of the UX I was hoping to build. The two-line LCD with a "mark" push-button, an "anchor circle" knob, and a "display page" button is right at the limit; I'm using analog inputs instead of digital for the buttons.

A Raspberry Pi can support more sophisticated displays, at some cost in power consumption. An e-ink display might be a better choice than the two-line LCD because -- well -- anchoring details change slowly.  Once you've drifted too far (or have a consistent COG away from the marked point with a steadily growing distance) then the alarm sounds and the display is more-or-less irrelevant. You're going to get up, and eyeball the situation to see what's going on. Wether or not the display updates doesn't matter much. 

We haven't drifted very often, so I don't have too much data.

  1. Once we slid to a new position. It was a stormy, blowy day. We eased out more scope. And we never moved again. We stood on deck, taking visual bearings. An e-ink display of the details wasn't what we depended on.
  2. We used to rely on an iPhone app to monitor our position. (We've switched to using SafeAnchor, we used to use an older app, no longer available.) We were moving slowly, but steadily. It was during a hurricane, we weren't surprised. We started the engine, raised the anchor and motored to a new place to reset. Again, we weren't using the display on the phone, we were looking at Pungo creek.
  3. And once we were not on the boat when she moved. That would have been awkward for our neighbors. So. We'd need to have a "reset the anchor alarm" switch in the cockpit. This would mark a new position. Fatty Goodlander's advice is to leave a big sign with a string showing them where it is.

A full 1.0 knot of speed is 1.7 feet per second. We often have 50 feet of line, meaning any movement under 100' is likely ordinary boat motion. That means 30 seconds until we're suspicious of a problem, and a full minute before we must sound the alarm. (In the middle, a constant COG and increasing distance is a leading indicator of trouble; alarm chirps might be helpful.)

As intermediate data gathering format, the Signal-K data stream is appealing. It steps away from the NMEA GPS talker messages. It's heavy-going for an Arduino Uno. But. Might work out well on something a little bigger.

Tuesday, April 6, 2021

A 5-point framework for Python performance management

See A 5-Point Framework For Python Performance Management.

It seems straight-forward to me. Have goals. Measure your ability to meet them.

I don't see too many teams doing this, though. 

I could be wrong, but, I think performance is left to arguments and complaints, not solid engineering.

Tuesday, March 9, 2021

Recommended Books

I get asked about good books for beginners. Here's an example:

"What Python books do you recommend for novices so they can learn from beginner to advanced?"

For me, this is nearly impossible to answer.

"Beginner" is often undefined. I have to turn this around and ask what you already know about -- well-- everything. Computing. Programming. Languages. etc. etc. 

"Advanced" similarly is undefined. Most folks have areas they're interested in. Machine Learning. IoT. Security. Cloud Engineering. Graphics. Games. Sound. etc. et yet even more ceteras.

And -- even more fascinating to me -- where are you on that journey? What have you done so far?

I'm am (overly) sensitive to being a Personal Search Concierge, PSC™. 

I know people who (actually) cannot make Google work. Seriously. Utterly unable to use it. I believe that they are incapable of reframing their question with synonyms, but instead insist on typing a single thing into the search bar, and if the first promoted response in the list of advertisements doesn't literally answer their question, they email me.

This leads me to a stammering stupidity when asked about Python books.

Yes, I'm an author. Yes, I read other books. But no, I don't think I can answer your question.

One possible non-answer: https://realpython.com/best-python-books/. Start here.

What does "advanced" mean?

Most of the Python experts I know are experts at applying Python to a problem domain. In rare cases, the problem domain is Python itself, but even then, the focus often narrows to a specific package in the standard library, or an aspect of the run-time.

In the process of solving problems with Python, most people tend to learn a fair amount of the language. I work with folks who are fabulous problem-solvers but who'll sometimes be surprised by a Python feature that's outside their already broad experience. 

What's central here is that they're apply Python to something. The thing that seems to distinguish novices from experts is the pursuit of a solution to a problem, and learning Python as part of solving the problem.

It's essential, then, to have a problem about which one is passionate. Given a problem, and passion to solve that problem, expertise will grow.

So that's my other possible non-answer: find a problem you're passionate about and apply Python to solving it.

And yes, that's not a book. Books can help with understanding the problem or working out a solution in Python. Rarely does one book do both.

A good friend of mine's Python expertise comes from arranging the metadata in thousands of photographs on his computer. Apple's photos app has gone through numerous changes, and his photo library had become a jumble of obsolete folders, no longer supported by the current app. So they mastered Python and Apple's scripting tools, and Photos, and Mac OS X to arrange their photos. 

There are many Civic Tech organizations like the Code for America where you can confront large, complex problems, and build tech skills while helping solve a real-world problem. 

Another possible non-answer: https://www.govwebworks.com/2018/12/03/investigating-the-civic-tech-movement/

Everyone's journey is unique. 

Tuesday, February 16, 2021

The "The reader won't know what that means" Comment

I have to admit, it's very difficult to respond to this comment. I have to fight down the urge to say things like "The Reader is not an idiot," or "At some point, we have to assume they've seen a computer before."

AFAIK there's little or no useful data data on how much background people buying technical books have.

Subjectively, it seems like folks who leave the most comments -- the noisy few -- seem to balk at chapter 1 things, write a bad review or blog post and move on. 

It appears they expect to see chapter 1 reinforce their impressions of their own skills and present the things they -- individually -- don't yet know but tend to suspect. It needs to involve minimal surprises, and no forward references to later chapters.

I wish I could write like that.

But I can't write like that. I feel the need to assume that some readers actually do know some things and want the forward references so they can skip around.

I feel the need to assume readers have seen another programming language and have worked through a tutorial or two. 

I may be wrong. It may be true that most readers buy exactly one book and expect this to be an introduction to computers, languages, and computer science. If so, I need to learn how to cater to their needs.

I may be right. People may have background in at least one language and buy multiple books. If so, I don't think "the reader won't know what that means" is really a helpful comment. 

For some technical concepts, I like to rely on inductive definitions. Rather than fussy formalisms, I think it's helpful to use concrete examples. If there are weird edge cases or seeming contradictions, a fussy definition may be helpful, but not as a starting point. The comment "the reader won't know what that means" prior to the first example is a teeth-grinding thing. Yes. They won't know. That's why the rest of the paragraph is an example so the reader will know what it means.

The good news is the reviewers are reading very closely and splitting each hair. I am eternally grateful for the opportunity to grind my teeth in exasperation. 

Tuesday, January 26, 2021

The Awkwardness of Fundamental Definitions

A pragmatic description of a language (like Python) from axiomatic -- or really axiom-like -- foundations is exasperatingly hard. I don't think I have any answers, but I sure do have a lot of challenges. I think there's a path that involves a lot of "don't look down -- just edge along the face of this cliff a few more feet and it's going to be okay."

The actual language foundations, and a more useful conceptual foundation don't always match up.

My specific example is trying to use a subset of Python to get started with.

One of the reasons we write software is to see useful results. So you need output, i.e., the print() function. On the other end of this, you might want to accept inputs. But. But... You could finesse that by simply assigning literal values instead of reading something. 

This leads to the print() function and expressions as a kind of minimal language to see the essence of programming. (I know this isn't an original thought. I'm setting up the conflict.)

Do we explain functions in general when we're explaining this subset of Python? Or do we stick with arithmetic? 

print("Hello, world!")
print("355/113=", 355/113)

Maybe the F-string?  Assignment? Or do we really need assignment? (Spoiler alert, no.)

What about other functions? Python has a bunch of built-ins. The math and random modules let us build small games without much intellectual overhead.

And how much can we explain about functions? The whole mathematical notion of function as mapping from domain to range: does that count? Or nah? What about Python's flexible argument and parameter handling? 

Exceptions? Do we have to explain them? Or do we shrug at division by zero and kind of ignore it? If we talk about exceptions, do we have to talk about stack frames and tracebacks?

At the foundation, a language is variable bindings and function evaluation. But. Do you explain any of that? And if so, how?

The additional complication is multiple authors, technical reviews, editors, and everyone else involved. There's a complex web of varying opinions on what's foundational and how much explanation is required. It's new to one person, so it should be up font. But, a lot of up-front material is boring. Everything in programming involves a tricky nuance somewhere; how much of that nuance is relevant now and how much is later and how much is digressive?


To an extent, there's a trust relationship between writer and reader. It may help to build this if the writer can provide a "trust me on this," scenario that (eventually) results in a more complete explanation. The logical conclusion here is that there's no place in the book for "too advanced, buy my next book." Instead, every difficult and nuanced thing would need to (eventually) be explained. That seems impossible, though.

With editors, co-authors, and reviewers, the trust relationship is exactly wrong. Everything needs to be challenged and clarified.

tl;dr Writing is sometimes hard. And that's the expectation. It's a narrow fairway surrounded by shoals. 

Tuesday, January 5, 2021

Diagrams and UML notation

When I started in this business I was given a flowcharting template.

See https://americanhistory.si.edu/collections/object-groups/flowcharting-templates. I'm pretty sure I had one of these: https://americanhistory.si.edu/collections/search/object/nmah_690078.

Since then, things have changed a little.

I fondly recall using the Rational Rose (and an earlier tool that did Rumbaugh OMT diagrams) to create object models.

But these were expensive.

After much searching, I found ArgoUML. This was my go-to-diagrammer of choice for many years. It's available here: https://argouml-tigris-org.github.io.

Then, I wound up using yuml at https://yuml.me/diagram/scruffy/class/draw. This was very nice because there was a source text version of the diagram. It was a high-level code-like description that would lead to a handy picture you could include in documentation.


Recently, I spent time using draw.io. Start here https://draw.io. You have plain-text source version of the diagram that's Git-friendly. I liked that. It has a lot of UML features, which is very nice, also.

But now. 

I'm using plantUML, and I think it's pretty handy. https://plantuml.com. It's a big-old JAR file that converts text to a diagram. There's no GUI component to this. You describe the image in a source-code like way. Run it through the tool, and you get a picture you can paste into documentation. Like yuml, it has an easy-to-understand high-level text description. I strongly suspect I could walk a Python AST and emit plantUML source as an intermediate language from which pictures can be created.

The Python-Markup tool (https://python-markdown.github.io/extensions/) has a third-party PlantUML plug-in. PyCharm can leverage this to draw while you're editing in the markdown window.

The Fiddly Bits

It's a little fiddly to get all the parts organized properly, but, it really, really does work. You can write technical documentation, with pictures.

  • Add the Markdown tool to PyCharm.
  • In the preferences for the PyCharm Markdown tool, install and enable PlantUML.
  • You can usually use conda to install graphviz as well as installing the plantuml-markdown tools. You can manually run the markdown_py application to create the HTML copies of the .md files. 
  • Update your OS environment settings to set the GRAPHVIZ_DOT environment variable to name the conda virtual environment where graphviz was installed. For macOS and Linux users update the ~/.zshrc (or ~/.bashrc) file, depending on which shell is in use. Windows users have my heartfelt sympathy; maybe set the system environment variables.
  • You may also need to create a plantuml shell script that's on your PATH. I put it in /usr/local/bin

See https://github.com/mikitex70/plantuml-markdown for details on installation.

After all this fussing around, it worked delightfully. I'm a convert to PlantUML.

I suggest the following in each diagram.

skinparam monochrome true
skinparam handwritten false
skinparam shadowing false
hide class circle

You may want to set a more global configuration, but I sometimes want to change the handwritten parameter to true for "draft" diagrams, separate from final. 


You can integrate plantUML with PyCharm to draw pictures while you're editing in the markdown window.

You do have to trust plantUML to draw more-or-less what you want. There are limits, and if you don't like what plantUML is doing, switch to draw.io. If you are flexible, however, it's really, really good.

Thursday, December 24, 2020

How to avoid writing a clickbait headline. (Click for details.)

It's hard to write shameless promotional material.

I already wrote the books, isn't that bold enough?

It isn't, though. 

Packt’s Head of Product, Oli Huggins, said: “We believe in helping to serve and support the global developer community. By selling our eBooks and Videos for $5, we hope to unlock exciting new opportunities for developers who, in other situations, wouldn’t have access to our products. A key part of our mission is to unlock new opportunities for developers, help them discover new technologies, and help put software to work in new ways."

Packt has curated some of our best titles together for the Python Programmers community:

I'm delighted to be part of the promotion. It's Christmas Eve. Even if you don't celebrate this specific holiday, the passing of the winter solstice is a time of renewal.