Tuesday, April 6, 2021

A 5-point framework for Python performance management

See A 5-Point Framework For Python Performance Management.

It seems straight-forward to me. Have goals. Measure your ability to meet them.

I don't see too many teams doing this, though. 

I could be wrong, but, I think performance is left to arguments and complaints, not solid engineering.

Tuesday, March 9, 2021

Recommended Books

I get asked about good books for beginners. Here's an example:

"What Python books do you recommend for novices so they can learn from beginner to advanced?"

For me, this is nearly impossible to answer.

"Beginner" is often undefined. I have to turn this around and ask what you already know about -- well-- everything. Computing. Programming. Languages. etc. etc. 

"Advanced" similarly is undefined. Most folks have areas they're interested in. Machine Learning. IoT. Security. Cloud Engineering. Graphics. Games. Sound. etc. et yet even more ceteras.

And -- even more fascinating to me -- where are you on that journey? What have you done so far?

I'm am (overly) sensitive to being a Personal Search Concierge, PSC™. 

I know people who (actually) cannot make Google work. Seriously. Utterly unable to use it. I believe that they are incapable of reframing their question with synonyms, but instead insist on typing a single thing into the search bar, and if the first promoted response in the list of advertisements doesn't literally answer their question, they email me.

This leads me to a stammering stupidity when asked about Python books.

Yes, I'm an author. Yes, I read other books. But no, I don't think I can answer your question.

One possible non-answer: https://realpython.com/best-python-books/. Start here.

What does "advanced" mean?

Most of the Python experts I know are experts at applying Python to a problem domain. In rare cases, the problem domain is Python itself, but even then, the focus often narrows to a specific package in the standard library, or an aspect of the run-time.

In the process of solving problems with Python, most people tend to learn a fair amount of the language. I work with folks who are fabulous problem-solvers but who'll sometimes be surprised by a Python feature that's outside their already broad experience. 

What's central here is that they're apply Python to something. The thing that seems to distinguish novices from experts is the pursuit of a solution to a problem, and learning Python as part of solving the problem.

It's essential, then, to have a problem about which one is passionate. Given a problem, and passion to solve that problem, expertise will grow.

So that's my other possible non-answer: find a problem you're passionate about and apply Python to solving it.

And yes, that's not a book. Books can help with understanding the problem or working out a solution in Python. Rarely does one book do both.

A good friend of mine's Python expertise comes from arranging the metadata in thousands of photographs on his computer. Apple's photos app has gone through numerous changes, and his photo library had become a jumble of obsolete folders, no longer supported by the current app. So they mastered Python and Apple's scripting tools, and Photos, and Mac OS X to arrange their photos. 

There are many Civic Tech organizations like the Code for America where you can confront large, complex problems, and build tech skills while helping solve a real-world problem. 

Another possible non-answer: https://www.govwebworks.com/2018/12/03/investigating-the-civic-tech-movement/

Everyone's journey is unique. 

Tuesday, February 16, 2021

The "The reader won't know what that means" Comment

I have to admit, it's very difficult to respond to this comment. I have to fight down the urge to say things like "The Reader is not an idiot," or "At some point, we have to assume they've seen a computer before."

AFAIK there's little or no useful data data on how much background people buying technical books have.

Subjectively, it seems like folks who leave the most comments -- the noisy few -- seem to balk at chapter 1 things, write a bad review or blog post and move on. 

It appears they expect to see chapter 1 reinforce their impressions of their own skills and present the things they -- individually -- don't yet know but tend to suspect. It needs to involve minimal surprises, and no forward references to later chapters.

I wish I could write like that.

But I can't write like that. I feel the need to assume that some readers actually do know some things and want the forward references so they can skip around.

I feel the need to assume readers have seen another programming language and have worked through a tutorial or two. 

I may be wrong. It may be true that most readers buy exactly one book and expect this to be an introduction to computers, languages, and computer science. If so, I need to learn how to cater to their needs.

I may be right. People may have background in at least one language and buy multiple books. If so, I don't think "the reader won't know what that means" is really a helpful comment. 

For some technical concepts, I like to rely on inductive definitions. Rather than fussy formalisms, I think it's helpful to use concrete examples. If there are weird edge cases or seeming contradictions, a fussy definition may be helpful, but not as a starting point. The comment "the reader won't know what that means" prior to the first example is a teeth-grinding thing. Yes. They won't know. That's why the rest of the paragraph is an example so the reader will know what it means.

The good news is the reviewers are reading very closely and splitting each hair. I am eternally grateful for the opportunity to grind my teeth in exasperation. 

Tuesday, January 26, 2021

The Awkwardness of Fundamental Definitions

A pragmatic description of a language (like Python) from axiomatic -- or really axiom-like -- foundations is exasperatingly hard. I don't think I have any answers, but I sure do have a lot of challenges. I think there's a path that involves a lot of "don't look down -- just edge along the face of this cliff a few more feet and it's going to be okay."

The actual language foundations, and a more useful conceptual foundation don't always match up.

My specific example is trying to use a subset of Python to get started with.

One of the reasons we write software is to see useful results. So you need output, i.e., the print() function. On the other end of this, you might want to accept inputs. But. But... You could finesse that by simply assigning literal values instead of reading something. 

This leads to the print() function and expressions as a kind of minimal language to see the essence of programming. (I know this isn't an original thought. I'm setting up the conflict.)

Do we explain functions in general when we're explaining this subset of Python? Or do we stick with arithmetic? 

print("Hello, world!")
print("355/113=", 355/113)

Maybe the F-string?  Assignment? Or do we really need assignment? (Spoiler alert, no.)

What about other functions? Python has a bunch of built-ins. The math and random modules let us build small games without much intellectual overhead.

And how much can we explain about functions? The whole mathematical notion of function as mapping from domain to range: does that count? Or nah? What about Python's flexible argument and parameter handling? 

Exceptions? Do we have to explain them? Or do we shrug at division by zero and kind of ignore it? If we talk about exceptions, do we have to talk about stack frames and tracebacks?

At the foundation, a language is variable bindings and function evaluation. But. Do you explain any of that? And if so, how?

The additional complication is multiple authors, technical reviews, editors, and everyone else involved. There's a complex web of varying opinions on what's foundational and how much explanation is required. It's new to one person, so it should be up font. But, a lot of up-front material is boring. Everything in programming involves a tricky nuance somewhere; how much of that nuance is relevant now and how much is later and how much is digressive?


To an extent, there's a trust relationship between writer and reader. It may help to build this if the writer can provide a "trust me on this," scenario that (eventually) results in a more complete explanation. The logical conclusion here is that there's no place in the book for "too advanced, buy my next book." Instead, every difficult and nuanced thing would need to (eventually) be explained. That seems impossible, though.

With editors, co-authors, and reviewers, the trust relationship is exactly wrong. Everything needs to be challenged and clarified.

tl;dr Writing is sometimes hard. And that's the expectation. It's a narrow fairway surrounded by shoals. 

Tuesday, January 5, 2021

Diagrams and UML notation

When I started in this business I was given a flowcharting template.

See https://americanhistory.si.edu/collections/object-groups/flowcharting-templates. I'm pretty sure I had one of these: https://americanhistory.si.edu/collections/search/object/nmah_690078.

Since then, things have changed a little.

I fondly recall using the Rational Rose (and an earlier tool that did Rumbaugh OMT diagrams) to create object models.

But these were expensive.

After much searching, I found ArgoUML. This was my go-to-diagrammer of choice for many years. It's available here: https://argouml-tigris-org.github.io.

Then, I wound up using yuml at https://yuml.me/diagram/scruffy/class/draw. This was very nice because there was a source text version of the diagram. It was a high-level code-like description that would lead to a handy picture you could include in documentation.


Recently, I spent time using draw.io. Start here https://draw.io. You have plain-text source version of the diagram that's Git-friendly. I liked that. It has a lot of UML features, which is very nice, also.

But now. 

I'm using plantUML, and I think it's pretty handy. https://plantuml.com. It's a big-old JAR file that converts text to a diagram. There's no GUI component to this. You describe the image in a source-code like way. Run it through the tool, and you get a picture you can paste into documentation. Like yuml, it has an easy-to-understand high-level text description. I strongly suspect I could walk a Python AST and emit plantUML source as an intermediate language from which pictures can be created.

The Python-Markup tool (https://python-markdown.github.io/extensions/) has a third-party PlantUML plug-in. PyCharm can leverage this to draw while you're editing in the markdown window.

The Fiddly Bits

It's a little fiddly to get all the parts organized properly, but, it really, really does work. You can write technical documentation, with pictures.

  • Add the Markdown tool to PyCharm.
  • In the preferences for the PyCharm Markdown tool, install and enable PlantUML.
  • You can usually use conda to install graphviz as well as installing the plantuml-markdown tools. You can manually run the markdown_py application to create the HTML copies of the .md files. 
  • Update your OS environment settings to set the GRAPHVIZ_DOT environment variable to name the conda virtual environment where graphviz was installed. For macOS and Linux users update the ~/.zshrc (or ~/.bashrc) file, depending on which shell is in use. Windows users have my heartfelt sympathy; maybe set the system environment variables.
  • You may also need to create a plantuml shell script that's on your PATH. I put it in /usr/local/bin

See https://github.com/mikitex70/plantuml-markdown for details on installation.

After all this fussing around, it worked delightfully. I'm a convert to PlantUML.

I suggest the following in each diagram.

skinparam monochrome true
skinparam handwritten false
skinparam shadowing false
hide class circle

You may want to set a more global configuration, but I sometimes want to change the handwritten parameter to true for "draft" diagrams, separate from final. 


You can integrate plantUML with PyCharm to draw pictures while you're editing in the markdown window.

You do have to trust plantUML to draw more-or-less what you want. There are limits, and if you don't like what plantUML is doing, switch to draw.io. If you are flexible, however, it's really, really good.