Rules for Debugging

Here's the situation.

Someone wrote code. It didn't do what they assumed it would do.

They come to me for help.

Here are my rules for debugging. All of them.

1. Try something else.

---

I don't have any other or more clever advice. When I look at someone's broken code, I'm going to suggest the only thing I know. Try something else.

I can't magically make broken code work. Code's not like that. If it doesn't work, it's wrong, usually in a foundational way. Usually, the author made an assumption, followed through on that assumption, and was astonished it didn't work.

A consequence of this will be massive changes to the broken code. Foundational changes.

When you make an assumption, you make an "ass" out of "u" and "mption".

Debugging is a matter of finding and discarding assumptions. This can be hard. Some people think their assumptions are solid gold and write long angry blog posts about how a language or platform or library is "broken."

The rest of us try something different.

My personal technique is to cite evidence for everything I think I'm observing. Sometimes, I actually write it down -- on paper -- next to the computer. (Sometimes I use the Mac OS Notes app.) Which lines of code. Which library links. Sometimes, i'll include in the code as references to documentation pages.

Evidence is required to destroy assumptions. Destroying assumptions is the essence of debugging.

Sources of Assumptions

I'm often amazed at how many people don't read the "But on Windows..." parts of the Python documentation. Somehow, they're willing to assume -- without evidence -- that Windows is POSIX-compliant and behaves like Linux. When things don't follow their assumed behavior, and they're using Windows, it often seems like they've compounded a bunch of personal assumptions. I don't have too much patience at this point: the debugging is going to be hard.

I'm often amazed that someone can try to use multiprocessing.apply_async() without reading any of the example code. What I'm guessing is that assumptions trump research, making them solid gold, and not subject to questioning or locating evidence. In the case of multiprocessing, it's important to look at code which appears broken and compare it, line-by-line with example code that works.

Comparing broken code with relevant examples is -- in effect -- trying something else. The original didn't work. So... Go to something that does work and compare the two to identify the differences.

Hacker Laws

Pretty funny: https://github.com/dwmkerr/hacker-laws

PyCon 2019

There are some things I could say.

But.

You can come to understand it yourself, also.

Go here: https://www.youtube.com/channel/UCxs2IIVXaEHHA4BtTiWZ2mQ

Start with the keynotes.  https://www.youtube.com/channel/UCxs2IIVXaEHHA4BtTiWZ2mQ/search?query=keynote

For me, one of the top presentations was this https://www.youtube.com/watch?v=9G2s1TN9QQY
There are several closely related, but I found this very helpful.

Fiction Writers and Query Letters

See http://flstevens.itmaybeahack.com/writing-world-building-and/ for some back-story on F. L. Stevens and the need to write a *lot* of query letters to agents for fiction. (The non-fiction industry is entirely different; there are acquisition editors who look for technical contributors.)

There's a tiny possibility of a Query Manager Tool (of some kind) on a writer's desktop.

Inputs include:

• Template letter.
• A table of customizations per Agent. The intent is to allow more than simple name and pronoun changes. This includes Agent-specific content requirements. These vary, and can include the synopsis, first chapter, first 10 pages, first 50 pages. They're not email attachments; they have to be part of the main body of the email, so they're easy to prepare.
• Another table of variant pitches to plug into the template. There are a lot of common variations on this theme. Sizes vary from as few as 50 words to almost 300 words. Summaries of published works seem to have a median size of 140 words. A writer may have several (or several dozen) variants to try out.

This can't become a spam engine (agents don't like the idea of an impersonal letter.) 

Also. A stateful list of agents, queries, and responses is important. Some Agents don't necessarily respond to each query; they often offer a blanket apology along the lines of "if you haven't heard back in sixty days, consider it a rejection." So. You want to try again. Eventually. Without being rude. And if you requery, you have to send a different variant on the basic pitch.

Some Agents give a crisp "nope" and you can update your list to avoid requerying.

For new authors (like F. L. Stevens,) there's a kind of manual query tracking mess. It's not really horrible. But it is annoying. Keeping the database up-to-date with responses is about as hard as a tracking spreadsheet, so, there's little value to a lot of fancy Python software.

The csv, string.Temple, email and smtplib components of Python make this really easy to do.  While I think it would be fun to write, creating this would be work avoidance. 

PyLit-3 Maintenance, Love and Care

The PyLit tool dates from 2009. Here's a historical reference: http://wiki.c2.com/?PyLit

It was Python 2. It had some minor problems. I forked it and cleaned it up for Python 3.

Then I set it aside for a few years (six or so.)

Dusting it off. Rearranging things. The legacy Python 2 version -- it appears -- is gone forever.

The current thing available in PyPI doesn't even download and install on a modern Python because the metadata makes it look like it won't be compatible with a Python 3.7 world. So. That needs to be fixed. And while I'm at it...

- Add tox support for Python 3.5, 3.6, and 3.7 properly.
- Restructure the docs to use Github Pages from master/docs.
- Get the download squared away so pip install will work.
- Use pathlib.
- Start down the road toward type hinting. Which will likely exclude py35 support.

I may, as part of type hinting, be forced to make some more changes to the essential structure of the app(s).

For now, I simply need to get it to be pip installable.

Python and pathlib and Windows -- this problem has been solved -- and yet...

The Passive-Aggressive Programmer strikes again. A sad story of sadness.

I tell everyone to stop using os.path and use pathlib. Everyone. Here's the link: https://docs.python.org/3/library/pathlib.html

It's essential to realize the semantic richness of OS filesystem paths. They're not simply strings. They have a string representation, but there's quite a bit going on there that is not captured trivially by strings and string parsing.  "path/basename.extension" is more than just slashes and dots.

Windows users, of course, have a nightmarish problem. Actually many nightmarish problems, one of which is pathnames.

I tell Windows developers to use pathlib, it will make their life somewhat more bearable.

And Yet. The Passive-Aggressive Programmer insists on using Windows as if it doesn't have a problem with \ in path strings.

Line 110 has a literal r"C:\windows\is\xtreme\evil" Note the \x in the path. Without the r"" string, this literal raises a SyntaxError.

Line 50 had subprocess.run(r"C:\path\to\xectuable -option" + " " + options + " " + filename). Note the r"" string. Note the Linux-style -option, too. They're wrapping an open source app in a Python shell.

You're with me so far? They're Windows devs. They've managed to use raw strings in two places. Right?

But. They're Passive-Aggressive. They don't like PR comments of any kind. They'll "agree" to a change, but then... This...

Line 50 should change. It's needs to use a list.

The Passive Aggressive Programmer can't make a list work.

list_of_arguments = ["C:\path\to\xectuable -option"] + options_list + ["C:\windows\is\xtreme\evil"]

See what they did there? They didn't want to change from string to list. (We had to go over it more than once.) They wanted to leave it alone. Grudgingly, they agreed to change from string to list.

But.

SyntaxError. See? The list just doesn't work. Python is weird. It's an undocumented WAT.

[Yes, some of us know the r's vanished. They author couldn't figure that out, though.]

And the pathlib suggestion?

Since the strings are now a SyntaxError, they need me to fix that for them. I made them change to a list, therefore, I caused the SyntaxError. It would be a distraction to spend tome researching pathlib. "I need to Google and think about how to handle the Unicode error" was the response.

Using Path("C:/path/to/xecutable") to avoid any Window-ism of any kind is an impossible burden. Impossible. It requires them to Google. Instead, the SyntxError is all my fault.

The previous examples of the use of raw strings?  Don't know why they're not helpful, but I'm not the one who's struggling to implement a change.

Don't Solve My Problem.

Two and a half examples of "Don't solve the problem I described. Provide the implementation I dream about."

Can't Use Enums for Constants

I was asked to see this because sometimes there's just too much abstraction https://stackoverflow.com/questions/2668355/how-much-abstraction-is-too-much

The accepted answer links to some useful design principles. Read the answer. It's useful.

The question objects to abstract superclasses without much (or any) actual implementation.  I've seen folks toy around with frameworks where there are classes that introduce a name, but little else. So I understand the complaint. I once tried to use Python classes as surrogates for Java interfaces. It was a bad idea. And irrelevant to solving the underlying problem.

The problem that lead to the "yet another abstraction" complaint, however, was not related to a design with empty layers of framework abstractions. It was not related to classes used to define an interface-like feature in Python. It wasn't related to *anything* in the Stack Overflow question or answer.

The "yet another abstraction" complaint was based on Python not having constants. Seriously. How did we get here? Right. They don't want a solution. They want to complain.

I lift this situation up to folks who are trapped in conversations where things devolve into bizarro-world like "Yet Another Abstraction is bad" when we're not talking about abstractions. The solution is simple, but, it's not what they wanted so, it's labeled as bad in some way.

The solution is bad because it's unexpected. Consequently peripheral, tangential, weird-ass nonsense will show up in trying to avoid an unexpected solution.

Can't Assign Numbers

There's an API to load some data.  They have 100's of clients happily loading data. In some cases, the clients must assign numbers in addition to names; it's a disambiguation thing. Most of the time, the name is good. In a few cases, (name, number) is a two-part key because they have multiple instances with the same name.

We're good here. The data structure's key can be (name, number) and the default number is zero. Works for almost everyone.

Almost everyone. Exception they have one client who cannot count or enumerate their data.

Really.

The client can't even pre-process the data to add numbers because reasons.

The stated reason is "the data originates off-line and the numbers might be inconsistent." The key needs a number. It doesn't need to consistent. The point is asking the client to own the identity -- a name and a number.

The solution seemed easy. Assign a number. If your data comes from a spread-sheet, use the row number. The =row() function works. Use that.  If your data doesn't come from a spread-sheet write a tiny utility to laminate a number into the data. This doesn't seem hard. And then the client owns the object identity.

Nope.

Can't do it. The web service will have to assign the number for them.

It's not a difficult feature to add. It's a complicated, stateful default value. This will turn into trouble tickets in the future when the numbers are unacceptable because they change with each load or something even more obscure than that.

Can't Fork a Repo

This isn't recent, and I may not have the details right. But.

The team had evolved an approach where they had several different pieces of software spread among multiple branches in a single Git repository.

This was weird. And they were -- of course -- about to start having CI/CD problems as they moved away from manual builds into a world of git commit hooks and relatively fixed CI/CD pipelines.

And they were really unhappy. They liked having multiple branches in a single repo. The idea of forking this into separate repos was unacceptable. Unworkable. Breaks everything. (Breaks everything they had. Everything they needed to replace. Or so they claimed.)

They had some vision of having the CI/CD jobs all reworked to move beyond the common dev/master world into their uniquely odd world of lots of parallel branches, each it's own private "master". But all in one repo.

They seemed to have locked into a strange world view, and weren't happy discarding it. The circular discussions of how multiple repos would break something something in their something was more examples of tangential, irrelevant discussion to cloak empty whining.

tl;dr

I think there are people who don't really want a "solution." They want something else.

There are people who have a vision: How Things Should Be (HTSB™,) They seem to be utterly unwilling to consider something that is not literally their (narrow) vision of HTSB.

It's very much as "Don't confuse me with facts, my mind is made up" situation. It's exasperating to me because of the irrelevant side-channel discussions they use to avoid confronting (or even stating) the actual problem.